Deletions are marked like this. | Additions are marked like this. |
Line 6: | Line 6: |
See [:Nntp.club.cc.cmu.edu:nntp] | See [:Infrastructure/Nntp.club.cc.cmu.edu:nntp] |
Line 11: | Line 11: |
See [:Selenium.club.cc.cmu.edu:selenium]. | See [:Infrastructure/Selenium.club.cc.cmu.edu:selenium] |
Line 16: | Line 16: |
See [:Indium.club.cc.cmu.edu:indium] | See [:Infrastructure/Indium.club.cc.cmu.edu:indium] |
Line 39: | Line 39: |
Posts to CMU BBoards are spooled directly to the reader machines. To set this up, put appropriate key-value pairs in the hashes at the tops of /root/bbfeed.pl /root/bbspooler.pl on [:Nntp.club.cc.cmu.edu:nntp]. | Posts to CMU BBoards are spooled directly to the reader machines. To set this up, put appropriate key-value pairs in the hashes at the tops of /root/bbfeed.pl /root/bbspooler.pl on [:Infrastructure/Nntp.club.cc.cmu.edu:nntp]. |
1. Machines
1.1. NNTP
Our transit/spool machine.
See [:Infrastructure/Nntp.club.cc.cmu.edu:nntp]
1.2. Selenium
Our old, deprecated for several years, news reader machine.
See [:Infrastructure/Selenium.club.cc.cmu.edu:selenium]
1.3. Indium
Replacement for selenium. Work in progress.
See [:Infrastructure/Indium.club.cc.cmu.edu:indium]
2. Software
We run [http://www.openusenet.org/diablo/ Diablo USENET Software] on both the transit/spool machine and the reader machine.
Apparently we had bad experiences with [http://www.isc.org/index.pl?/sw/inn/ INN] in the past.
2.1. Patches
We have a patch against Diablo that allows us to stow the executables into /usr/local/bin and keep configuration files and spool directories in /var/diablo (instead of having everything in /news).
See /afs/club.cc.cmu.edu/system/src/local/diablo/005/paths.patch
Diablo attempts to use AIO by default on Linux. This breaks dreaderd, so make sure USE_AIO is #define'd to 0 in dreaderd/defs.h.
See /afs/club.cc.cmu.edu/system/src/local/diablo/005/aio.patch
3. Setup
3.1. Transit/Spool
At the present, I (Keith) don't know all that much about the specifics.
3.1.1. BBFeed/BBSpool
Posts to CMU BBoards are spooled directly to the reader machines. To set this up, put appropriate key-value pairs in the hashes at the tops of /root/bbfeed.pl /root/bbspooler.pl on [:Infrastructure/Nntp.club.cc.cmu.edu:nntp].
3.2. Reader
On the reader machines, we run diablo to aggregate the news and bboard feeds from the transit/spool machine, and keep a local article cache. We also run dreaderd, so users can read from, list, and post to news groups.
3.2.1. Configuration Files
- dexpire.ctl
- Determines how overview information (article headers) is expired.
- We keep more headers for local articles (cmu.*, assocs.*, graffiti.*, org.*, and official.*) than for other articles.
- diablo.config
- Configuration file used by all diablo programs.
- Important things: paths, logging, contact email addresses, connection and process limits, data formats.
- dnewsfeeds
- Defines the newsfeeds consumed and produced by the diablo process.
- dreader.access
- Defines access controls for connection made to the dreaderd process.
- Determine whether access is local (I used system:campusnet, and some aarons hosts from selenium's file). Allow local posting, and local reading of bboards, and alt.binaries*. Allow limited non-local reading.
- dserver.hosts
- Defines the transit servers backending the dreader process.
- This should specify the diablo process running on the same machine as the trasit backend. Make sure the port number is correct.
- dspool.ctl
- Determines how articles are filed into spools.
- Local articles are given their own spool so they are retained as long as possible. Text articles in alt.* are put in a separate spool. Next, text articles in alt.* are placed in a third spool. Finally, binary articles are placed in a fourth spool, so they do not adversely affect the retention of text articles.
- moderators
- Sets email addresses for group moderators.
Sets moderator to bogus-[groupname]@club.cc.cmu.edu for bboards. Sets moderator to [groupname]@moderators.isc.org otherwise.
3.2.2. Cron Jobs
What do the sample adm scripts provided with diablo do?
- biweekly.atrim
- Trims history file.
- Trims active file.
- daily.atrim
- Rotates log files.
- daily.reader
- Removes cached articles (not necessary).
- Expires overview information (light cleaning).
- hourly.expire
- Expires articles from the spool.
- weekly.reader
- Expires overview information (deep cleaning)
What modifications are necessary to make them work in our environment?
- all
- Look for executables in /usr/local/bin rather than ~news/dbin.
- daily.atrim
- Don't attempt to rotate logs that don't exist; maybe an issue with our /bin/csh. The sample script croaks because parts of the list of log files in curly braces don't exist.
- weekly.reader
- Uncomment the dexpireover line with -O180, since it works fine for us.
There are also cronjobs for root for the bofh script dealing with lusers.
- Remove old badusers.2* files from /var/diablo/bofh
find ~news/bofh/badclients.2* -cmin +5760 | xargs rm -f
- Find rejection messages in the diablo general log, and set iptables rules appropriately if there were too many
cd ~news/bofh && ./findbadclients.sh && cat ./badclients.* | awk -- '{if($1 > 1000) print $2}' | ./bofh.sh