Tips for avoiding SPAM (c) Scott M. Baker, 1997 ------------------------------------------------------------------------------- What is SPAM? The term SPAM is generally used to refer to off-topic commercial messages which are posted to newsgroups. These messages are usually advertisements for some form of 'Pay' service, usually of an adult nature. Typically, advertisers post SPAM messages multiple times and to multiple newsgroups. While it's hard to blindly condemn commercialism, some of this has really gotten out of hand. Some SPAM advertisers post hundreds of bulky off-topic pictures to newsgroups where they clearly do not belong. Some of them even post adult material to non-adult groups. This kind of recklessness and complete disregard for Internet users has had a dreadful impact on usenet newsgroups. There are ways to get by in a SPAM-filled world though. Through the use of SBNews and the techniques I describe below, I have found it easy to eliminate nearly all of the SPAM out there. My experiements show that by using the techniques below, you can easilly reduce the amount of SPAM you download to less than 1%. That's less than one SPAM download for every hundred good downloads. How can SBNews be set up to avoid SPAM? I will present a list of techniques below, most of which that I use myself on my own configuration: 1. "Maximum XRef Limit". The most effective technique, which also requires very little manual effort is to use the Maximum XRef Limit. This setting is located under the pull-down menu "Configure:Preferences". Typically, a SPAM advertiser posts to many newsgroups at a time. When an article is posted to many newsgroups, the news server automatically generates an XRef header in the message which contains a list of which groups the article is posted to. To make a long story short, you can tell SBNews to ignore a message that is posted to too many newsgroups. I usually find a setting of 8 to be appropriate, although these days an even lower setting (5, or even 2 or 3) may be necessary. Note: You may also want to check the "preload xref hdrs" box, described in step 3. 2. "Lockout XRef". You can find this option under the pull-down menu "Configure:Lockout:Xref Group Name". As described above, the XRef line contains a listing of newsgroups to which the message is posted to. There may be some newsgroups which contains subject matter that you are absolutely certain that you don't want to receive (perhaps material that is offensive to you). If a message is cross-posted to one of these undesirable groups, then you probably don't want it. You can enter the names of groups that are undesirable into the Lockout XRef dialog box, and SBNews will ignore any message which is cross-posted to the groups that you list. Note: You may also want to check the "preload xref hdrs" box described in step 3. 3. "Preload XRef Headers". This option is located under the pull-down menu "Configure:Preferences". By default, SBNews does not pre-download the header lines containing xref information. This is done to make the header download process faster for people who do not use the XRef options described above. Thus, SBNews doesn't know that a message should be ignored until the message has begun downloading. To abort the message, a disconnect/reconnect cycle is required, which is a bit inefficient. However, if you make heavy use of the xref limit/lockouts (described above), then you will probably want to pre-download the xref headers. Checking this option will cause SBNews to download the headers ahead of time, so that SBNews can decide whether a message should be ignored without having to begin downloading the message. I highly recommend this option if you use steps 1 & 2 above. 4. "Minimum Message Lines". Located under the pull-down menu "Configure:Preferences". There are a lot of SPAM advertisers that post short text messages into the binary groups. Although these messages are short, they still do waste some of your download time parsing through them. I recommend setting the minimum lines setting to "100". If there's anything with less than 100 lines, then it probably isn't worth having anyway. 5. "Lockout Poster". Located under the pull-down menu "Configure:Lockout:Poster", "Current:Lockout", and "Previous:Lockout". The poster is the name of the person who sent the message. SPAM advertisers usually invent bogus names, but they do tend to reuse the names frequently. Sometimes they will consistently use the same domain part of a name. When you lockout a poster, SBNews will ignore any messages posted by that name. You can manually enter lockouts using the "Configure:Lockout:Poster" dialog box, or you can automatically lockout a person who posted an unsuitable image in the previous/current thumbnails by using the "Current:Lockout" and "Previous:Lockout" options. You don't have to specify a full name, you can specify just part of the name. For example, if you locked out "@bogusspam.com", that would match "tom@bogusspam.com", "joe@bogusspam.com", etc. 6. "Lockout Subject". Located under the pull-down menu "Configure:Lockout:Subject". SPAM advertisers have to mention their service somewhere, and they usually like to do it in the subject field of the message. Sometimes they'll stick an http address in there. As with the Lockout Poster option, you don't have to type a full subject line here, you can just enter the part of the subject that identifies the SPAM advertiser. For example, "phone sex" would eliminate any message subject that had that phrase in it. 7. Add "free" to Lockout Subject. Located under the pull-down menu "Configure:Lockout:Subject". Every commercial SPAM advertiser wants you to think that their pay service is free, so the majority of them put the word "free" in the subject line. Locking out free will get rid of a considerable amount of junk. There is a risk with this option -- you will lose some relevant data from people who actually are posting something that is free, but this is the exception rather than the rule. [Recommended cautiously due to possible loss of relevant material] 8. Add "http://" to Lockout Subject. Located under the pull-down menu "Configure:Lockout:Subject". Commercial SPAM advertisers like to put their http address in the subject line so that you know how to get to their service. As with the previous step, you will probably miss out on some actual relevant material from people who are posting http addresses to their website. Again, this is the exception rather than the rule. [Recommended cautiously due to possible loss of relevant material] 9. Use the button. You can use the button to parse through the headers of a newsgroup and lockout/reject messages as you see ones you don't want. This is rather labor intensive, but it does provide good results. SBNews will need to actually load the headers for a group before you can edit them (thus, you must be connected). 10. "Lockout Any". Located under the pull-down menu "Configure:Lockout:Any". You can lockout a phrase from any header line. This works on subject, from, posting-host, xref, and about a dozen other header lines in the message. Thus, if you know a phrase you absolutely don't want ("phone sex" is a good candidate here!), you can type in in here and SBNews will abort any message with the offending phrase. 11. "Lockout Posting Host". Located under the pull-down menu "Configure:Lockout:Posting-Host". This is a real power-user option. Each message includes a "NNTP-Posting-Host" field which identifies the host from which the message was sent. Usually this host is the ISP (Internet Service Provider) of the SPAM advertiser. To find "NNTP-Posting-Host" fields, you'll have to use the button, and the button located in the headers list to read an offending message, then scroll through header lines of the message itself. "NNTP-Posting-Host" should be one of them. Enter the host name into the lockout dialog. Like I said, this is a power user type function and should only be used by experienced persons. Most SPAM advertisers do not bother to supply a fictitious posting host, so this usually works. However, you will reject an entire host (which could be thousands of people), so this should be used with caution. 12. Submit complaints. The only way SPAM can really be stopped is if enough complaints are registered to the appropriate authorities. Some ISP's have very strict anti-SPAM policies and will terminate any users who post spam. This is usually not the case for the big commercial sites though. Who do you complain to? The SPAM posters usually go to great lengths to hide their true identities so they cannot be traced. However, if you user the button and the messages, you can find some interesting data in the message headers. Look at the "Path:", "Organization:", and "NNTP-Posting-Host:" lines -- these usually have some valid addresses. You can try sending complaints in that direction. The domain part is not enough to send to alone, you'll need a full email address. You can pre-append the names "webmaster@", "support@", "complaints@", "orders@", etc to the addresses and see if you can get somewhere. (For example, if you see "Organization: bogusspam.com", try posting to "webmaster@bogusspam.com") Sometimes your own ISP will have a complaints department to which you can attach a message to and they will follow-up and contact offender's ISP themselves. ------------------------------------------------------------------------------- Summary: 旼컴컴컴컴컴컴컴컴컴컴컴쩡컴컴컴컴컴컴컴컴컴컴컴컴컫컴컴컴컴컴컴컴컴컴컴컴컴커 쿥hat 쿥here 쿥hy � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿘aximum XRef Limit 쿎onfiguration:Preferences 쿔gnore messages posted to� � � 쿺ore than a specified � � � 쿻umber of newsgroups � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿗ockout XRef 쿎onfigure:Lockout:Xref 쿔gnore messages posted to� � � 퀂pecific groups � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿛reload XRef 쿎onfigure:Preferences 쿔n combination with � � � 쿪bove, pre-loads the � � � �"Xref" information so � � � 퀃hat SBNews can ignore a � � � 쿺essage without having to� � � 퀂tart downloading it. � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿘inimum Message Lines 쿎onfigure:Preferences 쿔gnore messages with too � � � 쿯ew lines in them to hold� � � 쿺eaningful data � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿗ockout Poster 쿎onfigure:Lockout:Poster 쿔gnore messages form a � � � 퀂pecified person (or any � � � �"From:" header line � � � 쿬ontaining the specified � � � 쿾attern) � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿗ockout Subject 쿎onfigure:Lockout:Subject 쿔gnore messages with a � � � 퀂pecific subject (or any � � � �"Subject:" header line � � � 쿬ontaining the specified � � � 쿾attern) � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿌dd "free" to Lockout 쿎onfigure:Lockout:Subject 쿗ots of 'Pay' services � 쿞ubject � 쿾ut 'Free' in the message� � � 퀂ubject. � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿌dd "http://" to 쿎onfigure:Lockout:Subject 쿗ots of 'Pay' services � 쿗ockout Subject � 쿾ut their http:// address� � � 쿶n the message subject � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 � button � Button 쿘anually view message � � � 쿴eaders and � � � 퀁eject/lockout the ones � � � 퀉ou don't want. � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿗ockout Any 쿎onfigure:Lockout:Any 쿗ockout any phrases which� � � 퀉ou know you don't want � � � 퀃o appear in desirable � � � 쿺essages. � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿗ockout Posting Host: 쿎onfigure:Lockout:Posting 쿗ockout a specific host � � 쿓ost �(i.e. ISP or service � � � 쿾rovider) which is � � � 쿾ermitting SPAM � 쳐컴컴컴컴컴컴컴컴컴컴컴탠컴컴컴컴컴컴컴컴컴컴컴컴컵컴컴컴컴컴컴컴컴컴컴컴컴캑 쿎omplain 쿻/a 쿞ubmit complaints to the � � � 퀃he offender's ISP to � � � 퀂top them. (sometimes it � � � 쿭oes work, but not � � � 퀅sually) � 읕컴컴컴컴컴컴컴컴컴컴컴좔컴컴컴컴컴컴컴컴컴컴컴컴컨컴컴컴컴컴컴컴컴컴컴컴컴켸 ------------------------------------------------------------------------------- So how will it end? You have to ask yourself, why are the commercial sites advertising in the newsgroups? There can only be one reason: Because it works. Because they are actually gaining customers by forcing advertisements down their throats. There's a simple solution to this -- do not use the pay sites. There's plenty of free stuff on the web -- just look around. The newsgroups are a good example -- once you filter out the SPAM, there is a large amount of on-topic material left behind. Maybe the solution is to communicate to the SPAMMERS that there are better ways to advertise. Posting 100 off-topic messages only enagages the average user into "ignore mode". Flooding the newsgroups with 100 off-topic messages only draws people away from the newsgroups. Posting 100 off-topic messages will generate complaints. Sooner or later a capable authority will get involved. My advice to all the SPAMMERS out there, "If you absolutely have to advertise in the newsgroups, post one on-topic advertisement. If people really want to visit your site, they'll do it." ------------------------------------------------------------------------------- Closing Notes: This document was originally written for SBNews version 4.8. I will try to keep it updated as new versions are released. The above mechanisms and techniques are some of the basic building blocks in SBNews and will probably be present for all time, in more or less their current format. You can find the latest version of SBNews at: http://smbaker.simplenet.com/sbnews/sbnews.html