[Top]
[Links about spam]
[Comments about Timo's method]

Foiling Spam
with an Email Password System
|

Spamming is futile, you will be filtered
A legal note: The author shall not be liable to the user, the reply
target or any third party for any direct, indirect or consequential
loss or damage arising from using, or a failure to be able to use,
the information in this message/file howsoever caused. No warranty
is given that all the information contained is correct, or that it
is current.
Please have the courtesy of studying this information locally
without further emailed questions on the subject or the details of
the included information. I'd like to help the gentle fellow users,
but I can't afford the time to give further free individual guidance
on email filtering and auto-responding. Bug reports and genuine
comments on this information are, however, very welcome.
Please use this information wisely!
Please be forewarned that there is a hot opposition to the C/R
(challenge-response) email filtering concepts, best observed in the
Usenet newsgroups
news.admin.net-abuse.email and comp.mail.misc.
The method even made a curious, inaccurate appearance on a personal
hate page (gone since) about another person.
However, also highly valid concerns have bee raised. The main
problem is the so-called backscatter
i.e. misdirected bounces and challenges. Much of the spam comes from
stolen addresses or even hijacked computers. Thus blanket challenges
can cause an inordinate amount of mayhem to innocent parties. (Also
see Jari
Aalto's posting at Google about some of the potential problems.)
As you may have seen one version of my signatures ends with this
notification:
Spam foiling in
effect. My email filter autoresponder will return a
required email password to users not yet in the privileges
database.
|
I am frequently asked by the similarly spam-targeted fellow users
about my spam foiling methods. Especially, if you often write to the
Usenet news you are bound to get spam email, i.e. unsolicited, often
fraudulent commercial junk mail to your electronic mail box. As you
will have seen, many posters cripple their addresses in these days
using something like ts@nospam.uvasa.fi. This
is not a good solution. I know from my own experience that
crippled addresses are inefficient. The crippling only slightly
slows down the propagation of your true email address to the various
unsolicited email lists. Furthermore, although this is just personal
opinion, distorting one's email address deters the useful, welcome
professional email even more than the password system. In my opinion
the only effective solution to the UCE (Unsolicited Commercial
Email) problem is the password system which I am about to describe
to you.
I use Unix elm (later mutt) as my main email program. Unix email can
conveniently be preprocessed with automatic filters such as the outdated
"filter" program, which is part of the elm package, the prevalent "procmail", and the rather demanding "deliver"
program. This document was originally written for the "filter" program.
It has later been converted for "procmail" since I changed over to using
it at the end of September 1999.
"procmail - autonomous mail processor"
Procmail is a powerful and flexible system, but getting started with
it is not easy even if one has a reasonable amount of experience
with Unix. One of the problems is that the original documentation is
not very user-friendly (the common problem with Unix facilities).
Nevertheless, you will be well-advised to get and print the relevant
man manual documentation on your Unix system. Type the commands
-
|
Command
|
What it is about
|
|
man procmail
|
The autonomous mail processor
|
|
man procmailex
|
Procmail rcfile examples
|
|
man procmailrc
|
Procmail rcfile
|
If the above commands won't work, you probably do not have a fully
installed "procmail" on your system. Please contact your provider's
system manager about it. For the useful procmail WWW pages by Era
Eriksson and Nancy McGough, respectively, please see the procmail links on my links
collection. I also have a procmail tips and
recipes page of my own.
The essence of setting up the filter system is
this. First, you need a ~/.forward file. If you do not have such a
file create it with a suitable editor. After that make its
permissions to be -rw-r--r-- with the following command
"chmod 644 ~/.forward". (You might also need to set
"chmod a+x ~/").
Find out where the filter program is located on your system with
"which procmail". You'll get something like "/usr/local/bin/procmail".
Put the following line into your ~/.forward file. This time, include
the quotes (").
"|IFS=' ' && exec /usr/local/bin/procmail || exit 75 #myid"
where myid is your user identification. If you are not sure what
your userid is, try typing
whoami
and your Unix system should give your userid. There is some further
information you need to find out before we can proceed. Go ("cd") to
the directory where your email folders are located. Then type
pwd
to get the location where you are. Assume that you get
/home/myid/.mail
Alternatively, to find out what your email directory is, or to
change it, call elm, choose o (options), and observe or adjust the
line "F)older directory". Then you can read e.g. the returned email
folder by calling
"elm -f =Returned.mail".
Next, build a filter rules file "~/.procmailrc". An example of a
fairly simple filter-rules file is given below. Please note to
substitute /home/myid as befits your own environment. (You can also
try to substitute it with ${HOME})
# Environment variables for procmail
MAILDIR=/home/myid/.mail
LOGFILE=/home/myid/.mail/procmail.log
#Let's delete all email from cyberspam.com (Note here :0 not :0: *)
:0
* ^From:.*@cyberspam\.com
/dev/null
#Let's put announcements from the mailer-daemon in a special folder
:0:
* ^From:.*MAILER-DAEMON
/home/myid/.mail/Returned.mail
#Let's not directly accept mail to mailing lists, only personal email
#Note to substitute myid with your own userid on the second line
:0:
* !^TO_myid
/home/myid/.mail/ProbableSpam.mail
#Let's put all the rest in the mailbox
:0:
${DEFAULT}
*)This is about the so-called file locking preventing the
confusion of the potential, simultaneous writing of several messages
at the same time. It does not concern (and should not be used for)
the general bitbucket /dev/null.
The much more complicated part of such a system is emailing the
automatic responses telling of your password. Basically, you'll have
to include into your filter-rules file
#Environment variables for procmail, including the additions below
#Get the subject
#Discard some dangerous special chars + any leading and trailing blanks
#Note: On some systems -xSubject: has to be -x"Subject: "
SUBJECT=`formail -xSubject: \
| sed -e 's/[;\`\\]/ /g' \
| expand | sed -e 's/^[ ]*//g' -e 's/[ ]*$//g'`
FROM=`formail -rt -xTo: \
| sed -e 's/[;\`\\]/ /g' \
| expand | sed -e 's/^[ ]*//g' -e 's/[ ]*$//g'`
#Accept email having password on the subject line
:0:
* ^Subject:.*mypass
${DEFAULT}
#Send password notifications (except to your own domain)
:0:/home/myid/.elm/password.script.lock
* !^From:.*mypc.mydom
| /home/myid/.elm/password.script "${FROM}" "${SUBJECT}"
The "/home/myid/.elm/password.script" is e.g. a Bourne shell script
that automatically replies to the sender "${FROM}" with your canned
message informing of your password "mypass". Both the filter-rules
file and the script below are a much simplified versions of what I
use, but even this much requires fair basic understanding of the
Unix system. Do not adopt the script below if you do not thoroughly
understand what you will be doing. You could harm your own email and
cause considerable grief to your site's postmaster. As an example,
you might accidentally generate infinite email loops at your site if
you do not understand the internal logic of this method.
-------------------------- snip ------------------------------------
#!/bin/sh
# Ensure System V vs. BSD compatibility, get yours from your sysadmin!
#
PATH=/usr/local/bin:/usr/ucb:/usr/bin:/bin ; export PATH
# Information from the filter's command line
#
sender="$1"
subject="$2"
# Path to a temporary, auxiliary file to build the reply
#
tempfile="/home/myid/.elm/password.script.tempfile.$$"
cat << xxENDITxx > ${tempfile}
*************************************************
* This is a computer-generated response message *
*************************************************
Dear "${sender}"
Thank you for your email about
"${subject}"
...Whatever opening pleasantries.... To send email to xx@yyy.zzz
please put "password" anywhere on your subject line. ...Whatever
closing pleasantries...
xxENDITxx
/usr/local/bin/mail -s "Returned email: Password required" ${sender} < ${tempfile}
rm -f ${tempfile}
-------------------------- snip ------------------------------------
Alternatively, a considerably less complicated and a slightly less
flexible password system can be built with without using external
scripts. Before proceeding you may wish to take a look at "Timo's procmail tips and recipes" in
general, and the item "How do I
automatically return certain email messages?" in particular.
-------------------------- snip ------------------------------------
#Define the password
PASSWD_=YourRequiredEmailPassword
#Whatever other recipes in between.
#Return email if the password is not there
:0:passwd.lock
#
# Check for (the lack of) the password
* $ ! ^Subject:.*${PASSWD_}
#
# Avoid email loops
* ! ^X-Loop: myid@myhost\.mydom
* ! ^From:.*myid@([-a-z0-9_]+\.)*myhost\.mydom
#
# Prepare and send the notification
# Be sure to customize your sendmail path
| (formail -r \
-i"Subject: Returned email: Password or privileges required" \
-A"X-Loop: myid@myhost.mydom" ; \
echo "*************************************************" ; \
echo "* This is a computer-generated response message *" ; \
echo "*************************************************" ; \
echo ; \
echo "Email password required!" ; \
echo "Please include (${PASSWD_}) anywhere on your subject line." ; \
echo "Then kindly resend your email to myid@myhost.mydom") \
| /usr/lib/sendmail -t
-------------------------- snip ------------------------------------
There
is a fairly simple variation to complement (rather than replace) the
public email password system. One could call it the WWW-page variant
or even the Guestbook rendition. It goes as follows: As you may
know, in building a WWW page the HTML code for sending email via the
WWW page is
<A HREF="mailto:myid@myhost.mydom">Click to send me email</A>
If one changes it to
<A HREF="mailto:myid@myhost.mydom(FirstName%20LastName%20Passwd)">Click to send me email</A>
Then the procmail recipe for accepting such email is simply
:0:
* ^To:.*FirstName.*LastName.*Passwd
${DEFAULT}
Naturally, this method is not a guarantee against spam. But it is
surprisingly effective in actual practice. I have used this
complementary variation along with my main email password spam
foiling method on several on my web pages since January 1998. Almost
no spam has resulted via this route.
The big advantage of this method is that it avoids the first contact
conundrum if both the parties are using a challenge-response email
password system.
A personal note: As a professor of accounting and finance I am
definitely not adverse to responsible business and advertising.
However, unsolicited junk email that is force-fed at a cost of the
recipients or their organizations is neither sound business practice
nor ethically acceptable. Furthermore, most of it these days seems
to be scams with a criminal intent. Or at the very least fraudulent
or substandard products, which often would be illegal to import
anyway.
All the best, Timo
--
Prof. Timo Salmi
mailto:ts.uwasa.fi
ftp &
http://garbo.uwasa.fi/
Hpage
http://www.uwasa.fi/laskentatoimi/english/personnel/salmitimo/
Department of Accounting and Finance,
University of Vaasa,
Finland
Timo's FAQ materials at
http://www.uwasa.fi/~ts/http/tsfaq.html
Dense spam-kill procmail filtering with a public email key in effect
Further useful links about spam
Not surprisingly on the net there is available an overabundance of
material about spam-avoidance.
Fight Spam on the Internet!
Scam Spam Reporting Page
Fighting
email spam and anti-UBE pointers Jari Aalto
The Coalition Against Unsolicited
Commercial Email CAUCE
Spam, Spam, Spam, Spam, Spam, Spam, Spam Web Developer's Virtual
Library
Anti-Spam (junk email) links and information Rex Tincher
What is spam?
Vicomsoft
on spam
Anti Spam Software & Internet
Filters Blog - Securence
Spämmäys
(in Finnish)
Ryhmän sfnet.viestinta.roskapostit virallinen kuvaus (in Finnish)
Ei-toivottu viestintä
Internetissä Kai Puolamäki (in
Finnish)
FAQ: Current Usenet spam thresholds and guidelines
Address Munging FAQ by WD Baseley
Address Munging Considered Harmful by Matt Curtin
Re: Spam-proofed "From:" lines on The Forum on Risks to the Public
in Computers and Related Systems
The Email Abuse FAQ by WD Baseley
E-Mail Abuse Internet Chain Letters, Hoaxes, and Spam; David Harley
Net Abuse FAQ Index
Spam statistics at newsadmin.com
Abkuerzungen: de.admin.net-abuse.mail
(in German)
How do spammers
harvest email addresses? By Uri Raz
FAQ: How do spammers get people's email addresses ?
Despammed mail filtration
service
SpamWeed Another case of
anti-spam software
The Spam Bouncer Spam-countering set of procmail recipes
SpamAssassin.
A mail filter to identify spam.
Spam Slammer download
SpamWeasel junkmail-busting freeware utility
MailWasher program to check
and manage your emails before you download them
Spam Catcher a
web-based utility
CanIt "anti-spam solution for
enterprises, campuses and ISPs"
SpamProbe - A Fast
Bayesian Spam Filter
Spamihilator Anti-Spam-Filter
A Plan for Spam
Paul Graham
Sam Spade, Spam hunter
SpamCop spam reporting & spam filtering
(Very useful)

Spamhaus anti-spam site
Phishing Wikipedia
free encyclopedia.
Anti-Phishing Working Group
alt.spam FAQ or
"Figuring out fake E-Mail & Posts"
PayPal:
Protect Yourself from Fraudulent Emails
Network Abuse Clearinghouse
Distributed
Checksum Clearinghouse
MAPS, LLC -- the Mail Abuse Prevention System
OsiruSoft's Open RelaySpam Stopper (for admins)
dr. Jørgen Mash's DNS
database list checker
The Office of the Finnish Data Protection Ombudsman
On Unsolicited Commercial Electronic Mail Regulation
A Statement before the U.S. House of Representatives
Spam Free Washington! About Bill 2752
A bill to regulate the transmission of unsolicited commercial
electronic mail Senator Frank Murkowski
The CAN-SPAM Act of 2003
When spam burns you:
why bulk e-mail is bad business Linda Formichelli
Death to Spam A Guide to Dealing with Unwanted E-Mail
Usenet Death Penalty
U.S. Unsolicited
broadcast fax restrictions
ELM - Electronic Mail for UNIX
The SPAM-L FAQ mailing
list dedicated to "Spam prevention and Discussion".
Countering Spam with
Ham-Authenticated Email and the Guarded Email Protocol David A.
Wheeler
The Next Step in
the Spam Control War: Greylisting Evan Harris
Spam Arrest Counter spam by
requiring clicking a web page
Whitelist.Com Home of the
e-mail Receptionist
SPAMBLOCKED.COM
Services
Origin of the
term "spam" to mean net abuse Brad Templeton
The Original Monty Python SPAM Skit
Challenge-Response Anti-Spam Systems Considered Harmful A "rant"
against C-R.
A challenging response to challenge-response
John Levine:
Challenge-response systems are as harmful as spam
Trial Spam Blocker Anti-Spam
Software
spamgourmet - disposable email
addresses
Filtering with a Gmail round-trip
How to Shield Yourself from Sophisticated Spam Online College.org
Discussion:
news:news.admin.net-abuse.email
Discussion of abuse of email systems.
news:sfnet.viestinta.roskapostit
Roskaviesteistä
(in Finnish)
news:comp.mail.misc (Occasionally) General discussions about
computer mail.
Re: Spam - share current experience, please?
The Economics of Spam The spam business isn't always what you'd think.
Comments and Questions about Timo's
Method
I don't quite understand the structure of
your email password logic. Please elaborate.
Let's start all the way from the basics of avoiding emailed spam,
also called UCE (Unsolicited Commercial Email).
- The most common, but a rather ineffective safeguard is address munging, such as using
ts@NOSPAM.uvasa.fi instead of the true ts@uvasa.fi. I can tell from
experience that this system gradually leaks. Another, similar method
is hiding behind a totally forged identity. That is something I will
not do. I dislike anonymity and
give very little credence to anonymous Usenet postings and email. I
personally always appear with my own name and my own address.
- Blacklisting is a very
common approach to the spam problem. In blacklisting one uses an
email filter to automatically discard email that comes from known
spammer domains or know spammer addresses or has common spam
keywords (such as XXX, cable descrambler, make money fast) in the
subject header. The disadvantage with blacklisting is that one has
to keep updating the database and it always will leak somewhat.
Furthermore, if a domain is blacklisted, nothing gets trough.
Incidentally, some ISPs (Internet Service Providers) filter
well-know spam sites (such as Cyberpromo) already at the system
level. Ours does. But this, of course, leaks a lot.
- At one extreme, in a sense, is whitelisting. That is letting through
only email that comes from users or domains that the user has
privileged in his/her database. In this method all else is
discarded. Used this way, the whitelisting is far too radical a
method of isolation. It is very problematic for making any new
contacts.
- Finally, there is the email password
system. Email that has a magic keyword in the subject
header or in the body of the text is allowed through. The problem
with this alternative is that the sender has to know and to use the
receiver's password.
My own spam foiling method is a combination of all the above
alternatives.
- At the core of my usage is a whitelist of domains and users whom I
will always let directly through.
- The second layer of my system is the password system. Senders
not on the whitelist will get an autoresponse from me giving my
simple, public email password which to put into his/her future email
subject headers in order to reach me.
- The outer layer of my system is a blacklist, which has on it
some abusive or otherwise persistently irksome cases whom I don't
want to hear from. The blacklisting has two main levels:
- Automatic returning of the email, unread.
- Automatic deletion without any further notice.
As an aside, the latter is a convenient alternative when one is,
say, on a mailing list, and one temporarily wishes to suspend
receiving material from that mailing list. It is easier and less
drastic than unsubscribing.
- There is some further elaboration in my system, like fully
automated spam complaints, the selective truncation of overly long
messages, etc. However, the previous items gave you the general
outline of the arrangement of my method. The details fine tuning are
beyond this response.
Re:
Mail Filters That Use Challenge-Responses (CRs) A Usenet news
posting by yours truly in comp.mail.misc, news.admin.net-abuse.email
I think that with a little more diligence
and snooping around, you can put together a spam filter that will
trap most of the junk, and keep the noise in your In-box at a
manageable level, without forcing anyone who sends you mail to jump
through hoops.
Blacklisting is what you suggest. While you are partially
right, I prefer my own cocktail of measures rather than resorting to
the blacklisting option only.
Please understand this. It is true that I enjoy working with
computer related problems. It is also true that I like to post on
the Usenet news also a lot on subjects that have nothing to do with
my actual work as a professor of accounting and business finance.
Nevertheless, for me email is above all a tool in my profession. I
must have, and I intend to have, an undisturbed access to that
essential tool. If some secondary email and contacts get lost
because of that, that is a price I am well willing to pay.
I seriously doubt that most people will not
be able to follow your automatic directions to send you
email.
Whitelisting is the solution. I appreciate the point which
you raise. However, the problem is decisively alleviated by my
extensive database of privileged users. Most of my regular email
contacts already have their email addresses in my whitelist
database. They do not have to deal with my password system, at least
not as long as their email addresses stay unchanged.
As before, please recall that my system is a combination of the
measures. Thus my password requirement will affect only users who
are not on my whitelist. Or on my blacklist, for that matter. But
the latter are outside the password system anyway, and can't use it,
no matter what.
The users with whom I need to converse in my profession on a regular
basis are mostly unaffected by my email filtering system. As for the
rest of those benevolent users who may have occasional email for me,
yes, it is most unfortunate that the net has come to this. However,
as I appear a lot to the net, I simply no more can afford the mutual
luxury of other arrangements. A sad fact of current Internet
life.
Isn't the system password ineffective once
the spammers get to know your password?
If
you have to ask that, perhaps you do not quite grasp the essence of
email spamming. By its very nature spamming is a huge mass activity.
There is no way the spammers have the resources to customize in
order to bypass a single individual's public password protection.
Furthermore, contrary to a getting a new, spam-free user id, an
email password is easy to change anytime. It is not a problem. I
have not needed to change my own email password even after over six
years of usage. Besides, most spammers use forged addresses. They
won't ever get the autoresponse giving the password.
Apropos, while the email password system is very efficient against
spammers, alone it is not equally efficient against deliberate mail
bombing directed against a person. The combination of measures,
including blacklisting most importantly, has to be resorted to if
that happens.
How good is your system in foiling the
spam?
The following is a good indication how efficient the email password
system is. I started using my current automatic email password
screening system in the summer of 1997 to complement my automatic
white/blacklisting. By the end of 1998, that is in one and a half
years, exactly two spam email messages had made it as far as my main
mailbox. In all practical terms, the email password requirement
prevents all emailed spam from reaching me.
A side effect of the system, not unexpectedly, has been that the
average quality and seriousness of the non-spam email that reaches
me has improved considerably. Especially, casual users who used to
send in the strangest of questions and requests usually do not
always bother to work through the email password system. Or they
have unreturnable addresses in the first place. This is another
useful side effect. I now very seldom, if ever, get email that is
unreturnable.
I am certainly not claiming that my arrangement is without
disadvantages. Some potentially useful contacts might be put off by
my system. Furthermore, in a few cases long email has been cut when
it should not have. Fortunately, outright technical problems are
fairly uncommon, since I have honed this system for so long now.
Is your spam foiling system used
much?
If you mean do many other users adopted this and have a system like
mine, the answer is that I do not know. I have seen a couple of
cases, but the system does not seem to be very common. One
indication could be this. I get a daily log of visits to my web
pages. The statistics of e.g. Wed 17-Apr-2002 show 81 visits during
one day to the current page. But in contrast, my related procmail tips page, got 238 visits during
the same day. That's not bad.
The posed question rises, however, another interesting, partly
unresolved issue. It is the question of the logic of initiating a
contact between two users of the email password system when the
parties are previously unknown to each other. Because the password
system still probably is rather rare this problem has not yet risen
more to the fore. Some common protocol is probably in order just
like in the exchange of public keys in PGP exchanging encrypted
messages. The options include publicizing one's public email
password on one's web page, in one's signature, or accepting email
sent through one's webpage (since then special arrangements are easy
to set up). In fact, I use a certain medley of such measures
myself.
I am interested in the earliest references
to whitelists, blacklists and so forth. When did you install your
method? Are you aware of any work on these concepts prior to
yours?
Ideas like this tend to be general. I am not able to pinpoint them
to any definite, earlier sources. But the ideas certainly are not
originally mine. I just reinvented the wheel a bit. My very first
applications of using an automatic acceptance and rejection
filtering method dates back to approximately around 1994, as far as
I can recall. I have been using and developing them ever since. The
email password requirement came along in 1997. Originally I started
by using the "filter" program. In 1999 I changed to my current "procmail" method.
I originally installed my system a few years before the Internet
deluge of spam (unsolicited commercial email) started. As it
happens, I was targeted and persistently stalked by a mentally ill
Canadian former student who kept on trying to bomb my mailbox with
strangely sick messages using alternating forged sources under
several guises. The filter system was quite effective against the
harassment. In a way the situation later turned out to be a blessing
in disguise. When the current spam deluge started, I was much better
equipped than the average user.
In retrospect, one can note that requiring non-whitelisted users an
email password falls into the category of what now are called
challenge-response (C-R) methods.
As for the WWW-page method variant also described on this page, I
can locate my notes of installing it at least back to the 18th of
January, 1998. Again, I do not claim to be the original inventor.
But I have devised and adopted this method independently, so in
that in a sense it is a development my own.
As a system administrator I object to the
part of your spam foiling robot which makes an automatic complaint
to the postmaster when your robot thinks the filtered email is spam.
That spam was forged. It was not sent from our site.
Dear System Administrator,
The highly unwelcome unsolicited commercial bulk / junk email is
most often sent from forged addresses so that it is very difficult
for an ordinary user like yours truly to manually decipher where a
spam really originated and what route exactly it has taken. I do not
have the facilities, the ability and most certainly not the time to
resolve these problems myself. Therefore I use the automatic filter
responses and email password systems. The email must have been sent
to such a blanket mailing list which my filter does not have
whitelisted.
I take it that you are an administrator along the alleged email
route, whether valid or forged. I am sorry that it is impossible for
my email filter evaluate the true origin of a message.
It is the unfortunate lot of the paid administrators of the sites to
resolve problems like this one, not to be angry at the spam victims.
I only wish to be left alone from email abuse, and go about my
ordinary tasks without undue email disturbance from anyone.
Thank you for your current and future understanding and cooperation
in doing what is your job.
- All the best, Timo
[ts(ät)uwasa.fi ]
[Photo ]
[Programs ]
[FAQs ]
[Research ]
[Lectures ]
[Acc&Fin ]
[Faculty ]
[University ]
[Revalidate]
C:\_G\WWW\~TS\INFO\SPAMFOIL.HTM