By submitting your email address, you agree to receive emails regarding relevant topic offers from TechTarget and its partners. You can withdraw your consent at any time. Contact TechTarget at 275 Grove Street, Newton, MA.
Security in general is usually a cat-and-mouse game, so it has its own unique economy, driven by threats to keep everyone employed—including the criminals. If we lived in a perfectly trustworthy society, the security profession would play a much smaller role of basic enforcement. Then again, there is no such thing as absolute security, regardless of how trustworthy a society or individual may be, because there will always be a threat of some kind, even to an offline computer.
In the controversial world of antispam, whenever someone makes a statement like "Spam filters do not stop spam," we all begin to hear a very loud noise in our ears. Organizations and individuals who spend their livelihoods designing and marketing the latest and greatest filter technology become offended. However, in the world of spam filters, it all comes down to a numbers game. Since the majority of spam filters catch 95–99 percent of spam, limiting the number of spam in a user's inbox from 20 mails to 1 each week is a significant improvement and is worth the investment. We all know what a pain it is to try sifting through email that is overloaded with spam.
Yet with all this in mind, we still need to keep in mind the following point: Spam filters do not stop spam. Why? Because spam still traverses the networks, uses network bandwidth, and gets delivered to a folder in almost all cases. Additionally, you, the user, are still forced to look at spam unless you want to miss the occasional false positive (legitimate mail mistakenly detected as spam) email that you will probably get at the office. So, in actuality, spam filters do not prevent anything—they merely classify and sort your email the best they can while lessening the change in behavior required for you to read through the email.
There are many other problems with the majority of antispam filters. Since spam continually evolves, you cannot just sit there and wait for the filter to automatically work; the spam filter must be "trained" to understand what is spam and what is not spam. Some antispam companies send signature "trained" updates to their spam filters; others simply succumb to the understanding that dedicated resources need to be applied to continue to stay on top of this annoying epidemic. Others use global checksum systems, which are a more effective implementation in comparison to the filters that require "training."
Something we have observed with phishers is that they seem to successfully pass their phish emails through the standard spam filters. This is largely due to the fact that they simply learned their traits from spammers, or they were once spammers and have now moved "up" to phishing. The majority of spam filters used today are based on Bayesian algorithm that looks for certain characteristics in the email and scores them. Bayesian filtering measures the probability of spam via its characteristics. This is a scoring system that can be trained by giving it samples of good email (ham) and spam. An example is Spam Assassin's (SA) engine. An email marked as spam within its filter might look like Figure 18 when you receive it.
Figure 18 Spam Assassin Scoring
Content preview: GLOBAL LOTTERY INTERNATIONAL 72657, NL-2115 DB EMIRATE, THE NETHERLANDS INCONJUCTION WITH GLOBAL LOTTERY INTERNATIONAL Dutch & UAE,EMIRATE FLY EMIRATE. From: The Promotions Manager International Global/ Emirate Club /Prize Award Department. REF: DATE: 25th march 2005. ATTN: ( CONGRATULATIONS ) [...]
Content analysis details: (17.2 points, 5.0 required)
|0.1||X_PRIORITY_HIGH||Sent with 'X-Priority' set to high|
|1.4||UNDISC_RECIPS||Valid-looking To "undisclosed-recipients"|
|2.4||RATWARE_OE_MALFORMED||X-Mailer has malformed Outlook Express version|
|1.7||MSGID_FROM_MTA_ID||Message-Id for external message added locally|
|1.4||DATE_IN_FUTURE_96_XX||Date: is 96 hours or more after Received: date|
|2.2||FORGED_YAHOO_RCVD||'From' yahoo.com does not match 'Received' headers|
|0.4||US_DOLLARS_3||BODY: Mentions millions of $ ($NN,NNN,NNN.NN)|
|1.5||RAZOR2_CF_RANGE_51_100||BODY: Razor2 gives confidence level above 50% [cf: 100]|
|0.1||RAZOR2_CHECK||Listed in Razor2 (http://razor.sf.net/)|
|2.9||NIGERIAN_BODY1||Message body looks like a Nigerian spam message 1+|
|3.0||FORGED_MUA_OUTLOOK||Forged mail pretending to be from MS Outlook|
With the minimum spam scoring requirement of 5.0, this particular email is clearly marked as spam, since it has a 17.2 point rating. As you can see in Figure 3.18, each line item has a point score that is used to tally the final aggregated content analysis rating. We see a 0.1 point rating for X_PRIORITY_HIGH, which is something that some users have on by default, especially if they are in marketing (just kidding). This received a low score since the probability is high that it is not always spam. The Razor (a distributed spam filtering network; see http://razor.sourceforge.net) check states that it's a 50/50 chance that it is spam, and the email contents are listed in Razor.
Next at 1.4 is the "undisclosed recipients," which indicates bulk mailing, but the system gives it a low score in case it is a valid solicited bulk mailing. The Message-ID was added from the original sender, which could be a sign of a spammer, since senders do not need to add their own Message-ID if they are sending legitimate email. The date of the Received header is 96 hours off from the actual date received. This is a good indication that this is spam.
A 2.4 score was given to an X-mailer header that had a bad Outlook Express version displayed, which dovetails nicely with the 3.0 score that basically states this email did a bad job of looking like Outlook. The message body received a 3.3 in total points, since it indicated qualities of a Nigerian scam, including the mention of "millions of dollars." And finally, a badly forged Yahoo.com domain is a dead giveaway. What we said earlier regarding Hotmail headers also goes for Yahoo; both have very specific style headers, and obviously this spoofed Yahoo! email did not match up.
In this Spam Assassin report, almost everything that could have been wrong with this spam email was wrong. However, many savvy spammers actually test against these numbers. The advantage of using Spam Assassin is that it is open source, it's free, and it works. The disadvantage of using Spam Assassin is that it is open source, it's free, and it works. This means that the tool has become a threat to both spammers and phishers. When there is a significant threat to the ROI, the phishers and spammers will invest their time to defeat the threat, which is where the cat-and-mouse game comes into play.
A quick look through these Bayesian filter scores with Spam Assassin and we can see that our phishing spam from Chapter 2 worked just fine. Why? We kept it simple. The less you try, the more you fly. A friend who worked for the National Security Administration (NSA) once told me that the best way to be anonymous is to blend in. The same goes for email. Detection systems will see the obviously suspicious activity, but by staying creative, yet cool, spam tends to fly under the radar. Obfuscation such as misspelled words or "creative" ways to spell words have been successful at bypassing many spam filters. Making your headers less obvious and possibly less forged could help. The use of trojans has assisted phishers and spammers in sending their spam past the filters, since the emails are authentic. They send them from some cable modem user, and they are not even trying to hide that fact. One of the common methods is to include a hash buster in the subject and body field. This can contain random characters, letters, words, and sometimes book phrases. This is in an attempt to add legitimacy to the email content and throw off the signature or hashing system used in some filters that hash an email to watch it for multiple emails with the same signature. By sending random data per email, the signature won't match against hash-based filters such as Razor and Distributed Checksum Clearinghouse (www.rhyolite.com/anti-spam/dcc/).
Now for the cat again: Most spam filters use a combination of hashing, probability scoring, keyword filtering, whitelisting, and blacklisting (RBL—http://rbls.org—is an example of a blacklist). Most spammers use techniques that are designed to thwart these techniques, but then again, antispam vendors know this and design systems to thwart against those techniques … I think you get my point.
One fairly new method spammers presented last year in retaliation for antispam techniques is what is known as reverse NDR, which stands for nondelivery receipt. Spammers are taking advantage of the NDR that is part of the SMTP RFCs (www.ietf.org/rfc/rfc0821.txt/ and www.ietf.org/rfc/rfc0822.txt/). An NDR is usually seen when you send an email to an address that does not exist. In response you will receive a message that looks like this:
Subject: Mail System Error – Returned Mail From: Postmaster@sendingemail.com Date: 04/03/2005 12:53 PM To: email@example.com Content-Type: multipart/report; report-type=delivery-status;
Boundary="=================================__= 7188110(20378)1092081234" X-SPAM-Status: No, hits=0.0 required 5.0 tests= version=2.20 Recipient: <firstname.lastname@example.org> Reason: 5.1.1
… User unknown Please reply to if you feel this message to be in error. ....
This report complies with RFC 822, and it is quite obvious that our spam engine did not even test it. So, the spammers found a loophole. Since NDRs are very necessary, you definitely want to know if you sent your email to an invalid address. And since they are part of "spec," they get cleared without any authentication or probability tests.
Here is the technique: The attacker wants to be able to get mail past your filter and have you read it. They create their spam message, but their sending address is spoofed as the victims they actually want to send it to:
From this point, when the spammer sends this email, he will try to contact email@example.com, and the MTA for receivingemail.com will send an NDR notice to firstname.lastname@example.org. Attached in the NDR report is the spam. Essentially, this takes us back to the open relay days, since spammers can utilize other mail servers to handle their bulk mailings, and that's virtually filter proof. It also has a high rate of visibility by the victims, since recipients will most likely view a Returned Mail notice. This technique can be adopted successfully by phishers as well on the basis of playing with the odds, since phishers are already playing the odds, guessing how many people have a certain type of bank account while blindly sending emails to everyone. Phishers can do the same with NDRs, if you received an NDR that stated you sent a message to email@example.com instead of 'firstname.lastname@example.org. They can then direct you to report the incident by clicking a form and, once again, steal your credentials. It's all about a little creativity, and you would be surprised at the successful return rate.
The road ahead in the fight against spam is still a bit foggy, but security in depth has so far been the most successful tool against this overwhelming problem. Solutions such as Sender-Policy-Framework (SPF; http://spf.pobox.com/) and Sender-ID (www.microsoft.com/mscorps/safety/technologies/senderid/ default.mspx/) have been proposed, but they are a far cry from worldwide adoption, since many of these proposals either have fundamental flaws or are hampered by inconvenience. With all the various antispam initiatives and an overly saturated market fraught with a plethora of vendors focusing on the antispam problem, why doesn't spam go away? More important, what will be done to stem the quickly growing extension of spam, phishing?
10 tips in 10 minutes: Phishing exposed
Tip 1: Email basics for Exchange admins
Tip 2: Understanding email delivery
Tip 3: Anonymous phishing email
Tip 4: How phishers forge email headers
Tip 5: Phishers use of open relays and proxy servers
Tip 6: How phishers send anonymous email
Tip 7: Phishers techniques for email harvesting
Tip 8: Phishers, hackers and insiders
Tip 9: Sending spam; phishing tools of the trade
Tip 10: Phishing email and spam filters