FST Key to Hot FBI Cyber-Terror Issue
Comments by FBI Cyber Crimes Assistant Director Shawn Henry made last October to reporters centered on a noticeable rise in cyber-terrorism and specifically cited "spear phishing" as a rapidly rising threat.
ISR's Forensic Sender Test™ helps UCE control systems combat all types of email identity crimes. Providing FST service in your mail system component or service not only brings tremendous value to your customers, but also affords an opportunity to show initiative on a “hot button” issue.
Contact us to learn more.
The FST API
Input Sources
There are many possible ways for a mail system to use the Forensic Sender Test™ and depending on the exact implementation, two basic ways to find the alleged sender address(es) to test against the delivering IP address.
For SMTP Firewalls
When used in a protocol-level firewall, the inputs are simply the connected peer's IP address and the email address portion of the SMTP MAIL FROM command. It is recommended that the mail system programmer be fully familiar with BOTH the official requirements of RFC 5321 Section 3.3 "Mail Transactions" AND the common practice of submitting a MAIL FROM address containing only an email address without even the text identifiers "<" and ">".
NOTE: There is much discussion on the required use of the "null sender" seen as "<>" in the MAIL FROM command which theoretically exists to keep a bounce of a bounce from being transmitted. We recommend that you neither generate external bounce messages nor accept the null sender in your MAIL FROM command. Our reasoning for this is that modern mail systems are capable of refusing mail after the RCPT TO command in the SMTP transaction making bounce messages unnecessary. Old systems that use bounce messages instead of instant verification (such as Exchange 2000) are used by spammers to relay spam. That means that your system is either part of the problem or absorbing the destructive power of the problem and either way, the null sender exists only as a problem. (Submitting a null sender to the FST is a waste of a transaction because there is nothing to test.)
For Post-SMTP Processing
After a message has been received, a post processor must be able to reliably recognize several pieces of information in the email headers to be certain that all possible senders have been checked for authenticity. Some organized criminals, for instance, will use a valid MAIL FROM in the SMTP transaction but a bank's actual customer service email address in the From field in the mail header. RFC 5322 actually describes many more sources of sender addresses. Different mail clients interpret these header fields in their own way, but the primary sources of input are:
- The IP address of the host that delivered the email to the
mail exchanger: Some SMTP server programs inject this IP address
as an easily identifiable header field and others leave it in
the more ubiquitous "Received" header stamp stacked at the top
of the message. Since this is a pivotal piece of information,
you'll want to be certain that your post-processor gets the
correct IP address and identifies things like internal mail
relays.
- If you didn't pre-screen the message with a call to the FST
to check the MAIL FROM during the SMTP transaction, you'll need
to find it now: This address SHOULD, in theory, be in the
Return-Path field of the header, but may be stamped as a header
field or embedded in the Received stack.
- The From header field: This field can actually contain a
list of addresses separated by commas and optionally surrounded
by <>. Check
RFC 5322 for complete details. Each address
presented in From should be sent to the FST because of the
identity theft possibilities. A spam scoring system might
incorporate each FST result as a separate score component.
- The Sender header field: If the From field contains more
than one address, the Sender field must contain exactly one
address. This address is more important than any address in the
From field for determining the authenticity of the overall
message, but of equal importance in making the identity theft
determination. Financial institutions do NOT send email from a
mix of addresses and especially do not include domains outside
of their own.
- The Reply-To header fields: These fields are primarily
important when making identity theft determinations. In
person-to-person email, redirected replies often facilitate the
use of aggregating mailboxes and may or may not even be a part
of a spam score. But in email from identity theft targets,
security and privacy policies (as well as a some regulations)
make it impossible or at least grossly irresponsible to see a
Reply-To address cause an FST failure in an authentic email.
- Other fields: RFC 5322 makes mention of various other header fields which may contain possible sender addresses. The use of those fields is up to the post-processor programmer and the specific needs and trends of the mail system. Currently, neither identity thieves nor mail clients are RFC-aware enough to make the testing of other fields valuable, but that could always change since the fields exist in a formal definition.
The HTTP Transport
The FST interface is reached via standard HTTP request with its inputs in the URL and output presented in a plain text HTTP response. Since a mail system might employ a direct socket link to accomplish the request, the following sample request packet may be helpful:
GET /FST2.dll?Key=xyz&IP=1.2.3.4&Senders=joe@test.com%20jane@test.com&Also=whois
HTTP/1.1
Accept: */*
User-Agent: Custom Socket Interface
Host: service.fst.ingenisys.com
Connection: Keep-Alive
And a sample result packet:
HTTP/1.1 200 OK
Date: Wed, 01 Oct 2008 21:28:53 GMT
Server: Microsoft-IIS/6.0
Content-Type: text/html
Content-Length: 62
IPF=111
MXF=110 110
SUM=110
WHOIS=Some Company Name, Inc.
The specific parameters of the URL and named result codes returned are discussed in detail on their own pages. General guidelines for the transport are as follows:
- The request MUST be URL encoded. Some HTTP components do
this for you, but if you plan to use a direct socket connection,
it's up to the programmer. URL encoding algorithms are available
for virtually every programming language with a quick Google
search.
- The value of the Key parameter may change from time to time
for security purposes. Your system should store and read this
value in a manner that can be updated without recompilation of
executable code or interruption of mail processing.
- Delimit lists in parameters using a space character (%20
encoded). (This is used in the "Senders" and "Also" parameters.)
- The response is given in a series of name/value pairs with
everything to the left of the first "=" on a line being the name
and everything after being the value. Values MAY contain "=".
Most modern programming languages have built-in facilities for
accessing name/value lists.
- Each request on a connection should be handled as
asynchronous but if your software is able to do so, continued
processing of an email while waiting for an FST response is up
to your own preferences and possibilities.
- The same connection may and should be kept alive for
streaming requests. This cuts down on the overhead of building
and destroying the TCP connection (which is necessary to ensure
the integrity of the transaction). You will be allocated a
maximum number of concurrent connections based on your volume.
This number will more or less define the maximum number of
processing threads (or forks) described below.
- The total time to process an FST request may be as high as a
few seconds so your post processor should use as many concurrent
threads as you deem appropriate for your volume. If you are able
to use the same threads to process continuous emails, connection
pooling becomes inherent in your design. If you plan to spawn
(or fork) a terminating thread for each email received into the
system, your threads should communicate with a central
connection pooler.
- Your system must use a timeout of 30 seconds on each request. (This is many times the actual expected wait time and is elongated to minimize unnecessary fail-over invocations.) A timeout should invoke the fail-over as described in the next section.
You will be assigned to a dedicated development support agent during your integration process. If any of the above is unclear, your DS agent will clarify it to your satisfaction. Our goal is to ensure your success and we will go to great lengths to achieve that goal.
Failure Safety and Load Balancing
The FST is delivered via a distributed network of servers. The method of selecting and connecting to a server is specifically designed to achieve:
- Fail-over redundancy
- Dynamic resource allocation
- Load balancing
- Maintenance and upgrade transparency
When opening an HTTP connection to the FST service, DNS should be queried for the A record of "service.fst.ingenisys.com". Multiple IP addresses will be returned. The connection should be attempted randomly on each of the available IP addresses until a connection is established. If a connection fails, that IP should be removed from the candidate list and another random selection made. If all IP addresses are eliminated without a successful connection, DNS should be re-queried and the process started over.
In the event that an FST query is issued and no response received within the 30 second timeout window described in the section above, the connection should be dropped and a new DNS query and HTTP connection constructed.