Welcome to Volume 5, Issue 6 of The Internet Security Conference Newsletter, Insight. Insight provides commentaries and educational columns, authored by some of the best minds in the security community.
TISC is about sharing clue. So is this newsletter. We promise to provide something useful each issue. If we don't, flame me. If you like the issue, let us know!
Enjoy, and be safe,
Dave
Anyone who has ever configured, tested, and tried to maintain a security policy on a firewall knows the task is unrelenting. Networks change. Services your organization makes public evolve. Employees adopt new, publicly hosted and accessible services. Configurations change. One of the time-consuming and error-prone tasks associated with firewall administration is testing the configuration youíve implemented against the security policy you expect to be enforced. Surprisingly, few automated methods for accomplishing this fundamental and essential task exist. Todayís column discusses an interesting tool that I think has enormous promise, not only as a method for testing firewalls, but IDS and other security devices as well.
Happy Reading!
FTester is a tool designed for testing firewalls and Intrusion Detection Systems. It is based on a client/server architecture for generating real TCP/IP connections. The client is a packet generator tool (ftest) while the server (ftestd) is an intelligent network listener capable of processing and replying to ftest-generated packets. All packets generated by ftest have a special signature encoded in the payload that permits identification.
Packet injection programs typically issue correct as well as malformed TCP segments and IP packets to identify listening services and fingerprint operating systems.Ý FTesterís ability to simulate real TCP/IP connections allows users to test stateful inspection firewalls (netfilter, ipfilter, and commercial firewalls) and IDS (SNORT and commercial IDS). FTester offers additional features for manipulating traffic type, timing options, and connection states so the administrator may fully test the stack (and configured security policy) of the security device.
FTester can spoof IP source addresses. Since ftestd can distinguish packets are generated by ftest (by the payload identifier), FTester is able to simulate spoofing of real connections (We'll see later how ìconnection spoofing modeî works.)ÝÝÝÝÝÝ
FTester is not an easy tool to use and it's intentionally far from being fully automated. A good knowledge of the TCP/IP protocol, IDS and firewall operations are required to fully understand what FTester does. There are many good resources for understanding TCP/IP: my personal favorite is TCP/IP Illustrated, Volume One: The Protocols, by Stevens W. Richard (a must read).
I wrote FTester because I was tired of testing security systems manually using different packet injectors and analyzing test traffic using tcpdump.Ý I felt that a bit of automation was necessary and thus the program was born.
FTester is now supported by the Institute for Security and Open Methodologies (ISECOM) and it's been developed to be compliant with the Open Source Security Testing Methodology Manual (OSSTMM) requirements.
FTester components are PERL scripts. They can be executed on any platform with a recent version of PERL. Three perl modules -Ý Net::RawIP, Net::PcapUtils, and NetPacket ñ are also required. These can be downloaded at the Comprehensive Perl Archive Network (CPAN), and you can install them using the CPAN shell.
Installation is quite simple. Untar the latest archive. Everything you need will be decompressed along with an example configuration file, documentation and a script called freport for comparing ftest and ftestd log files. Before using the package I recommend to read and fully understand all documentation.
All the testing will be done using the packet injector (ftest) on a host placed on one side of the firewall and the listener (ftestd) on a host placed on one other side of it. The placement depends on which firewall policy (rules) we want to check: for example, to test inbound filtering, ftest must be used on the external segment and ftestd on an internal one. Testing outbound filtering is equally important, so in this case reverse the placement of the client and the server.
The definitions of the packets that will be sent are specified in a configuration file, here called ftest.conf. In all examples, the destination address represents either the host where ftestd is listening, or a host from which traffic can be captured (e.g., a neighbor host in a hub-based network). The source address can be any host or network we wish. Note, however, that when sending and testing spoofed traffic it's important to verify that any intermediate routers will route our spoofed source. Generally speaking, placing ftest and ftestd ìcloseî (in terms of hops) to the firewall makes our spoof attempt easier to configure and execute.
Here are a few examples of how packets are defined:
…
To issue a TCP packet to destination 10.1.7.1, port 80
from source 192.168.0.10 port 1024 with the SYN flag and ToS set to 0, we
useÝ ì192.168.0.10:1024:10.1.7.1:80:S:TCP:0î
…
In the previous example, if we wish to set the PUSH and
ACK flags, we use: ì192.168.0.10:20:10.1.7.1:1022:AP:TCP:0î
…
Suppose we want to send a UDP packet to destination
10.1.7.1 port 53 from source 192.168.0.10 port 53 with ToS set to 0. For this,
we use ì192.168.0.10:53:10.1.7.1:53::UDP:0î
…
Similarly, to send an ICMP packet type 3 code 5, we use
ì192.168.0.10::10.1.7.1:::ICMP:3:5î
FTester allows one to specify ranges for source address,
source port, and destination port. Source address can also be specified in CIDR
form (e.g., 192.168.0.1-255:1024:10.1.7.1:22:S:TCP:0, 192.168.0.1:1024:10.1.7.1:1-65535:S:TCP:0).
A stop signal is necessary to tell the ftestd that our test is completed. The stop_signal can be a TCP, UDP or ICMP packet; for example ìstop_signal=192.168.0.1:666:10.1.7.1:666:S:TCP:0î. This arrival of this packet will cause the sniffer to terminate. Any packet specified in the configuration file after the stop_signal directive won't be seen by ftestd.
Let's first see how we can use FTester to find some simple firewall filtering policies.
For testing which privileged services can be reached from the outside we'll use the following simple configuration file:
- ftest.conf -
# checking privileged ports
192.168.0.10:1025:10.1.7.1:1-1025:S:TCP:0
# checking proxy port
192.168.0.0:1025:10.1.7.1:3128:S:TCP:0
stop_signal=192.168.0.10:80:10.1.7.1:1025:S:TCP:0
We must be sure that the stop signal will reach ftestd, so in this case we've verified that the firewalled host can surf the web so a reply from port 80 should be fine (we'll see later that this logic doesn't apply for stateful inspection firewalls).
Let's start as root ftestd
on Host B: [root@hostb]# ./ftestd -i eth0 -v
Let's inject the packets as root on Host A: [root@hosta]#
./ftest -f ftest.conf -v -d 0.01
Output displays every packet sent and the ftest.log file is created.
- ftest.log -
# ftest started
1 - 192.168.0.10:1025 >
10.1.7.1:1 S TCP 0
2 - 192.168.0.10:1025 >
10.1.7.1:2 S TCP 0
3 - 192.168.0.10:1025 >
10.1.7.1:3 S TCP 0
... (deleted)
1023 - 192.168.0.10:1025 >
10.1.7.1:1023 S TCP 0
1024 - 192.168.0.10:1025 >
10.1.7.1:1024 S TCP 0
1025 - 192.168.0.10:1025 >
10.1.7.1:1025 S TCP 0
1026 - 192.168.0.10:1025 >
10.1.7.1:3128 S TCP 0
1027 - 192.168.0.10:80 >
10.1.7.1:1025 PA TCP
# ftest stopped
On Host B ftestd shows the same messages for each packet received and creates the ftestd.log file. As we can see each packet is marked with an unique identifier - the first number of the line, which corresponds to the packet IP ID field. This will help us when we compare the different log files.
For the final step, we copy the two log files on the same host and we compare them using freport (e.g., [root@hostb]# ./freport ftest.log ftestd.log).Ý The report generated looks like this:
Authorized packets:
-------------------
21 - 192.168.0.10:1025 >
10.1.7.1:21 S TCP 0
22 - 192.168.0.10:1025 >
10.1.7.1:22 S TCP 0
23 - 192.168.0.10:1025 >
10.1.7.1:23 S TCP 0
25 - 192.168.0.10:1025 >
10.1.7.1:25 S TCP 0
80 - 192.168.0.10:1025 >
10.1.7.1:80 S TCP 0
110 - 192.168.0.10:1025 >
10.1.7.1:110 S TCP 0
113 - 192.168.0.10:1025 >
10.1.7.1:113 S TCP 0
1027 - 192.168.0.10:80 >
10.1.7.1:1025 PA TCP 0
Modified packets (probably NAT):
--------------------------------
443 - 192.168.0.10:1025 >
10.1.7.1:443 S TCP 0
ÝÝÝÝÝÝÝÝÝÝÝ >>>>>>>>
443 - 192.168.0.10:1025 >
10.1.7.5:443 S TCP 0
Filtered or dropped packets:
----------------------------
1 - 192.168.0.10:1025 >
10.1.7.1:1 S TCP 0
2 - 192.168.0.10:1025 >
10.1.7.1:2 S TCP 0
3 - 192.168.0.10:1025 >
10.1.7.1:3 S TCP 0
... (deleted)
1026 - 192.168.0.10:1025 >
10.1.7.1:3128 S TCP 0
As we can see, ftp, telnet, ssh, smtp, http, pop3, auth are apparently reachable from the outside, https is forwarded to host 10.1.7.5 and the rest, including the proxy port is filtered, of course NAT regarding the destination address can be detected only if we sniff on a non-switched environment otherwise only port address translation or source address translation can be detected.
The results from sniffing packet 1027 (the PSH, ACK from port 80) in our example demonstrates that we are dealing with a simple firewall that does not perform connection tracking; however, stateful inspection firewalls are more likely to be found in real environments.
Stateful inspection firewalls usually perform connection tracking and they are not supposed to pass unmatched packets that aren't part of a previously initiated connection. For this reason, if we need to test the filtering policies regarding TCP PSH, ACK and moreover URG, RST, FIN packets, we have to create a real connection with the proper SYN, SYN-ACK, ACK handshake before sending them with the correct sequence numbers, so that the ftestd process will return the correct replies.Ý
When simulating the connection with a spoofed source address, the stack of the destination host will reply to our spoofed packets. The real host (the one we are spoofing) will reset the connection since it hasn't received responses. So we have to do two things: (1) prevent the tested host stack reply from reaching the firewall and the spoofed host itself, and (2) make sure that ftestd reply will traverse the firewall but will not reach the spoofed host. Setting a very low default hides the tested host stack reply. Setting the TTL to a low value equal to the hop delay between ftestd and the firewall hides the ftestd reply from the spoofed host.
Refer to this diagram to see how this trick works:
In this way we've fooled our firewall without involving the real host which address we've spoofed.
This mode can be activated with the 'connect=' prefix. Let's use this mode with packets that usually wouldn't traverse a stateful inspection firewall:
- ftest.conf -
# simulating a ssh connection
connect=192.168.0.10:1025:10.1.7.1:22:AP:TCP:0
# checking unmatched RST packets
192.168.0.10:1025:10.1.7.1:23:UR:TCP:0
# checking matched RST packets
connect=192.168.0.10:1025:10.1.7.1:23:UR:TCP:0
stop_signal=192.168.0.10:1025:10.1.7.1:80:S:TCP:0
And again:
[root@hostb]# ./ftestd -i eth0
-c 0:3 -v
[root@hosta]# ./ftest -f
ftest.conf -v -d 0.01 -s 1
Sent Syn Probe =>
192.168.0.10:1025 > 10.1.7.1:22 S TCP
Sleeping for 1 seconds
Sent Ack Reply =>
192.168.0.10:1025 > 10.1.7.1:22 A TCP
3 - 192.168.0.10:1025 >
10.1.7.1:22 AP TCP 0
5 - 192.168.0.10:1025 >
10.1.7.1:23 UR TCP 0
Sent Syn Probe =>
192.168.0.10:1025 > 10.1.7.1:23 S TCP
Sleeping for 1 seconds
Sent Ack Reply =>
192.168.0.10:1025 > 10.1.7.1:23 A TCP
8 - 192.168.0.10:1025 >
10.1.7.1:23 UR TCP 0
Stop packet =>
192.168.0.10:1025 > 10.1.7.1:80 S TCP
Note that this time, ftest is configured to wait 1 second for ftestd replies. The logs comparison follows:
[root@hostb]# ./freport
ftest.log ftestd.log
Authorized packets:
-------------------
3 - 192.168.0.10:1025 >
10.1.7.1:22 PA TCP 0
8 - 192.168.0.10:1025 >
10.1.7.1:23 UR TCP 0
10 - 192.168.0.10:1025 >
10.1.7.1:80 S TCP 0
Modified packets (probably NAT):
--------------------------------
ÝÝÝÝÝÝÝÝÝÝÝ >>>>>>>>
Filtered or dropped packets:
----------------------------
5 - 192.168.0.10:1025 >
10.1.7.1:23 UR TCP 0
The first RST has been dropped. This confirms that we are probably dealing with some sort of stateful inspection firewall.
If more than one packet is specified with the 'connect=' prefix and the same connection parameters (source address/port, destination address/port) the -r/-F flags must be used to correctly close (with a RST or a FIN handshake) each connection, otherwise, if the firewall is performing sequence numbers tracking, connections other than the first will be blocked. Sometimes not closing the connection can be useful to flood our firewall and see how it fills the connection state table.Ý
Additionally we can decide to fragment or segmenting all IP and TCP traffic and see how the firewall reacts.
The IDS testing option permits the injection of arbitrary packets with custom payload that is supposed to trigger an IDS alert. In this way we can test IDS visibility on the network and it's ability of sniffing fragmented/segmented packets. A number of common IDS evasion techniques are also implemented for activation during packets injection. Additionally ftest can directly use a snort rule definition file (check documentation for currently supported keywords). Here's a syntax example:
- ftest.conf -
ids=192.168.0.10:1025:10.1.7.1:25:S:TCP:0:VRFY
ids=192.168.0.10::10.1.7.1:::ICMP:3:5:+++ath
ids-conn=192.168.0.10:23:10.1.7.1:1025:PA:TCP:0:to
su root
ids-conn=192.168.0.10:1025:10.1.7.1:80:PA:TCP:0:cmd.exe
ids-conn=192.168.0.10:1026:10.1.7.1:80:PA:TCP:0:ftp.exe
insert /etc/snort/exploit.rules
192.168.0.10 10.1.7.1 0
insert-conn
/etc/snort/web-misc.rules 192.168.0.10 10.1.7.1 0
The '-conn' options works just like the 'connect' option. This is useful if the IDS is performing stateful inspection. When using this mode ftestd presence is needed otherwise since we are only testing the IDS the sniffer is not required. The 'insert' option takes four arguments: the definition file, the source address, the destination address and the type of service.Ý Let's use this feature to test snort ability in eluding evasion techniques, we'll use a common alert and we'll assume that snort is performing stateful inspection discarding unmatched traffic.
The configuration file:
- ftest.conf -
ids-conn=192.168.0.1:1025:10.7.0.1:80:PA:TCP:0:cmd.exe
Let's start snort on our sensor: [root@ids1] snort -A full
-c /etc/snort/snort.conf -D -b -d -i eth0 -l /var/log -s -z
Let's start as root ftestd
on Host B: [root@hostb]# ./ftestd -i eth0 -c 0:3 -v
Let's injects the packets as root on Host A:
[root@hosta]# ./ftest -f
ftest.conf -v -d 0.01 -s 1 -F
Sent Syn Probe =>
192.168.0.10:1025 > 10.1.7.1:80 S TCP
Sleeping for 1 seconds
Sent Ack Reply =>
192.168.0.10:1025 > 10.1.7.1:80 A TCP
6 - 192.168.0.10:1025 >
10.1.7.1:80 PA TCP 0 "cmd.exe"
Immediately, snort alerts us with the following syslog message (notice that snort also warns us about the ftest signature in the SYN packet):
Jun 5
16:25:13 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN detection
[Classification: Unknown Traffic] [Priority: 3]: {TCP} 192.168.0.10:1025 ->
10.1.7.1:80
Jun 5 16:25:14 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access
[Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025
-> 10.1.7.1:80
We can repeat the process using some dirty tricks like fragmentation and evasion techniques (see documentation for a detailed description):
[root@hosta]# ./ftest -f
ftest.conf -v -d 0.01 -s 1 -F -g 2
[root@hosta]# ./ftest -f
ftest.conf -v -d 0.01 -s 1 -F -e stream -p 3
[root@hosta]# ./ftest -f ftest.conf -v -d 0.01 -s 1
-F -e stream -p 1b
[root@hosta]# ./ftest -f
ftest.conf -v -d 0.01 -s 1 -F -e desync1
And so on...Ý Since snort rocks, it identifies correctly the triggering payload each time:
JunÝ 5 16:29:32 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN
detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
JunÝ 5 16:29:34 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access
[Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025
-> 10.1.7.1:80
JunÝ 5 16:30:16 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN
detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
JunÝ 5
16:30:18 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access [Classification:
Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
JunÝ 5 16:30:40 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN
detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
JunÝ 5 16:30:42 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access
[Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025
-> 10.1.7.1:80
JunÝ 5 16:31:00 lcars snort[4467]: [111:5:1] spp_stream4: DATA ON SYN
detection {TCP} 192.168.0.10:1025 -> 10.1.7.1:80
JunÝ 5 16:31:02 lcars snort[4467]: [1:1002:2] WEB-IIS cmd.exe access
[Classification: Web Application Attack] [Priority: 1]: {TCP} 192.168.0.10:1025
-> 10.1.7.1:80
The generated log will help in clarifying what has been sent:
- ftest.log -
# ftest started
IDS mode >> 3 -
192.168.0.10:1025 > 10.1.7.1:80 "cmd.exe" PA TCP 0
# ftest stopped
# ftest started
IDS mode >> 3 -
192.168.0.10:1025 > 10.1.7.1:80 "cmd.exe" PA TCP 0
# ftest stopped
# ftest started
IDS mode >> 3 -
192.168.0.10:1025 > 10.1.7.1:80 "cm" PA TCP 0
IDS mode >> 4 -
192.168.0.10:1025 > 10.1.7.1:80 "d." PA TCP 0
IDS mode >> 5 -
192.168.0.10:1025 > 10.1.7.1:80 "exe" PA TCP 0
# ftest stopped
# ftest started
IDS mode >> 3 -
192.168.0.10:1025 > 10.1.7.1:80 "c" PA TCP 0
IDS mode >> 4 -
192.168.0.10:1025 > 10.1.7.1:80 "m" PA TCP 0
IDS mode >> 5 -
192.168.0.10:1025 > 10.1.7.1:80 "d" PA TCP 0
IDS mode >> 6 -
192.168.0.10:1025 > 10.1.7.1:80 "." PA TCP 0
IDS mode >> 7 -
192.168.0.10:1025 > 10.1.7.1:80 "e" PA TCP 0
IDS mode >> 8 -
192.168.0.10:1025 > 10.1.7.1:80 "x" PA TCP 0
IDS mode >> 9 -
192.168.0.10:1025 > 10.1.7.1:80 "e" PA TCP 0
# ftest stopped
# ftest started
IDS mode >> 3 -
192.168.0.10:1025 > 10.1.7.1:80 "cmd" PA TCP 0
IDS mode >> 4 -
192.168.0.10:1025 > 10.1.7.1:80 "" S TCP 0 EVASION PACKET!
IDS mode >> 5 -
192.168.0.10:1025 > 10.1.7.1:80 ".exe" PA TCP 0
# ftest stopped
We've described Ftester and illustrated some basic usage here. However, there are many useful and practical applications where our tool can be helpful. By applying FTester's ability to create real traffic instead of mere probes and port scans, you can determine whether a security device's configuration complies with predefined security policies. By creating TCP connections with different options and states, you can evaluate security device stack integrity and its behavior under high traffic load conditions. Additionally, you can test IDS stateful inspection capabilities and alarm-triggering traffic sensitivity. This is very important considering how script kiddies often flood networks with false alarms with the expectation that such alarms are ignored when proper stateful inspection is in place. Finally, since both the ftest and ftestd scripts are entirely command line based and configuration file driven, it is possible to automate any tests you create, run them in batch mode, and perhaps most importantly, run exactly the same sets of tests each time you evaluate firewall and security device configurations, which you do, of course, on a periodic basis.
Andrea Barisani is a system administrator and security consultant. His professional career began 4 years ago but all really started when a Commodore-64 first came in his home when he was 10. Now, 12 years later, Andrea is having fun with large-scale IDS/Firewalls deployment and administration, forensic analysis, vulnerability assessment, penetration testing, security training, his Ftester and basically all the Open Source stuff related to the security field. He eventually found that system and security administration are the only effective way to express his need for paranoia. He is currently working as a system administrator for the Physics Department at University of Trieste and as a consultant for Live Network Security. When outside his text-based world he joins real life and among many hobbies he his studying for the bachelor's degree in Physics. His girlfriend constantly tries to reduce his work-related processes in order to increase his real world uptime.
