Tests of Tiny Personnal Firewall 2

Key criteria in choosing a personnal firewall are :

• Effectiveness of security protection : penetration, Trojans, controlling leaks, denial of service.

• Effectiveness of intrusion detection: few false positives, alerting of dangerous attacks.

• User interface: ease of use, instructiveness, simplicity, quality of online help. Does the interface suit the way you use your PC ?

• Price.

How did we test firewall/intrusion detection effectiveness?

1. Ping and accessing shares to and from the test host.

2. A powerful, well known "remote control" Trojan (Netbus Pro v2.1 [2]) was installed on the system on a nonstandard port (to make detection more difficult), the Netbus server started and attempts made to connect from a remote system.

3. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sT -P0 -O IP_ADDR).

4. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sP -P0 -O IP_ADDR).

5. A test using Leaktest [4] was done.

6. We checked the system ressource usage of the firewall during the tests (just in case).

7. We tried to launch a modified (by us) release of IEXPLORE.EXE (C:\Program Files\Internet Explorer\IEXPLORE.EXE ) to check if the firewall detects the problem.

NB : These tests do not pretend to be exhaustives. By the way the aim is to be sure that the tested software offers at least expected security (or not) for a personnal use (do not compare this to professional use).

Jump to the test results.

The Tiny Personnal Firewall 2.0.12 [3] is full of interesting features :

• Possibility to allow/disallow some applications to connect to the net.

• Possibility to allow/disallow some services (ports) .

• Possibility to allow/disallow some protocols.

• Possibility to define trusted IP addresses (connected to your LAN interface).

• Possibility to define time range for blocking some traffic (when asleep for example) related to an application / rule.

• Possibility to define a password protection (such as in Conseal) for rules and one for log view.

• Download size : 1.3 MB

1. Ping: Impossible , unless you authorize it by yourself. This is a good result.

2. The Netbus server: Tiny Personnal Firewall 2 does not detect the Netbus server when started, but unless you authorize it, it's impossible to connect through it. The result of this test is good.

3. An nmap scan without Tiny Personnal Firewall 2 (on Win 2000 OS SP1 with a "standard" installation, it means NetBios active and so on) :
   
   $ nmap -v -sT -P0 -O IP_ADDR
   
   Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
   Initiating TCP connect() scan against (IP_ADDR)
   Adding TCP port 445 (state open).
   Adding TCP port 135 (state open).
   Adding TCP port 1025 (state open).
   Adding TCP port 913 (state open).
   Adding TCP port 139 (state open).
   
   The TCP connect scan took 0 seconds to scan 1523 ports.
   
   For OSScan assuming that port 135 is open and port 1 is closed and neither are firewalled
   
   Interesting ports on (IP_ADDR):
   (The 1518 ports scanned but not shown below are in state: closed)
   Port State Service
   135/tcp open loc-srv
   139/tcp open netbios-ssn
   445/tcp open microsoft-ds
   913/tcp open unknown
   1025/tcp open listen
   
   TCP Sequence Prediction: Class=random positive increments
   Difficulty=6634 (Worthy challenge)
   
   Sequence numbers: 747E9CE8 747F63FC 74800BF5 7480E3FE 7481BC4F 7482B3B2
   
   Remote operating system guess: Windows 2000 RC1 through final release
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 10 seconds
   
   Gloups : you'd better have a firewall installed :+) !!!
   
   An nmap TCP scan with Tiny Personnal Firewall 2 (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) with options set to "Ask me first" or "Don't bother me" do give some events registered in the log (unless you specify it) but blocks the traffic, which is finally a good result :
   
   $ nmap -v -sT -P0 -O IP_ADDR
   
   Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
   Initiating TCP connect() scan against (IP_ADDR)
   The TCP connect scan took 2334 seconds to scan 1523 ports.
   Warning: No TCP ports found open on this machine, OS detection will be MUCH less reliable
   Interesting ports on (IP_ADDR):
   (The 1522 ports scanned but not shown below are in state: filtered)
   Port State Service
   1032/tcp closed iad3
   
   Too many fingerprints match this host for me to give an accurate OS guess
   TCP/IP fingerprint:
   T5(Resp=N)
   T6(Resp=N)
   T7(Resp=N)
   PU(Resp=N)
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 2652 seconds
   
   This means that with Tiny Personnal Firewall active ports looks unexistant and access attempts are logged. This is a good result.
   
   Here I don't understand why they let the 1032 port seemed existant . This scan is logged in the Firewall Log as NMAP blocked attempt ... strange because not any rules talks about NMAP. So there's an internal rule built in that does this. I don't think it's a good idea to hide rules...
   

4. An nmap UDP scan with Tiny Personnal Firewall 2 (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) doesn't give events registered in the log but blocks the attemps which is a good result :
   
   
   $ nmap -v -sU -P0 IP_ADDR
   
   Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
   Initiating FIN,NULL, UDP, or Xmas stealth scan against (IP_ADDR)
   The UDP or stealth FIN/NULL/XMAS scan took 90 seconds to scan 1448 ports.
   (no udp responses received -- assuming all ports filtered)
   All 1448 scanned ports on (IP_ADDR) are: filtered
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 1755 seconds
   

   This means that the security seems efficient for UDP.

5. The Leaktest : Tiny Personnal Firewall 2 doesn't detect the launch of Leaktest and as for netbus , if you don't allow it it won't be able to connect. The result of this test is good.

6. Tiny Personnal Firewall 2 , in normal operations uses up to 1 % max. Memory usage is 5 MB, up to 5.6 MB peek.

7. The substitution test : (you can do it by yourself for example : you replace Iexplorer.exe with leaktest.exe - yes this one - by renaming the last one and launch it). The result is Tiny Personnal Firewall 2 allow the trojan horse to connect... the result of this test is bad. I feel surprise about the result of this test because even with MD5 checksum activated, Tiny Personnal Firewall 2 didn't block it... very strange... I hope MD5 is not done on the application name !

1. Tiny Personnal Firewall can be configured to block all trafic.

2. You can specified ports associated to an allowed application (it won't stop the substitution test, but it's nice).

3. Allows to warn through email.

1. Tiny Personnal Firewall does not provided description for standard services such as DHCP for example.

2. MD5 checksum does not work properly

3. Not enough events logged.

• Correct the MD5 problem..

• Don't hide specific rules (like NMAP detection), provide them in the default list.

• Provide standard protocol (eg DHCP) in the default list.

• Allow to import / export the rules.

• Do not ask to reboot after install.

• Improve the log function.

• Product internationalization.

A very good firewall with a very simple GUI but really efficient.

Evaluation :

• Installation process (2) : 15/20

• Configuration , GUI (3) : 16/20

• Filtering security (5) : 18/20

• Additionnal security (3) : 15/20

• Software load and memory usage (2) : 16/20

• Import/Export configuration (2) : 0/20

• Help , FAQ (2) : 10/20

• Product internationalization (1) : 0/20

Total : 13.25 / 20

Note : the result may be modified with the release , and when adding new criteria or re-evaluating their weight or their content.

1. Nmap - Network mapper, a really efficient tool to check networks
   http://www.insecure.org/nmap

2. Netbus Pro - Remote control program often used as an attack tool to control remote PCs.
   http://www.netbus.org/
   download

3. Tiny Personnal Firewall 2
   http://www.tinysoftware.com

4. Leaktest - Small testing software written by Steve Gibson to check firewalls. It makes a simple TCP (ftp) connexion that simulate sennding of personnal content, which can also be used to take remote controle in reverse mode (arg).
   http://grc.com/
   download