FireWall.net - Guide to install & configure a PC FireWall
o Home > freedom > Tests of freedom

>Sofware tests
--------------------------------------------------------
>Installation
--------------------------------------------------------
>Configuration
--------------------------------------------------------
>Checking
--------------------------------------------------------
>FAQ
--------------------------------------------------------


Contact Firewall.net :
e-mail-mail.

Tests of freedom

oTests oOverviewoPrice oResultsoAdvantagesoDisadvantages oImprovementsoSummaryoReferenceso

A - Security effectiveness Tests

Key criteria in choosing a personnal firewall are :

  • Effectiveness of security protection : penetration, Trojans, controlling leaks, denial of service.

  • Effectiveness of intrusion detection: few false positives, alerting of dangerous attacks.

  • User interface: ease of use, instructiveness, simplicity, quality of online help. Does the interface suit the way you use your PC ?

  • Price.

How did we test firewall/intrusion detection effectiveness?

  1. Ping and accessing shares to and from the test host.

  2. A powerful, well known "remote control" Trojan (Netbus Pro v2.1 [2]) was installed on the system on a nonstandard port (to make detection more difficult), the Netbus server started and attempts made to connect from a remote system.

  3. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sT -P0 -O IP_ADDR).

  4. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sP -P0 -O IP_ADDR).

  5. A test using Leaktest [4] was done.

  6. We checked the system ressource usage of the firewall during the tests (just in case).

  7. We tried to launch a modified (by us) release of IEXPLORE.EXE (C:\Program Files\Internet Explorer\IEXPLORE.EXE ) to check if the firewall detects the problem.

NB : These tests do not pretend to be exhaustives. By the way the aim is to be sure that the tested software offers at least expected security (or not) for a personnal use (do not compare this to professional use).

Jump to the test results.

B - Overview

The freedom 2.01b firewall [3] is full of interesting features :

  • Many tools (remembering what atguard's used to) : ad filters , email filters and form (http) filler.

  • Possibility to allow/disallow some applications to connect to the net.

  • Possibility to see all blocked packets.

  • Download size : 3.1 MB

C - Prices

Free for personnal (home) use.

D - Security Effectiveness
  1. Ping: impossible if you uncheck the "Allow pings to your Machine" in the Advanced Preferencies of Personnal Firewall from freedom 2.01b . The result of this test is good.

  2. The Netbus server: freedom 2.01b does detect the Netbus server when started, if you Disallow it, netbus server will complain about busy ports. It will complains either if you try to modify the port numbers. Connexions to the netbus server are impossible. The result of this test is good.

  3. An nmap scan without freedom (on Win 2000 OS SP1 with a "standard" installation, it means NetBios active and so on) :

    $ nmap -v -sT -P0 -O IP_ADDR

    Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
    Initiating TCP connect() scan against (IP_ADDR)
    Adding TCP port 445 (state open).
    Adding TCP port 135 (state open).
    Adding TCP port 1025 (state open).
    Adding TCP port 913 (state open).
    Adding TCP port 139 (state open).

    The TCP connect scan took 0 seconds to scan 1523 ports.

    For OSScan assuming that port 135 is open and port 1 is closed and neither are firewalled

    Interesting ports on (IP_ADDR):
    (The 1518 ports scanned but not shown below are in state: closed)
    Port State Service
    135/tcp open loc-srv
    139/tcp open netbios-ssn
    445/tcp open microsoft-ds
    913/tcp open unknown
    1025/tcp open listen

    TCP Sequence Prediction: Class=random positive increments
    Difficulty=6634 (Worthy challenge)

    Sequence numbers: 747E9CE8 747F63FC 74800BF5 7480E3FE 7481BC4F 7482B3B2

    Remote operating system guess: Windows 2000 RC1 through final release

    Nmap run completed -- 1 IP address (1 host up) scanned in 10 seconds

    Gloups : you'd better
    have a firewall installed :+) !!!

    An nmap TCP scan with freedom 2.01b (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) with options (Enable DHCP, Enable RPC and Enable Ident all checked) gives events registered in the log which is a good result for detection , but the protection is unefficient :

    $ nmap -v -sT -P0 -O IP_ADDR
    Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
    Initiating TCP connect() scan against (IP_ADDR)
    Adding TCP port 135 (state open).
    The TCP connect scan took 648 seconds to scan 1523 ports.
    For OSScan assuming that port 135 is open and port 67 is closed and neither are firewalled
    Interesting ports on (IP_ADDR):
    (The 1519 ports scanned but not shown below are in state: filtered)
    Port State Service
    67/tcp closed bootps
    68/tcp closed bootpc
    113/tcp closed auth
    135/tcp open loc-srv

    TCP Sequence Prediction: Class=random positive increments
    Difficulty=14685 (Worthy challenge)

    Sequence numbers: 774E9648 774F1714 774FDFBA 7750C00C 7751F39E 7752BCD3
    Remote operating system guess: Windows 2000 RC1 through final release

    Nmap run completed -- 1 IP address (1 host up) scanned in 648 seconds

    This means that with these options too many ports remains opened even if access attempts are logged, and that it remains possible to guess wichi OS !! This is a bad result.

    An nmap TCP scan with freedom 2.01b (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) with options (Enable DHCP checked and Enable RPC , Enable Ident both uncheked) gives events registered in the log which is a good result for detection :

    $ nmap -v -sT -P0 -O IP_ADDR

    Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
    Initiating TCP connect() scan against (IP_ADDR)
    The TCP connect scan took 350 seconds to scan 1523 ports.
    Warning: No TCP ports found open on this machine, OS detection will be MUCH less reliable
    Interesting ports on (IP_ADDR):
    (The 1520 ports scanned but not shown below are in state: filtered)
    Port State Service
    67/tcp closed bootps
    68/tcp closed bootpc
    1127/tcp closed supfiledbg

    Too many fingerprints match this host for me to give an accurate OS guess TCP/IP fingerprint:
    T5(Resp=Y%DF=N%W=0%ACK=S++%Flags=AR%Ops=)
    T6(Resp=Y%DF=N%W=0%ACK=O%Flags=R%Ops=)
    T7(Resp=Y%DF=N%W=0%ACK=S++%Flags=AR%Ops=)
    PU(Resp=N)
    Nmap run completed -- 1 IP address (1 host up) scanned in 375 seconds

    This means that the opponent migth see some existing ports but they look closed to him, some fingerprints remains available even ifnot enough to guess which OS it is. This is an average result.

    It is recommended that at least you - if you don't want to use a real firewall - select these last options to have minimal security efficiency.

  4. An nmap UDP scan with freedom 2.01b (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) gives events registered in the log which is a good result for detection and in this case the protection seems efficient :

    $ nmap -v -sU -P0 IP_ADDR

    Starting nmap V. 2.53 by [email protected] ( www.insecure.org/nmap/ )
    Initiating FIN,NULL, UDP, or Xmas stealth scan against (IP_ADDR)
    The UDP or stealth FIN/NULL/XMAS scan took 1744 seconds to scan 1448 ports.
    (no udp responses received -- assuming all ports filtered)
    All 1448 scanned ports on (IP_ADDR) are: filtered
    Nmap run completed -- 1 IP address (1 host up) scanned in 1744 seconds

    This means that the security seems efficient for UDP.

  5. The Leaktest : freedom does detect the software start (like Netbus) , the connection attempt looking like a ftp connection is filtered as long as you answered no to the question, the result of this test is good.

  6. Freedom goes up to 99% of CPU during the heavy portscans :-). In normal operations it uses up to 4% max. Memory usage is 7 MB, up to 13,3 MB peek.

  7. The substitution test : (you can do it by yourself for example : you replace Iexplorer.exe with leaktest.exe - yes this one - by renaming the last one and launch it). The result is as long as you've given rights to the usual software to connect (in our example : Iexplorer - Internet Explorer) to Internet freedom does not makes any difference, it will allow the trojan horse to connect... the result of this test is bad.

E - Advantages
  1. Freedom can be configured to ignore ping (from any sources).

  2. Close some unused ports.

  3. Allows to forbid some applications.

F - Disadvantages
  1. freedom cannot makes any difference between local network connexions and Internet connexions.

  2. The installation process and registering could be really faster and easier.

  3. The log is unreadable when network operations occurs (window scrollings).

  4. Totally incompatible with Conseal firewall product (generate Blue Screen Of the Death !).

G - Suggested improvements
  • Improve the installation process (fasten it).

  • Improve the logs.

  • Improve all the security efficiency !

  • Improve compatibility (too much warns about other firewall products).

  • Product internationalization.

H - Summary

A good idea, which needs much more security and work to be really usefull. Users may like the ad filtering...

Evaluation :

  • Installation process (2) : 5/20

  • Configuration , GUI (3) : 10/20

  • Filtering security (5) : 10/20

  • Additionnal security (3) : 0/20

  • Software load and memory usage (2) : 10/20

  • Import/Export configuration (2) : 0/20

  • Help , FAQ (2) : 10/20

  • Product internationalization (1) : 0/20

Total : 6,5 / 20

Note : the result may be modified with the release , and when adding new criteria or re-evaluating their weight or their content.

I - References
  1. Nmap - Network mapper, a really efficient tool to check networks
    URL http://www.insecure.org/nmap

  2. Netbus Pro - Remote control program often used as an attack tool to control remote PCs.
    URL http://www.netbus.org/
    URL download

  3. Zero knowledge Freedom tool
    URL http://www.freedom.net

  4. Leaktest - Small testing software written by Steve Gibson to check firewalls. It makes a simple TCP (ftp) connexion that simulate sennding of personnal content, which can also be used to take remote controle in reverse mode (arg).
    URL http://grc.com/
    URL download
This site is copyright Chryjs 1999-2001, all copies forbidden.