Many of you are likely frequent readers of Matousec Transparent Security Lab’s white papers and tests, including the
Firewall Challenge. The Firewall Challenge is intended to show firewalls’ and security suites’ capabilities in resisting “leaktests” (tests that simulate data leakage) and other lesser-known ways of bypassing protection.
As can be seen from the list of solutions tested, both standalone firewalls and security suites are submitted for this report. One of those security suites is Kaspersky Internet Security 2009, which holds 7th place in the ratings with a score of 87% across 73 tests, a “very good” protection level, and a complexity level of “10+”.
Agnitum’s products have traditionally enjoyed well-earned leadership in these tests for several years. Currently Outpost Firewall Pro 2009 holds
1st place with 99% (the build tested was an initial one, released July 1, 2008), “excellent” protection and complexity level “10” (minor errors revealed by the test were fixed in subsequent builds, which enables us to claim a 100% result on all suggested levels if the product were retested in its current condition).
So here is our question. Why did a product with a lower total score receive a higher complexity level– 10+? The answer is actually quite simple. To reach a higher level, a product must get no less than 50% at the current level. The products that managed to pass two rather esoteric tests at the last – 10th level – received a 10+ mark according to this criterion. However, this showing doesn’t have a crucial impact on the final results, as a product can marginally pass all previous levels with just 50% and still show a higher result on the last level.
As for Outpost solutions, it’s no secret that Agnitum offers an integrated solution as well: Outpost Security Suite Pro, based on the standalone Outpost Firewall Pro and incorporating the same arsenal of proactive measures. As a suite, Outpost Security Suite Pro also includes antivirus and antispam capabilities, but these tools don’t play any significant role in the Matousec tests, nor do they interfere with the test procedures (even in the performance segment).
Now, let’s take a look at this
article (one of many published world-wide on the Internet). Now take another look at a new so-called “test leader” - Kaspersky Internet Security 2009 (the 7th in the overall rating, as noted above)! Surprised? It’s worth noting that Kaspersky Lab doesn’t take Outpost’s results into account because the most recent build of Agnitum’s security suite “hasn’t been tested by Matousec” and, therefore does not even merit a mention as far as Kapersky is concerned.
However, both Agnitum’s products are so close in their proactive protection and anti-leak capabilities that it seems to us rather strange to separate their achievements and give the winner’s crown to another vendor’s product – a security suite! – in the Firewall Challenge.
We’re confident that those who are interested in this topic are smart enough to read between the lines and make their own independent analysis of the test results. Here’s the link once again:
http://www.matousec.com/projects/firewall-challenge/results.php.
While we fully support Kaspersky’s efforts to promote the Matousec tests as a reliable source of information about the robustness of security products, we believe that the tests and their results should be weighted to reflect accurately the likelihood of users encountering each situation in the real world.
Below is the data derived from the Matousec tests (courtesy of Kaspersky Lab which created the initial chart, to which we have added the Outpost results).
By the way, Outpost demonstrated a 100% result on all levels but two while Kaspersky Internet Security scored 100% just on 4 levels out of 10. We leave you to draw your own conclusions …
Vitaly Yanko
Director of Marketing and Sales, Agnitum
As Agnitum is going to cease the support of obsolete antivirus bases (due to the implementation of a new mechanism) from December 1, 2008, you need to migrate from older versions of Outpost Security Suite Pro (2007 and early builds of 2008), which will stop getting antivirus updates at the end of this month — to the 2009 version.
