At the start of this week I met with the co-founder and CTO of Lastline, who claimed that the industry needs something better than Virus Total.
Giovanni Vigna, co-founder and CTO of Lastline, said that Virus Total is “the wrong tool” and that the industry needs something better.
He said: “I think that this is something that the security community has already acknowledged and the anti-virus companies know that we need to go beyond static signatures and we need dynamic data and dynamic execution to work with.”
In an article posted in January 2013, David Harley, senior research fellow at ESET said that Virus Total itself had spoken out several times against the misuse of the service as a substitute for testing and in a whitepaper, Harley and Virus Total’s Julio Canto said that “Virus Total is self-described as a tool, not a solution”.
They said: “As with any other tool (especially other public multi-scanner sites), it’s better suited to some contexts than others. It can be used for useful research or can be misused for purposes for which it was never intended, and the reader must have a minimum of knowledge and understanding to interpret the results correctly. With tools that are less impartial in origin, and/or less comprehensively documented, the risk of misunderstanding and misuse is even greater.”
Following the news story from earlier this week, I caught up with Harley to get his thoughts on the role of Virus Total within the information security industry. He said that he felt that Giovanni wanted Virus Total to be something it was never intended to be.
“We (and the guys at Virus Total) get ‘upset’ for a number of reasons when it’s misused for detection performance testing,” he said.
“Perhaps the most succinct way of putting it is that the site cannot be configured to provide a fair test of all the capabilities of the scanners it makes use of. That’s what sites like AV-Comparatives and AV-Test are there to do, or try to do: to provide accurate evaluation based on whole product testing. If that’s what Lastline wants, it could work with such a service, or try to set up something of its own.”
Asked if he felt that Virus Total was “fit for purpose”, Harley said you have to remember that Virus Total itself describes itself as “a free service that analyses suspicious files and URLs and facilitates the quick detection of viruses, worms, Trojans, and all kinds of malware”.
He said: “In other words, it looks at the problem from the other end: the malware, not the security software. In doing so, it’s useful to the end-user who wants to check a possibly malicious object. It’s also useful to companies whose products it uses, since it offers access to samples they may not have seen, plus limited performance data and specialist services that aren’t necessarily obvious to outsiders and individual users.”
However he said it is not a free plug-in replacement for the malware analysis that various elements of the security industry need to implement, but in the end, the person who submits the suspicious object has control over the submi
ssion, but not over the configuration of the tools that Virus Total uses.
Harley said: “Anti-virus product testing is difficult. I don’t know how you could provide a Virus Total-like service that allowed the service user to control configuration that wouldn’t widen the opportunities for misuse and misconfiguration where the service user doesn’t have full understanding of the malicious and security technologies involved.
“Actually, it’s even worse than that. We’ve already seen deliberate misuse of Virus Total by the ‘You should be buying _our_ services instead of AV’ crowd. This hypothetical Virus Total-like service would increase the potential for abuse by intentional misconfiguration added to other misleading approaches such as cherry picking samples.”
