There is some kind of a struggle in the newsgroups between those who accepts the ‘undocumented’ programming and those who does not. I will try to express my thoughts concerning this issue here.
Nowadays, all popular AntiVirus (AV) software products can be divided into two main groups:
– those who has protection (and use undocumented staff)
– and those who does not (and use only documented approaches)
The ‘protection’ in this context is just some logical part of the product which ensures that AV cannot be terminated by malware modules. The protection logics can be implemented only using undocumented approaches, because Windows does not provide interfaces to ensure that some code should always keep running. Instead, Windows API provides flexible way to manage system resources. This of course, means that any entity can obtain access to resources if it has appropriate rights.
Such situation leads to the following scenario.
– The malware module is able to terminate (any) process if it has appropriate rights.
– The malwares also can modify the memory, context, PEB, and other properties of a process and make everything it wants in order to … hide its activity.
– Malware also can be represented as a rootkit – this is even worse, because rootkits have extremely big power, since they operate in kernel mode.
AntiVirus software also tries to do its best. Those who does not have protection scheme or those who does not use undocumented techniques fall into the limited ability to control the system. In this situation the AntiVirus software tries to fully control and filter all threats that are coming through acceptable for controlling channels.
A typical example of the written above is the powerful heuristics that is used in combination with file system filter driver (fully documented approach), which does not allow malware even to be copied on the target PC. And thus, there is no need to protect AV module – because heuristics will do the job. The disadvantage of this way is that if the system makes error – i.e. it treats malware as a normal executable, it (possibly) has no chances to control the system after the one mistake …
However, there is another approach. Combining documented ways with the undocumented. This involves new challenges and brings new problems. Undocumented is prone to changes. It means that once next build of OS or SP will be shipped AV makers probably will need to rewrite their code. In this case they may even globally change the architecture of the product because some major features can be cut from OS.
The positive side of this approach is that it provides extended challenges to control the OS. Thus, properly written AV that uses protection schemes is almost impossible to bypass by malware. Why I say ‘impossible’? Because software programs are written by the people. People do mistakes. If there is a need to bypass AV the malware writer can use exotic ways to accomplish that. I will discuss these ways later in my next posts.
‘Protection’ is a set of complex methods that allows controlling different facilities of operating system. These facilities cannot be controlled without interaction with undocumented techniques, because public API gives you a limited ability to control OS. The more directions are controlled the higher possibility that you may catch the bad thing ‘on a fly’ and do not allow it to harm your OS.