Finding Malware with Firewall Rules

A customer new to our host-based firewall management wanted to be able to detect if a server had been compromised with malware. One way to spot this is to look for and log unauthorized outbound traffic. But how do you tell if traffic is unauthorized?

For this tutorial, you’ll take advantage of two properties of iptables: that its rules are order-dependent, and that it is aware of the state of your connections. (Windows’ native firewall has neither of these properties, so this will only work on your Linux/UNIX workloads.)

Order dependence means that you’ll be able to put rules to allow (and not log) authorized traffic first. You can then create broader rules that will catch anything not specifically authorized and log it.

Connection state awareness gives you a means to distinguish between outbound traffic that is permitted (for example, allowing web servers to initiate connections to your app servers on known ports, or to respond to an incoming request from your load balancer) and traffic that is not (such as initiating a connection to an unfamiliar IP outside your company.)

The types of connections iptables recognizes are NEW, ESTABLISHED and RELATED. NEW and ESTABLISHED are self-explanatory; RELATED connections happen when an existing connection generates a new, child connection.

iptables can ACCEPT, DROP or REJECT packets related to a connection. When DROP is used, iptables drops the packets without responding. REJECT sends a reject message back to the packet sender.

You don’t need to make any changes to your inbound rules. For outbound traffic, create a set of three (or more) rules:

1) a rule that allows connections of any type from your server to the servers and ports you want it to be able to initiate connections to, with no logging.

In this example, I’m allowing my servers running this firewall policy to make ANY connections to any other internal server with a 10.*.*.* or a 192.168.* address.  But you’ll want to restrict this to specific server groups and known internal ports.

Pull down the IP zones selector and create a new group:

The popup will prompt you for more info:new-zone-popup
This will cover connections that your servers initiate to other internal servers (e.g., databases).

2) a rule that allows outbound traffic from any RELATED or ESTABLISHED connection to the box, with no logging. These are for legitimate connections that you’re allowing in from outside and that your box is responding to.

3) a rule that logs all NEW connections, initiated by your box, to ANY IP. The reason the ANY works here is because you’ve already filtered out NEW connections to your own servers with rule 1, and RELATED or ESTABLISHED connections anywhere with rule 2. Any connections initiated by your box to non-internal IPs will get logged. Some of these may be ok, others may warrant looking into.

On my Amazon Linux test box, the firewall messages are logged to /var/log/messages and look something like:

Mar 23 19:16:41 ip-10-244-xx.xx kernel: UNAUTHORIZED: IN= OUT=eth0 SRC=10.244.xx.xx DST=54.65.xx.xx LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14528 DF PROTO=TCP SPT=58615 DPT=22 WINDOW=14600 RES=0x00 SYN URGP=0

This is what my final policy looked like. You can also add a default drop rule to both inbound and outbound.outbound-logging-full-rules-view

If you already have rules for outbound connections set up, then just set up rule #3. Make sure you have rules for all of your authorized traffic, though; otherwise you’ll end up with a lot of messages, and traffic that could indicate a compromise will get lost in the flood.

Now you can use log-prefix to tag these messages:

add log-prefix in FW rule

…then write a rule in Log-based IDS to look for your log-prefix string…


…and alert you when a match comes up in your security events.

security event

You’ll then want to examine any message that comes in tagged UNAUTHORIZED to see if it’s legitimate traffic (and should have a rule permitting it) or not.  Of course, the best way to prevent malware from spreading it to block unknown traffic altogether.  But if you’re just starting to use host-based firewalls and want more insight into what’s going on on your network, this is one way to do it.

Stay up to date

Get the latest news and tips on protecting critical business assets.

Related Posts