(EN) Quick log parser for iptables

Just a quick reminder for when you have to do some basic iptables’ logs parsing job.

Let’s say you have this kind of logs to parse :

Nov 24 08:43:35 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.57 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31678 DF PROTO=TCP SPT=21748 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0 
Nov 24 08:43:35 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.57 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31688 DF PROTO=TCP SPT=21750 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0 
Nov 24 08:43:38 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.57 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31723 DF PROTO=TCP SPT=21748 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0 
Nov 24 08:43:38 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.57 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31726 DF PROTO=TCP SPT=21750 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Nov 24 08:43:38 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.98 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31726 DF PROTO=TCP SPT=21750 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Nov 24 08:43:38 xmpptest kernel: FW_INPUT_DROP: IN=eth0 OUT= MAC=6a:9d:bb:6a:2f:f4:00:0d:b9:28:1b:6e:08:00 SRC=10.100.3.201 DST=10.100.2.56 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31726 DF PROTO=TCP SPT=21750 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0

Here’s a python2 script that permits to count and sort denied source IP, so you can identify the source of an attack.

iptables_log_parser.py

#!/usr/bin/env python
#-*- coding: utf-8 -*-

import re
import sys
from collections import Counter

DENY_PATTERN = re.compile('SRC=(\S*) .*DST=(\S*) .*PROTO=(\S*) .*DPT=(\S*)')
LINE_FORMAT=' {0:<12.12} {1:<12.12} {2:<6.6} {3:<6.6} {4}'

def process_log_file(logfile):
    """Reads through the log_file, and returns a counter based on Deny-lines."""

    # Process file line by line
    with open(logfile, 'r') as data:
        seen = Counter()

        # find all Deny line and append them in a list
        for line in data :

            # If line has 'Deny ' in it, then check it some more
            if 'FW_INPUT_DROP' in line:
                seen.update(DENY_PATTERN.findall(line))
    return seen
    
def print_counter(counter):
    """Pretty print the result of the counter."""

    print(LINE_FORMAT.format('source', 'destination', 'prot',  'port', 'hitcnt'))
    print(LINE_FORMAT.format(*tuple(('------------------',) * 5)))
    for (SRC, DST, PROTO, DPT), count in counter.most_common():
        print(LINE_FORMAT.format(SRC, DST, PROTO, DPT, count))

if __name__ == '__main__':
    # if file is specified on command line, parse, else ask for file
    if sys.argv[1:]:
        print "File: %s" % (sys.argv[1])
        logfile = sys.argv[1]
    else:
        logfile = raw_input("Entrez le fichier de log a parser, e.g /var/log/secure: ")

    denial_counter = process_log_file(logfile)
    print_counter(denial_counter)

Output exemple for the given input above :
File: logfile2

 source       destination  prot   port   hitcnt
 ------------ ------------ ------ ------ ------------------
 10.100.3.57  10.100.2.56  TCP    80     4
 10.100.3.98  10.100.2.56  TCP    80     1
 10.100.3.201 10.100.2.56  TCP    80     1

./Kriss