Chapter 5 - Security Log Analysis using Python

5.1 Chapter Overview

Security logs are one of the most important sources of evidence in cyber security. Logs record activities such as logins, failed attempts, blocked connections, application errors, malware alerts and system changes.

Python helps security analysts automate log analysis. Instead of reading thousands of lines manually, Python can search for suspicious events, count failed logins, summarize firewall actions, identify source IPs and generate investigation reports.

Learning Outcome: By the end of this chapter, learners should be able to parse, analyze and automate log monitoring from servers, firewalls and applications using Python.

Server Logs

→

Python Parser

→

Detection Rules

→

Alert Summary

→

Report

5.2 Learning Objectives

Understand the purpose of security logs in cyber security operations.
Identify common log sources including servers, firewalls and applications.
Read text, CSV and JSON logs using Python.
Use string methods and regular expressions to extract useful fields.
Detect failed logins, blocked connections and suspicious IP activity.
Summarize logs by IP address, username, action, severity and status code.
Generate CSV reports from log analysis.
Create simple automated log monitoring scripts.

5.3 Types of Security Logs

Log Type	What It Records	Security Use
Authentication Logs	Successful and failed login attempts.	Detect brute force and suspicious access.
Firewall Logs	Allowed and blocked network traffic.	Investigate blocked IPs and policy violations.
Web Server Logs	HTTP requests, status codes and user agents.	Detect web attacks and abnormal traffic.
Application Logs	Application errors, access and transactions.	Investigate suspicious user actions.
Endpoint Logs	Malware detections, process events and device activity.	Detect endpoint compromise.
VPN Logs	Remote access sessions.	Review unusual remote login behavior.

5.4 Common Log Fields

Field	Meaning	Example
Timestamp	When the event happened.	2026-06-02 10:15:22
Source IP	Where the activity came from.	192.168.1.50
Username	User account involved.	admin
Action	What happened.	ALLOW, DENY, LOGIN_FAILED
Status Code	Response result.	200, 403, 404, 500
Severity	Risk level.	INFO, WARNING, ERROR, CRITICAL

Useful Log Analysis = Time + Source + User + Action + Result + Pattern

5.5 Reading Basic Text Logs

Python can read logs line by line and search for keywords such as ERROR, FAILED, DENY or BLOCK.

Create a Sample Security Log

log_data = """2026-06-02 10:01:15 INFO user=amin action=login_success ip=192.168.1.10
2026-06-02 10:02:11 WARNING user=admin action=login_failed ip=203.0.113.5
2026-06-02 10:02:45 WARNING user=admin action=login_failed ip=203.0.113.5
2026-06-02 10:03:10 ERROR user=admin action=account_locked ip=203.0.113.5
"""

with open("security.log", "w", encoding="utf-8") as file:
    file.write(log_data)

print("Sample log created.")

Read and Print WARNING / ERROR Lines

with open("security.log", "r", encoding="utf-8") as file:
    for line in file:
        if "WARNING" in line or "ERROR" in line:
            print(line.strip())

5.6 Extracting IP Addresses with Regular Expressions

Regular expressions help extract patterns such as IP addresses from log lines.

import re

line = "2026-06-02 WARNING user=admin action=login_failed ip=203.0.113.5"

pattern = r"\b(?:\d{1,3}\.){3}\d{1,3}\b"

ip_addresses = re.findall(pattern, line)

print(ip_addresses)

Extract IPs from Full Log File

import re

pattern = r"\b(?:\d{1,3}\.){3}\d{1,3}\b"

with open("security.log", "r", encoding="utf-8") as file:
    for line in file:
        ips = re.findall(pattern, line)

        for ip in ips:
            print("Found IP:", ip)

5.7 Server Log Analysis

Server logs help detect failed login attempts, abnormal access and service errors.

Count Failed Login Attempts by IP

from collections import Counter
import re

failed_ip_counter = Counter()

pattern = r"ip=(\d+\.\d+\.\d+\.\d+)"

with open("security.log", "r", encoding="utf-8") as file:
    for line in file:
        if "login_failed" in line:
            match = re.search(pattern, line)

            if match:
                ip = match.group(1)
                failed_ip_counter[ip] += 1

print("Failed Login Summary:")
for ip, count in failed_ip_counter.items():
    print(ip, count)

Alert if Failed Logins Exceed Threshold

threshold = 2

for ip, count in failed_ip_counter.items():
    if count >= threshold:
        print("ALERT: Possible brute force from", ip, "Attempts:", count)

5.8 Firewall Log Analysis

Firewall logs record allowed and denied network connections. They help analysts detect blocked attacks, suspicious outbound traffic and policy violations.

Create Sample Firewall Log CSV

import csv

rows = [
    ["time", "src_ip", "dst_ip", "dst_port", "action"],
    ["10:01", "192.168.1.20", "8.8.8.8", "53", "ALLOW"],
    ["10:02", "203.0.113.5", "192.168.1.10", "3389", "DENY"],
    ["10:03", "203.0.113.5", "192.168.1.10", "22", "DENY"],
    ["10:04", "192.168.1.30", "198.51.100.10", "443", "ALLOW"]
]

with open("firewall_log.csv", "w", newline="", encoding="utf-8") as file:
    writer = csv.writer(file)
    writer.writerows(rows)

print("Firewall log CSV created.")

Analyze Denied Connections

import csv
from collections import Counter

blocked_ips = Counter()

with open("firewall_log.csv", "r", encoding="utf-8") as file:
    reader = csv.DictReader(file)

    for row in reader:
        if row["action"] == "DENY":
            blocked_ips[row["src_ip"]] += 1

print("Blocked Source IPs:")
for ip, count in blocked_ips.items():
    print(ip, count)

5.9 Application Log Analysis

Application logs may contain user activity, login events, errors and suspicious behavior.

Sample JSON Application Log

import json

logs = [
    {"time": "10:01", "user": "amin", "event": "login", "status": "success"},
    {"time": "10:02", "user": "admin", "event": "login", "status": "failed"},
    {"time": "10:03", "user": "admin", "event": "password_reset", "status": "success"},
    {"time": "10:04", "user": "guest", "event": "login", "status": "failed"}
]

with open("application_log.json", "w", encoding="utf-8") as file:
    json.dump(logs, file, indent=4)

print("Application JSON log created.")

Analyze Failed Application Logins

import json

with open("application_log.json", "r", encoding="utf-8") as file:
    logs = json.load(file)

for log in logs:
    if log["event"] == "login" and log["status"] == "failed":
        print("Failed login:", log["user"], log["time"])

5.10 Simple Automated Log Monitoring

Automated monitoring checks logs regularly and alerts when suspicious patterns are found.

import time

log_file = "security.log"
keywords = ["ERROR", "account_locked", "login_failed"]

print("Starting simple log monitoring...")

with open(log_file, "r", encoding="utf-8") as file:
    for line in file:
        for keyword in keywords:
            if keyword in line:
                print("ALERT:", line.strip())

print("Monitoring completed.")

Continuous Monitoring Concept

import time
from pathlib import Path

log_file = Path("security.log")

last_size = 0

while True:
    current_size = log_file.stat().st_size

    if current_size > last_size:
        with open(log_file, "r", encoding="utf-8") as file:
            file.seek(last_size)
            new_lines = file.readlines()

            for line in new_lines:
                if "ERROR" in line or "login_failed" in line:
                    print("New alert:", line.strip())

        last_size = current_size

    time.sleep(5)

Note: Continuous monitoring scripts should be tested in a lab before production use.

5.11 Generate Security Log Report

A good log analysis script should produce a report that can be reviewed by security teams.

import csv
import re
from collections import Counter

ip_counter = Counter()
failed_login_count = 0
error_count = 0

ip_pattern = r"ip=(\d+\.\d+\.\d+\.\d+)"

with open("security.log", "r", encoding="utf-8") as file:
    for line in file:
        if "login_failed" in line:
            failed_login_count += 1

        if "ERROR" in line:
            error_count += 1

        match = re.search(ip_pattern, line)

        if match:
            ip_counter[match.group(1)] += 1

with open("security_summary_report.csv", "w", newline="", encoding="utf-8") as file:
    writer = csv.writer(file)
    writer.writerow(["Metric", "Value"])
    writer.writerow(["Failed Logins", failed_login_count])
    writer.writerow(["Errors", error_count])

    writer.writerow([])
    writer.writerow(["IP Address", "Event Count"])

    for ip, count in ip_counter.items():
        writer.writerow([ip, count])

print("Security summary report created.")

5.12 Detect Suspicious Web Server Activity

Web server logs can reveal scanning attempts, missing pages, suspicious paths and server errors.

web_logs = [
    '192.168.1.10 GET /index.html 200',
    '203.0.113.5 GET /admin 403',
    '203.0.113.5 GET /wp-login.php 404',
    '198.51.100.8 GET /api/users 200',
    '203.0.113.5 GET /config.php 404'
]

for line in web_logs:
    if " 403" in line or " 404" in line:
        print("Suspicious or failed web request:", line)

Count 404 Errors by IP

from collections import Counter

not_found_counter = Counter()

for line in web_logs:
    parts = line.split()
    ip = parts[0]
    status = parts[-1]

    if status == "404":
        not_found_counter[ip] += 1

for ip, count in not_found_counter.items():
    if count >= 2:
        print("Possible web scanning from:", ip)

5.13 Practical Detection Rules

Detection Rule	Logic	Possible Meaning
Multiple failed logins	Same IP has many login_failed events.	Possible brute force.
Repeated DENY events	Same source IP repeatedly blocked.	Possible scanning or attack.
Many 404 errors	Same IP requests many missing pages.	Possible web directory scanning.
Admin login from unusual IP	Admin account login from unknown network.	Possible account compromise.
Critical application error	ERROR or CRITICAL in application logs.	Application issue or attack attempt.

5.14 Interactive Log Analyzer Demo

Paste sample log lines and click Analyze. The demo counts WARNING, ERROR and failed login lines.

Sample Logs

Click Analyze Logs.

5.15 Practical Activities

Activity 1: Read Log File

Create a text log file and print only WARNING and ERROR lines using Python.

Activity 2: Extract IP Addresses

Use regular expressions to extract IP addresses from a log file.

Activity 3: Firewall Log Summary

Create a CSV firewall log and count how many DENY actions each source IP has.

Activity 4: JSON Application Logs

Create a JSON application log and print all failed login events.

Mini Project

Build a security log analyzer that reads server, firewall and application logs and generates a CSV summary report.

5.16 Interactive Final Assessment Quiz

Each correct answer gives +1 mark. Each wrong answer gives -0.5 mark.

1. Security logs are useful for investigation and monitoring.

True False

2. Which Python module is commonly used for regular expressions?

re paint music video

3. Firewall logs can show allowed and denied network traffic.

True False

4. Which file format is commonly used for structured application logs?

JSON JPG only MP3 only EXE only

5. Multiple failed logins from the same IP may indicate brute force activity.

True False

6. Python Counter can help count events by IP address.

True False

7. A 404 status code in web logs can indicate a missing page request.

True False

8. Log monitoring should never produce reports.

True False

9. CSV files can be used for firewall log analysis.

True False

10. Security log analysis can help detect suspicious patterns.

True False

Your Score: 0

5.17 Chapter Summary

In this chapter, learners studied security log analysis using Python. They learned how to parse text logs, extract IP addresses, analyze server logs, firewall logs and application logs, detect suspicious events, automate monitoring and generate CSV reports.

Remember: Logs tell the story of what happened. Python helps analysts read that story faster, more accurately and more consistently.