^.^;

Advanced Logging

Configure syslog, rsyslog, Windows Event Forwarding (WEF). Centralized log collection. Compliance retention.

IntermediatePhase 3: Password SecurityLive Log Streams

Why Advanced Logging?

Logs are your primary source of truth during incidents. Without centralized logging and proper retention, you're blind. Advanced logging collects events from every host, network device, and application—then ships them to a central SIEM for analysis, detection, and compliance.

What Advanced Logging Covers:

  • Syslog: Standard protocol for Linux/Unix logs (auth, kernel, app logs)
  • Windows Event Forwarding (WEF): Centralized Windows logs (Security, System, Application)
  • Log Shippers: Filebeat, Fluentd, Logstash, NXLog for forwarding logs to SIEM
  • Retention & Compliance: Meet GDPR, PCI-DSS, HIPAA requirements (90 days to 7 years)

Syslog & rsyslog

Syslog is the default logging protocol for Linux/Unix. It collects logs from kernel, auth, cron, apps, and sends them to /var/log or a remote server.

Common Syslog Files:

  • /var/log/auth.log — SSH logins, sudo usage
  • /var/log/syslog — General system events
  • /var/log/kern.log — Kernel messages (driver issues, hardware)
  • /var/log/apache2/access.log — Web server access

rsyslog Configuration (Centralized):

# /etc/rsyslog.conf
# Forward all logs to remote SIEM at 10.0.1.100:514
*.* @10.0.1.100:514   # UDP
*.* @@10.0.1.100:514  # TCP (more reliable)

# Or save specific logs locally
auth,authpriv.*  /var/log/auth.log
kern.*           /var/log/kern.log

💡 Best Practice: Use TCP for critical logs (auth, security). UDP can drop packets under load.

Windows Event Forwarding (WEF)

WEF centralizes Windows Event Logs from all domain computers to a Windows Event Collector (WEC) server. No third-party agents needed—built into Windows.

Key Event Logs:

  • Security — 4624 (logon), 4625 (failed logon), 4688 (process creation)
  • System — Service starts/stops, reboots, hardware errors
  • Application — Software crashes, errors
  • Sysmon — Advanced process monitoring (network connections, file creation, registry edits)

WEF Setup (High-Level):

1.Enable WinRM on all clients: winrm quickconfig
2.Configure WEC server to collect logs (Event Viewer → Subscriptions)
3.Create subscription: Select Security, System, Sysmon logs. Filter by Event IDs
4.Apply via GPO to all domain computers

💡 Install Sysmon on all Windows hosts for enhanced visibility (process trees, network connections, DLL injection).

Log Shippers

Log shippers read logs from files or event streams and forward them to a SIEM (Splunk, Elastic, QRadar). They handle parsing, filtering, buffering.

🚀 Filebeat (Elastic)

Lightweight, purpose-built for log shipping to Elastic Stack

  • • Ships logs to Logstash or Elasticsearch directly
  • • Pre-built modules for Apache, Nginx, Syslog, Windows
  • • Low resource usage (~10MB RAM)

🌊 Fluentd

Open-source, cloud-native, highly extensible

  • • 500+ plugins (S3, Kafka, Elasticsearch)
  • • JSON-based log parsing
  • • Good for Kubernetes logging

📦 Logstash (Elastic)

ETL pipeline for logs—parse, transform, enrich

  • • Grok filters for custom log parsing
  • • GeoIP enrichment, DNS lookups
  • • Higher resource usage (~500MB RAM)

🪟 NXLog

Windows-first log collector

  • • Reads Windows Event Logs, files, registry
  • • Forwards to Splunk, Elastic, syslog
  • • Good alternative to Splunk Universal Forwarder

Log Retention & Compliance

Compliance standards (PCI-DSS, HIPAA, GDPR, SOX) require specific log retention periods. Failure = fines, audit failures.

🔐 PCI-DSS (Payment Cards):

3 months online + 1 year archived. Security logs for all payment systems, firewalls, web servers.

🏥 HIPAA (Healthcare):

6 years minimum. All access to PHI (Protected Health Information) must be logged + auditable.

🌍 GDPR (Privacy):

No fixed requirement but recommend 90 days for security logs. Must demonstrate breach detection capability.

📊 SOX (Financial Reporting):

7 years for financial systems. All database access, admin actions, config changes.

Critical Logs to Collect

🔐 Authentication Logs:

SSH logins, RDP sessions, VPN connections, AD authentication (success + failures)

🌐 Network Logs:

Firewall allow/deny, IDS/IPS alerts, DNS queries, proxy logs, NetFlow

🖥️ Endpoint Logs:

Process creation, file modifications, registry changes, service starts/stops (use Sysmon!)

☁️ Cloud Logs:

AWS CloudTrail, Azure Activity Logs, Google Cloud Audit Logs (who did what in cloud)

🔧 Admin Actions:

User creation, privilege escalation, config changes, sudo usage

Best Practices

Centralize Everything: Don't rely on local logs. Attackers wipe logs first.
Time Sync (NTP): All hosts must have accurate time. Log correlation fails without it.
Log Integrity: Use write-once storage or log signing (WORM, S3 Object Lock)
Filter Noise: Don't forward everything. Focus on security-relevant events
Test Restoration: Can you restore logs from archive? Test quarterly
Monitor Log Volume: Sudden drop = logging agent failure or attacker tampering

Real-World Impact: Equifax Breach (2017)

Equifax was breached for 76 days before detection. One reason: expired SSL certificate on log forwarding broke centralized logging. Attackers moved laterally undetected.

What Went Wrong:

  • • ❌ Log collection failure went unnoticed
  • • ❌ No alerting on missing logs
  • • ❌ SOC blind to lateral movement
  • • ❌ 147 million records exfiltrated over 2.5 months

Lessons:

  • • ✅ Monitor log shipper health (heartbeats, volume tracking)
  • • ✅ Alert on certificate expiration
  • • ✅ Redundant log paths (multiple collectors)
  • • ✅ Regular testing of log collection

Centralized Logging Platform

Monitor log sources, view real-time log streams, and troubleshoot collection issues.

1,247
Logs Collected
3/4
Active Sources
3
Critical Alerts

Log Sources

Linux Auth (Syslog)

Authentication

active
Logs/min:45
Last log:2 seconds ago

Windows DC (WEF)

Active Directory

active
Logs/min:120
Last log:1 second ago

Firewall (Palo Alto)

Network

active
Logs/min:300
Last log:Just now

Web Server (Apache)

Application

error
Logs/min:0
Last log:5 minutes ago

Live Log Stream

14:32:01[Linux Auth]INFOSSH login success: user=jsmith from 10.0.1.45
14:32:03[Windows DC]WARNINGEvent 4625: Failed logon attempt for user administrator
14:32:05[Firewall]CRITICALBlocked connection attempt to known C2 server 185.220.101.5
Terms of ServiceLicense AgreementPrivacy Policy
Copyright © 2025 JMFG. All rights reserved.