Network logging failures almost always trace back to one of four things: the log daemon is stopped, the device is forwarding to the wrong destination, a firewall drops UDP 514, or the log volume is full. Confirm the service is running, verify the forwarding target, open the port, and free storage.
Symptoms
- Central SIEM shows gaps for specific hosts while other devices continue to report normally
- Local /var/log or Windows Event Viewer has no new entries since a specific timestamp
- Log files exist but timestamps are wildly out of order or off by hours
- Disk fills rapidly and the logging service crashes or refuses further writes
- Syslog receiver reports zero inbound events on UDP 514 despite active traffic
Common Causes
Logging daemon stopped or crashed
rsyslog, syslog-ng, journald, or the Windows Event Log service is not running. Without the collector active, nothing lands on disk or leaves the host, regardless of how the sources are configured.
Wrong forwarding target or malformed rule
A typo in /etc/rsyslog.conf, a stale IP after a SIEM migration, or an incorrect facility filter silently drops events. The service runs fine while messages go nowhere useful.
Firewall or ACL blocking syslog ports
UDP 514, TCP 6514 for TLS syslog, or 5985/5986 for Windows Event Forwarding are dropped by host firewalls, network ACLs, or cloud security groups between source and collector.
Clock drift breaking correlation
NTP is unconfigured or the peer is unreachable, so timestamps diverge across devices. Logs still arrive, but investigations become impossible because events cannot be ordered.
Storage exhaustion or rotation failure
logrotate is misconfigured, retention is too aggressive, or the log partition is full. Writes fail, the daemon may abort, and older archives are deleted before ingestion.
Step-by-Step Fix
- Confirm the log service is actually running on the source
On Linux run systemctl status rsyslog (or syslog-ng, systemd-journald) and check for active status plus a recent PID. On Windows open services.msc and verify Windows Event Log is running with automatic start. Restart the service and watch for immediate crash loops in the unit journal. - Send a synthetic test event to prove the local path
Use logger -p local0.info "ranknow-test $(date)" on Linux and confirm it appears in /var/log/messages or journalctl. On Windows use eventcreate /T INFORMATION /ID 999 /L APPLICATION /D "test". If the local write fails, the daemon or permissions are the problem, not the network. - Validate the forwarding configuration line by line
Inspect /etc/rsyslog.conf and any files in /etc/rsyslog.d/. A forwarding rule looks like *.* @192.0.2.10:514 for UDP or @@host:6514 for TCP. Run rsyslogd -N1 to check syntax before restart. For Windows, review the subscription in Event Viewer under Subscriptions and confirm the collector URL. - Verify the network path and open the correct ports
From the source run nc -uvz collector 514 for UDP or openssl s_client -connect collector:6514 for TLS syslog. Check iptables -L -n, nftables, firewalld, or Windows Defender Firewall for drop rules. Do not forget upstream ACLs on the switch, router, or cloud security group. - Confirm the collector is listening and accepting the source
On the receiver run ss -ulnp | grep 514 (or 6514) and tcpdump -ni any udp port 514 to watch traffic arrive. Many collectors filter by source IP or require the sender to be registered. Add the host, reload, and retest. - Fix clock drift with a working NTP source
Run timedatectl on Linux or w32tm /query /status on Windows. If offset exceeds a few seconds or the peer is unreachable, point chrony or systemd-timesyncd at a reliable internal NTP server, force a sync, and re-verify. Consistent time is what makes correlation possible. - Free storage and repair log rotation
Check df -h on the log partition and du -sh /var/log/*. If full, compress or archive old files, then review /etc/logrotate.d/ entries for the affected service. Ensure rotate count, size, and postrotate reload directives match the actual write rate. - Tighten permissions on log directories and sockets
Logs stop when the daemon cannot write. Verify ownership with ls -l /var/log and confirm SELinux or AppArmor is not blocking (ausearch -m avc). On Windows, check that the NETWORK SERVICE account retains write access to the Event Log files under %SystemRoot%\System32\winevt\Logs.
Common logging symptoms mapped to likely cause and first action
| Symptom | Likely cause | First action |
|---|---|---|
| No local or forwarded logs | Daemon stopped | Restart service and review journal for crash reason |
| Local logs fine, collector empty | Firewall or wrong target | Test port with nc, verify config target IP |
| Only some hosts missing | Per-host ACL or subscription | Check collector allow-list and source registration |
| Timestamps out of order | NTP drift | Reconfigure NTP peer and force resync |
| Service stops under load | Disk full or rotation broken | Free space, fix logrotate, restart daemon |
| TLS syslog fails | Certificate expired or untrusted | Renew cert, update CA bundle on both ends |
Prevention
- Monitor the log pipeline itself: alert when a host stops sending for longer than its normal cadence
- Standardise on TCP or TLS syslog for critical sources so delivery failures surface immediately
- Enforce NTP across all devices through DHCP option 42 or Group Policy
- Size log volumes for peak burst rates and test logrotate under load before production
FAQ
Should I use UDP or TCP for syslog forwarding?
UDP 514 is the historical default and is lightweight, but it silently drops messages under congestion. For anything you care about, especially security events, use TCP or TLS syslog on 6514. TLS also protects log contents in transit, which matters if audit or regulatory scope applies.
Why do my Windows Event Forwarding subscriptions show sources but no events?
This usually means the collector reached the source but the WinRM listener, subscription XPath filter, or channel access permissions are wrong. Run wecutil gr <subscription> on the collector and winrm enumerate winrm/config/listener on the source. Verify the collector's computer account has read access to the Event Log channels being queried.
How much log retention is actually required?
It depends on your regulatory scope and incident response needs, not on a universal number. Many organisations keep 30 to 90 days hot for investigations and longer cold storage for compliance. Confirm requirements with your compliance function and size storage against measured daily volume plus a comfortable buffer.
Need help stabilising your logging pipeline? Contact RankNow's IT support team for a structured assessment.