Files
44r0n7 0265afa054 chore: bootstrap lean sysadmin-chronicles repo
Import the runnable game code, content, docs, scripts, and repo guidance while leaving local agent state, dependency installs, build output, and backup copies out of the published tree.
2026-05-02 11:49:07 -04:00

44 lines
3.2 KiB
JSON

{
"id": "disk-logs",
"title": "Disk Space & Log Rotation",
"category": "storage",
"tags": ["disk", "df", "du", "logs", "logrotate", "cleanup"],
"updated": "2025-08-22",
"summary": "Finding what is filling the disk and keeping logs from growing unbounded.",
"sections": [
{
"heading": "Checking Disk Usage",
"body": "<p><code>df</code> shows you how full each filesystem is. <code>du</code> tells you where the space went.</p>",
"code": "df -h # human-readable filesystem summary\ndf -h /var/log # check a specific mount\n\ndu -sh /var/log/* # top-level breakdown of /var/log\ndu -sh /var/* | sort -rh # sort by size, largest first\ndu -sh /var/log/*.log # sizes of individual log files"
},
{
"heading": "Finding Large Files",
"body": "<p>When du does not point at an obvious culprit:</p>",
"code": "# Files over 100MB anywhere on the system\nfind / -xdev -size +100M -type f 2>/dev/null\n\n# Files in /var that have grown recently\nfind /var -xdev -mtime -1 -size +10M -type f 2>/dev/null"
},
{
"heading": "Emergency Cleanup",
"body": "<p>If disk is at 100% and a service is failing because of it:</p>",
"code": "# Truncate a log file without deleting it (safe for running processes)\ntruncate -s 0 /var/log/nginx/access.log\n\n# Remove old compressed logs (the .gz files are already rotated)\nrm /var/log/nginx/*.gz\n\n# Clear journald logs older than 2 days\njournalctl --vacuum-time=2d"
},
{
"heading": "logrotate Basics",
"body": "<p>logrotate is the standard tool for rotating and compressing logs on a schedule. It is usually run daily from cron. Config files live in <code>/etc/logrotate.d/</code>—one file per service.</p>"
},
{
"heading": "Writing a logrotate Config",
"body": "<p>Example for an nginx access log:</p>",
"code": "/var/log/nginx/access.log {\n daily\n rotate 14\n compress\n delaycompress\n missingok\n notifempty\n sharedscripts\n postrotate\n /bin/kill -USR1 $(cat /run/nginx.pid 2>/dev/null) 2>/dev/null || true\n endscript\n}"
},
{
"heading": "Testing logrotate",
"body": "<p>Run logrotate manually in debug mode to verify a config without actually rotating anything:</p>",
"code": "logrotate -d /etc/logrotate.d/nginx\n\n# To force a rotation right now (useful for testing):\nlogrotate -f /etc/logrotate.d/nginx"
},
{
"heading": "Key logrotate Directives",
"body": "<table><tr><th>Directive</th><th>Meaning</th></tr><tr><td><code>daily/weekly/monthly</code></td><td>Rotation frequency</td></tr><tr><td><code>rotate N</code></td><td>Keep N old copies</td></tr><tr><td><code>compress</code></td><td>gzip old files</td></tr><tr><td><code>delaycompress</code></td><td>Skip compressing the most recent rotation (useful when the app still has it open)</td></tr><tr><td><code>missingok</code></td><td>Do not error if the log file does not exist</td></tr><tr><td><code>notifempty</code></td><td>Skip rotation if the file is empty</td></tr><tr><td><code>size 100M</code></td><td>Rotate when file exceeds this size instead of on schedule</td></tr></table>"
}
]
}