Where the answers usually are.
THE IMPORTANT LOGS
- Web server access log: every request, with status codes
- Web server error log: application and server errors
- Authentication log: SSH and sudo activity
- System log: kernel and service messages
Database error log Mail log, if you run mail Application logs
WHERE THEY LIVE
Usually under /var/log, with locations varying by distribution and service.
READING THEM
Use tail to watch a log live while reproducing a problem. That is the fastest way to connect a symptom to a cause.
Use grep to filter for a specific address, status code or error string.
WHAT TO LOOK FOR
Sudden increases in 500 errors, which indicate application failures Repeated authentication failures, which indicate an attack Out-of-memory messages in the system log, which explain mysterious service failures One address making disproportionate requests
ROTATION
Configure log rotation or logs fill the disk. Confirm it is actually running; a misconfigured rotation is a common cause of a full disk.
RETENTION
Keep enough history to investigate something noticed a week later.