The basics, in priority order.
THE ESSENTIALS
SSH keys rather than passwords, with root login disabled A firewall denying everything except what you need System updates applied promptly Services bound to localhost unless they must be public Applications running as unprivileged users Backups, stored off the server, and tested
THE NEXT LAYER
fail2ban or equivalent, blocking repeated failed attempts Intrusion detection, alerting on changed system files Log monitoring, so you notice unusual activity Regular malware scanning
WHAT MOST COMPROMISES EXPLOIT
Unpatched software with a known vulnerability Weak or reused passwords on SSH or a control panel A vulnerable web application, which is then used to reach the rest of the server An exposed database or admin interface
THE UNCOMFORTABLE TRUTH
An unmaintained VPS is less secure than shared hosting, because on shared hosting someone else is patching the operating system.
If you cannot commit to ongoing maintenance, buy managed, or stay on shared hosting.
WHERE TO START
The first-hour checklist. Everything else builds on it.