What you gain and lose.
WHAT YOU GAIN
Speed, substantially Security, since there is little to attack Low resource use Simple backups, since the files are the site No plugin maintenance
WHAT YOU LOSE
Immediate publishing Easy editing for non-technical people Plugins Built-in search, comments and forms Anything server-side
WHAT THE LOSSES COST IN PRACTICE
The editing one, mostly.
If the business owner cannot update the site, they will not update it.
A site nobody updates stops being useful.
WHAT CAN BE ADDED BACK
Forms, through a service Search, through a script or service Comments, through a service
Each adds a dependency and frequently a third-party script.
THE HIDDEN COST
By the time you have added forms, search and an editor, some of the simplicity is gone.
THE QUESTION TO ASK
Who will edit this site in two years, and can they?