Secure sites can be faster, not slower.
THE OLD CONCERN
Encryption adds processing overhead, so https was assumed to be slower.
THE CURRENT POSITION
The overhead is negligible on modern hardware. More importantly, HTTP/2 requires https in every browser, and HTTP/2 is substantially faster than HTTP/1.1 for typical pages.
WHAT HTTP/2 IMPROVES
Multiple resources over one connection rather than several Header compression Better handling of many small files
A page with many images and scripts benefits considerably.
WHAT THIS MEANS
Enabling https can make your site faster, not slower, because it unlocks HTTP/2.
WHAT STILL SLOWS A SITE
No caching Large unoptimised images Too many plugins Redirect chains, including the http to https hop on every internal link
That last one matters: update internal links to https rather than relying on the redirect, or every internal navigation costs an extra round trip.
CHECKING
An online SSL or performance test reports whether HTTP/2 is in use.