A dedicated server will likely cost you more than $100 per month; it's definitely not cheap web hosting. The benefit? Your website lives on a server all by its lonesome, so it takes advantage of the server's full resources. You'll probably need to handle firewalls, updates, and maintenance yourself, however, unless you opt for a managed server, which costs even more money.
If privacy is your number one concern, perhaps you should go with Protonmail. You can upgrade to an email with your domain easily on a web interface under “Protonmail Plus” or “Protonmail Visionary” plans. However, the plans can be rather expensive, as getting only 20 GB storage space could cost you at least €24/user/month/yearly. Most email hosting providers retain your emails for months even after you delete your account. This Swiss email provider will destroy all your data when you delete your account.