If that’s the problem, then yes, you’d have to edit Apache’s configuration files, which is something your server admin would have to do (which, keep in mind, on a VPS may very well be you, and many hosts do not provide assistance on self manage servers). If you can access EE via the IP address, it’s certainly not EE, as EE doesn’t control anything at a server level - your browser can either make a request to the index.php file or it cannot. Can you access any files via the root domain? For instance, if you upload a JPEG file to the same folder as your ExpressionEngine index.php, can you access it at http://example.com/image.jpg?
And do you know where that HTML file that’s being output is located? It’s not a file provided with EE, and you can make sure it’s not an EE template by searching your templates (left hand side of the Template Manager) for the word ‘default’ to see if that HTML content is in any of your templates.
One other possibility is a bad .htaccess file; if you are using one, you may consider removing it for the time being until you get this sorted out.
As an aside, any particular reason you switched to a dedicated VPS?
Summary:
1) Can you access a newly uploaded image file placed in the same folder as EE’s index.php?
2) Is that HTML content in an EE template, or do you know of an HTML file on the server with that content?
3) Are you using .htaccess, and if so, does removing it help?
4) Optional (:😁:) - why did you switch to VPS?