I have a couple sites on Slicehost and I love it. The only thing I miss from shared hosting sites is the easy to use management console for setting up email and such. Other than that Slicehost is the way to go.
A pet feature I wish they had was exposing a way to download the actual Xen image your site is hosted within. Then you could be developing your production server sans internet, and rsync the resulting image.
I generally hate those consoles, as they make things work in unexpected ways. Recently, I spent a while tracking down a bug on a managed VPS with Plesk installed where mail sent by scripts on the server to the same ___domain (let's call it example.com) were going to local delivery at example.com instead of the MX entry of exchange.example.com (a separate server).
Many times, I want the email system to share its login database with some web app anyway.
What I would really love to see is affordable hosting like this but with the ability to do stuff like run a custom web server. I know this thread isn't really about hosting but has anyone seen something like this or is it even technically possible without going fully dedicated?
