Oh I never realized I was supposed to install -dev packages, I thought I had to compile myself.

Whether you need -dev packages (for headers) depends on your operating system. I run Gentoo. All headers are always on the system. Other distributions ship the headers separately in -dev packages to save space and you need to install them. You likely can install all -dev packages for everything on your system so you do not need to get the individual ones.