Turns out you can yeah! I'm not sure the quality would be anything useful for imaging applications though. But for lighting / non-imaging stuff go for it!
It's not that hard to find decent lenses. I bought a lens with a long focal length and wide aperture from surplusshed.com and installed it into a PVC tube, then attached my camera to it (no lens). I used it for solar observation- a few times, the focused light ended up melting the tube, part of the camera, and anything else it came in contact with.
But it makes more sense to buy a cheap but decent refractor or reflector unless you are truly invested in DIY. there's a lot of fiddly details to get right to get decent image quality.
