What's wrong with recycling the bottle caps and not having them all over the place because they're small and otherwise fall onto the streets, sewers, and trails?? It's really not a problem to have the lid attached to the bottle - works totally fine for me whether it's milk, soda, or apple juice.
I'm not OP, but to me, the bureaucratic issue is that plastic bottles should be wholesale banned instead of this halfass attempt at regulation or heavily purchase-controlled.