Yes, using natural gas for the things that only natural gas can do would be an excellent public policy.
However that's not the policy we are pursuing. We are pursuing very blunt instruments to try to discourage investment in developing natural gas fields full stop. Which is bad public policy.
The other issue is by raising the price of natural gas, we are raising its price for all uses. And the price of natural gas tends to determine the price of fertilizer, which tends to determine the price of food. So this policy makes food more expensive.
Even worse, natural gas and things like coal and oil are in some sense substitutes. Not perfect substitutes, but what happens when we make oil more expensive is that people shift to coal and natural gas to generate energy, and so these two become more expensive as well. Similarly, if we were take all coal offline, that would increase demand for natural gas and oil as substitutes, and again, the price of these commodities would rise, which would increase the price of food.
What would be nice would be to see if we could de-couple these prices somehow -- e.g. tax usages of natural gas downstream of the price of the commodity itself. Other ideas are also possible.