GPT-4. I tried with GPT-3, because GPT-4 is so slow, but it failed to reliably return JSON responses (perhaps it would work with better prompts?)

You should take a look at guidance [0] to help with JSON returns from LLMs. Have been using it for some side projects and it seems to work pretty reliably.

[0] https://github.com/microsoft/guidance

Looks super handy, thank you!