Web workers won't work in this case because they need to serialize all data going into and out of them (with the exception of TypedArrays).
So passing a string to a worker and having it JSON.parse it works great. But when you go to pass that object back to the main thread, it implicitly does a JSON.stringify and a JSON.parse back on the main thread (technically it's called a "Structured Copy", but it's mostly the same thing), putting you in the exact same situation.
And thanks to you for helping show me that I wasn't the only one to try that!
This whole thread has been really nice to read, because I beat my head against a wall for a long time before I finally found a solution, and I'm glad to read that I wasn't the only one to think this was a lot more deceptively hard than I thought at first thought (or second, or third...)
If this really NEEDS to be a client-side-only solution, I still believe a worker is the only way to go. Only, in this case, it needs to behave like an API. So your worker not only parses the JSON, but also responds to post messages with only the data that is requested from it.
Why? Because a large JSON structure is most probably just a large JSON structure, but you most probably don't need it as a whole. You may need a total count of items, you may need a paginated set of items, or only a certain item or a set of fields of items — well, an API.
> it implicitly does a JSON.stringify and a JSON.parse back on the main thread (technically it's called a "Structured Copy", but it's mostly the same thing)
Except funnily enough JSON.stringify + JSON.parse is usually recommendation as it's either comparable or faster than the structured copy the engine itself does :/
Example fibonacci worker code that doesn't block the UI, even at larger calculations
[1] https://github.com/bharathnayak03/react-webworker-hook