So, a funny thought experiment is what happens when you parse a JSON file at the same time? You also index by primary key (the field name).
So, I mirror this thinking and having be just an object with the keys being the primary key. Then, I simply index all the children by their fields based on insights from the developer via the index keyword.
So, if you have
record R { public int id; client int owner; int age; index age; }
table<R> rows;
then queries for age can be accelerated by the table.
like "iterate rows where age==42" will basically hone in on the bucket of age==42. I currently only index clients by hash and integers.
The critical aspect which makes this work is that I monitor all mutations. When a child object has a field mutated, then it is removed from all indices and placed into an unknown index. Any queries will also consider it as the purpose of queries to simply narrow the field. Once data changes are persisted, the index is updated and items are moved out of the unknown bucket. This works fairly well because the indices are primarily used during the privacy check phase.
So, I mirror this thinking and having be just an object with the keys being the primary key. Then, I simply index all the children by their fields based on insights from the developer via the index keyword.
So, if you have
record R { public int id; client int owner; int age; index age; } table<R> rows;
then queries for age can be accelerated by the table.
like "iterate rows where age==42" will basically hone in on the bucket of age==42. I currently only index clients by hash and integers.
The critical aspect which makes this work is that I monitor all mutations. When a child object has a field mutated, then it is removed from all indices and placed into an unknown index. Any queries will also consider it as the purpose of queries to simply narrow the field. Once data changes are persisted, the index is updated and items are moved out of the unknown bucket. This works fairly well because the indices are primarily used during the privacy check phase.