This hasn't worked out because knowing what to query and how to query requires intelligence, and that's contained in weights.
To some extent we've already seen this with MoE and Frankenstein models.
This hasn't worked out because knowing what to query and how to query requires intelligence, and that's contained in weights.