Open source models in general. Meta has for instance released DINO which is a self supervised transformer model. LLMs are also going multi modal (see LLaVA for instance). The name "LLM" has stuck but they should really be called Large Transformer Models. LeCun is working on self supervised visual world models (I-JEPA) which if successful and released could form the basis for killer drones. It's still a lot of engineering work to fine tune and put a model like this on embedded hardware on a drone, but at some point it might be easy enough for small groups of determined people to pull it off.
For a drone, an LLM derived solution is far too slow, unreliable, heavy and not fit for purpose. Developments in areas like optical flow, better small CNNs for vision, adaptive control and sensor fusion are what's needed. When neural networks are used, they are small, fast, specialized and cheap to train.
A multimodal or segmentation algorithm is not the solution for bee-level path planning, obstacle avoidance or autonomous navigation. Getting LLMs to power a robot for household tasks with low latency to action and in an energy efficient manner is challenging enough, before talking about high-speed, highly maneuverable drones.
Tesla is running these models on 4 year old hardware to control a car in real time (30 fps). You don't need a full 100B model to control a drone, and it doesn't have to be as good as a car to cause a lot of damage. Reportedly both Ukraine and Russia are putting together on the order of a thousand drones a day at this point, Tesla includes the compute to run this in every car they make already today. Hardware is also moving fast, how come people forget about Moore's law and software improvements? To me there's no question that this tech will be in tens of thousands of drones within a few years.
