I was actually really impressed with the Stadia lag when playing Destiny using a Chromecast and the Stadia controller. I saw image compression but the lag and framerate were very good. Not just playable but good.
That then kills the "no hardware" argument though.
Afaik the Stadia controller talks to the service directly and not to the chromecast. Any other normal setup will be more laggy than the Stadia Controller + Chromecast setup.