I mean the Librem 5 "works" in the way you describe.
I don't think you're going to really be able to reliably avoid the problem that very very very few people outside of AOSP are doing any work on OSS for phones.
An argument can be made that it's morally wrong, but if you're going down that road, there's a lot of things should be morally more compelling than Apple's AppStore policies.