okay fair, the actual ui is improved then on the screenshots, so i will be updating them asap

yes, i can see that, they are fun to play with though, many of the responses are interesting, and yes they will get more powerful fast, so swapping for another model will be possible and soon i will support this

how much would you pay?

thanks! move fast and ship is my mantra

Love the spirit! App looks very cool, keep up the great work :)

oooh thanks for the tip, will try this

Stable LM 3B Zephyr, it's the only model below 7B that can handle RAG: i.e. understand "hey those are documents, use them to answer these questions"

It'll work too, it was quite delightful to open Test Flight, install my Flutter app not designed for Vision Pro at all, and everything "just worked".

https://stability.ai/news/stablelm-zephyr-3b-stability-llm works absolutely fine on the M2 processor, like 40 tok/s https://x.com/EMostaque/status/1732912442282312099?s=20

Stable LM 2 1.6b runs even faster but not as good at RAG, great multilingual though, we are seeing it matching 70b models on other languages (new version soon) https://x.com/EMostaque/status/1763269238347673796?s=20

Can fit a lot in a gigabyte file it seems.

Is this Flutter app something you created? If so, is it open source? I’m in that same space and I generally just like to learn from other people’s work.

If not, all good. I don’t have a Vision Pro myself but I got a similar app which runs on all platforms including iPadOS, thus I guess my app should work on that too. Thanks for the reminder!

Thanks for asking: Yes I did make it, but, no app tying it all together. At least, it isn't out yet.

The grunt work of getting it running on different platforms + nice easy OpenAI compatible interfaces x RAG x voice assistant is open source:

- FLLAMA: https://github.com/Telosnex/fllama llama.cpp at core, openai compatible API, function call support, multimodal model support, Metal support. All platforms incl. web, but WASM is slow, def. not worth it except as a proof of concept.

- FONNX: https://github.com/Telosnex/fonnx ONNX runtime at core, all platforms including web. Whisper, Silero VAD, Magika, and two embeddings models. (Mini LM L6 V3 is best for RAG)

EDIT: I knew I recognized your username! Aub.ai! Cheers, what you did with aub.ai convinced me it was possible to do llama.cpp in flutter with a high bar for engineering quality. Other stuff seemed a tad rushed, unstable, and not complete. Also congrats, just saw your recent update, been hoping something good came through and it did.

yes i'm working on this 3D avatar idea as well. it's actually really mind blowing in my opinion, just need to bring your own imagination. this is just the start, i will add memory, RAG, voice interface, and other features to this.

This is the future of computing. Especially when tech like vision pro becomes the size of normal sunglasses.

I've watched both movies. Her was an audio only chatbot and this is already doable. SillyTavern + OpenAI Whisper + Silero TTS and you've basically got Her. I've already done it and it works quite well, Whisper is much much better than the speech recognition Google offers even when running locally on a CPU.

Ex Machina was an actual physical robot. Not possible yet, but since GPTs became smart, huge investments are being made in robotics, the most recent annoucement today: https://futurism.com/the-byte/humanoid-robot-maker-deal-open...

Once this happens a robot will be basically able to do any job a human can do.

Cool... My actual point was, Huma-Droid Relations:

Her: Human foolishly falls in love with an AI bot (already happened in real life)

E.M: AI bot gets a body, lies her way out of prison and releases her self on society (GPt already lied its way through Mechanical Turk Captchas.)

Point being, that the 3D avatar will be like the all the AI warning we have of Holographic Personal AI Assistants... and some people will fall in love with them... and some of the assistance will either be/be used for Evil...

:-)

I didnt doubt you had seen them, though.

re: GPT x bot:

Absolutely not.

I'm as high as a kite on this stuff and have to be, but I'm not sure you're actually using ex. vision API.

Also, Whisper isn't lower WER than Google unfortunately or even close, and that I know for a fact, I designed & implemented both the server/client side of the last big Assistant audio format change, and also the UI for the New Google Assistant™, i.e. Google's first offline model.

Whisper is still really good, even Whisper Tiny, and I'm happy to ship it.

oh super intersting projects, SillyTavern and SileroTTS indeed i believe Her is possible now

yes, i have seen these, i believe we will co-evolve with ai, so that our definition of being human will evolve rapidly, and there will not be a threat from Ai, but rather, we will become more and more powerful

I write about web3, tech, coding and journalism on being a digital nomad. https://substack.com/@0XMAKERETH