I used FlowWise[1], LM Studio[2], the llama2[3] model, and Ollama[4] (for embeddings) to create a local-only RAG chatbot so I could chat directly with Tristram Shandy, Gentleman[5]. For the context document I used the text of the novel of the same name, downloaded from Project Gutenberg.
Primarily it was a PoC to see if a document based chatbot could work without crossing trust boundaries by calling out to untrusted APIs. It only makes calls to localhost.
If you’re familiar with the novel you will be pleased to know that the chatbot ended a recent answer with, “I must go now as I have an appointment with my chamber pot and I wouldn’t want to keep it waiting.”
Primarily it was a PoC to see if a document based chatbot could work without crossing trust boundaries by calling out to untrusted APIs. It only makes calls to localhost.
If you’re familiar with the novel you will be pleased to know that the chatbot ended a recent answer with, “I must go now as I have an appointment with my chamber pot and I wouldn’t want to keep it waiting.”
[1]https://github.com/FlowiseAI/Flowise
[2]https://lmstudio.ai/
[3]https://llama.meta.com/
[4]https://ollama.ai/
[5]https://www.gutenberg.org/ebooks/1079
Everything runs on a Mac Mini with the M2 Pro CPU/GPU and Mac OS Sonoma.