I'm now experimenting with Open WebUI and local bge-m3 (embeddings), bge-reranker and Gemma-4-26b (instruct). I'm slowly learning how to integrate and test RAG systems. It's not so easy. If the instruct model is too eager to please it's not so clear if it uses the provided sources at all. I test with a rather "outdated" instruct model, llama-3.1-9b. It doesn't have enough stored information to "decode" more than a re-hash of the prompt.
My current test set includes Nancy Leveson "Engineering a Safer World", Robert Rosen "Essays on Life Itself" and Enrico Martino "Intuitionistic Proof Versus Classical Truth". There are lots of structural similarities between "Systems Safety" and Rosennean (M,R)-systems. The connection to Brouwer is more subtle.
My takeaway on RAG: people who just consume such solutions will have a hard time spotting shortcomings.
Edit: after a lot of tuning my homelab RAG system is now useful.
#homelab #RAG #vectordatabase