My Blog

Local RAG

I get interested in RAG since I implemented some slack bot for our internal support channel. I always want to fully understand what it is, especially what is "embedding". I found this video is probably the best resource for me to try something locally.

I end up with something similar to the video, a 100 line python file that implements the RAG, messy but works. And the "embedding" is just a mapping from words and sentences to a high dimension space (as vectors, or just points in that space). It is used for getting relevant documents, basically as a ranking algorithm. Essentially you can have a RAG without embeddings, any ranking algorithm will do. But embedding probably yields a good result? I don't know. I trust the IR guys to do their job.

What I really want is for the RAG to consume all my local epubs, so I can ask questions about these books. Maybe I can have a virtual author that talks to me, and teach me. That would be really fun.

Also, I was using gemini a lot for this task, can only recommend it.

links

social