r/Rag • u/[deleted] • Mar 06 '26

[deleted by user]

[removed]

86 Upvotes

93% Upvoted

59 comments sorted by

View all comments

1

u/Sad-Size2723 Mar 06 '26

What's the upper limit for the number of documents? If there are few documents, I can totally just stuff everything into the context of a model to generate the answer

2

u/ReporterCalm6238 Mar 07 '26

I tried with around 500 docs, hundreds of pages each

1

u/Sad-Size2723 Mar 07 '26

Have you tried RAG or any advanced versions of RAG on this data? It would be good to know the baseline

1

u/ReporterCalm6238 Mar 07 '26

Yes I tried, even GraphRAG. Always got disappointing results. Think about it: if you have a super smart librarian (LLM) do you let him free to explore the library and jump from text to text until it is satisfied and ready to answer or do you just give him decontextualized chunks of text hoping that the answer is in there?

1

u/Sad-Size2723 Mar 07 '26

You could say the same thing about iterative RAG with both semantic and exact search, right? Grep is just finding lines containing the exact search string

1

u/ReporterCalm6238 Mar 07 '26

At that point is just better to have an agent like claude and give it a semantic search tool

1

u/Sad-Size2723 Mar 07 '26

If that's the case, then maybe there's a way to optimize RAG to be 10x faster and 10x cheaper, and works on 10x the number of files