Retrieval-augmented generation (RAG) is a pattern where an application first retrieves relevant passages from its own documents, then passes them to a large language model (LLM) so the model answers from that material instead of relying only on what it learned during training. The retrieved text acts as grounding: it gives the model current, specific and checkable context, which reduces made-up answers and lets you point users to the source.

Why RAG matters

LLMs have a fixed training cutoff and no access to your internal wikis, tickets, contracts or code. RAG closes that gap without retraining the model. Typical reasons teams adopt it:

  • Answers must cite internal or private documents.
  • Content changes often, so re-indexing is cheaper than fine-tuning.
  • Auditors or users need to see which passage an answer came from.

How a RAG pipeline works

Most RAG systems share the same stages:

  • Ingestion: parse PDFs, Office files, web pages or code into text.
  • Chunking and indexing: split text into pieces and store them with embeddings in a vector index or another search structure.
  • Retrieval: find the passages that match a query, often combining vector similarity with keyword search.
  • Generation: give the retrieved passages and the question to an LLM and ask for a grounded answer.

Each stage can be a separate library or a single engine. Your choices depend on document types, freshness needs and how much control you want over the pipeline.

Categories of open-source tools

Full RAG engines

If you want a complete system with document handling and an answer layer, look at an engine rather than assembling parts. RAGFlow is an open-source RAG engine with agent capabilities that turns unstructured documents into a grounded context layer for LLM applications, licensed under Apache-2.0. Teams that want a ready-made document-to-answer path often start here.

Document and web ingestion

Retrieval quality depends heavily on clean input. crawl4ai is an open-source web crawler and scraper for LLMs and AI agents that converts websites into clean, LLM-ready Markdown; it is Apache-2.0 licensed and can be self-run. For code and engineering documentation, potpie builds a context graph for an AI-native software development lifecycle, which is useful when retrieval must follow relationships between components.

Retrieval strategies beyond plain vector search

Vector search is not the only option. PageIndex is a document index for vectorless, reasoning-based RAG under the MIT license, an approach that retrieves by navigating document structure rather than by embedding similarity. Compare it against embedding-based retrieval on your own corpus before committing.

Code-aware retrieval

For codebases, chunkhound is an MIT-licensed MCP server aimed at giving assistants a deep understanding of your engineering context. If your assistant works through the Model Context Protocol, browse the MCP servers catalog for other options.

Learning from examples

awesome-llm-apps is an Apache-2.0 collection of 100+ AI agents, agent skills and RAG apps that developers can clone and run. It is a practical way to see working RAG code before writing your own.

Applied answering and tutoring

Some projects show RAG in finished products. Vane is an MIT-licensed AI-powered answering engine, and DeepTutor, an Apache-2.0 project from HKUDS, is a lifelong personalized tutoring system. Both are useful to study for user-facing patterns, though check each one's fit before adopting it for production.

How to choose

Ask these questions before picking tools:

  • Document types: PDFs and scans need stronger parsing than plain Markdown.
  • Freshness: decide how often sources change and how you will re-index.
  • Retrieval style: standard embedding search, hybrid keyword and vector search, or structure-based retrieval.
  • Licensing: most projects listed here are Apache-2.0 or MIT, but confirm terms for your use case.
  • Operations: check whether you need to self-host, and what the project's activity and community look like before depending on it.

Getting started

  • Pick a small, representative document set and define 20 to 50 test questions with expected sources.
  • Begin with an end-to-end engine such as RAGFlow, or with an example from awesome-llm-apps, to get a baseline.
  • Measure whether retrieved passages contain the answer before tuning the language model.
  • Swap in a specialized component, such as crawl4ai for ingestion or PageIndex for retrieval, only where the baseline shows a gap.
  • Browse the developer tools and libraries catalog pages for SDKs that fit your stack.

Recommendation

For most teams, start with a complete engine like RAGFlow to validate the workflow, then refine ingestion and retrieval with focused tools. Evaluate retrieval quality on your own questions before you judge any RAG stack, since the best choice depends on your documents and how users ask about them. Explore the full AI tools and AI agents categories to compare options.