AI-based knowledge management system: how it works, buy or build, and what it costs

An AI-based knowledge management system (AI KMS) answers questions in plain language from your own documents, for employees or for customers inside your product. It shows where each answer came from and uses only the documents the person asking may open. At Ronas IT, the smallest version is an assistant over one set of documents inside a tool your team uses today: from $3,000, with a week as the shortest timeline. AI search and answers across several of your systems are priced the same way, per AI feature, with the connections to your systems' APIs included.

This guide is for operations, support, and IT leads who want staff to stop searching several tools for one answer. It explains how these systems work, when a ready-made product is enough, and what a custom build includes. Our AI software development page describes the service itself.

What an AI-based knowledge management system is

A knowledge management system is where a company keeps what its people know: guides, policies, product details, past decisions, and answers to common questions. An AI-based one adds a layer that reads this content and answers questions about it. Instead of a list of documents that match some keywords, the person gets a short answer with links to the passages it is based on.

The storage, editing, and approval parts stay the same as in any knowledge management system. Our guide to enterprise knowledge management systems covers their basic features, such as knowledge capture, storage, and access control. AI changes how people find and use the content:

• Search by meaning. A question like “can I expense a taxi to the airport” finds the travel policy even if the policy never uses the word taxi.
• Answers with sources. The system writes a few sentences from the passages it found and links each statement to its document, so the reader can check it.
• One place to ask. A single question can cover the wiki, shared drives, the ticket system, and chat archives, if each of them is connected.
• Help with upkeep. The system can suggest tags and summaries for new documents and show which questions found no answer, which tells content owners what to write next.

How AI knowledge management works

The system described here first finds the passages in your documents that answer the question, then has a language model write the answer from those passages. Each statement can then be checked against its source, and a change to a document shows up in answers after the next scheduled indexing. This design is called retrieval-augmented generation (RAG).

The term comes from a 2020 research paper by Patrick Lewis and colleagues, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, which describes RAG as combining what a model learned in training with passages retrieved from an index. The system in this article instructs the model to answer only from the retrieved passages instead.

Steps 1 and 2 run ahead of time, and steps 3 to 6 run for each question:

1. Connectors read documents from each source on a schedule, together with the list of people and groups who may open each one.
2. The indexer splits each document into passages of a few paragraphs and turns every passage into an embedding, a list of numbers that describes its meaning, stored in a vector database.
3. The permission filter removes every passage the person asking is not allowed to see before anything reaches the model.
4. The search compares the question with the remaining passages by meaning and by keywords and picks the best few.
5. The language model writes a short answer from those passages and marks which passage supports each statement.
6. The interface shows the answer with links to the sources and lets the person mark it as right or wrong. Those marks, and the questions with no answer, go to the content owners.

Fine-tuning, which means training a model further on your documents, is a different method. It can change how a model writes, but it cannot show which document an answer came from or hide a document from one person and show it to another, and it has to be repeated when documents change. For a knowledge system, retrieval does that work.

Permissions are the part that is easiest to get wrong

An assistant that answers from a document the person could not open is a data leak, even if the answer is correct. The assistant has to apply the same access rules as the source system. Commercial products work this way too: Microsoft's documentation says Microsoft Copilot “only surfaces organizational data to which individual users have at least view permissions.” In a custom build, the connectors copy the access lists together with the documents and update both when either changes.

What it helps with, and where it does not

An AI KMS helps when the answers already exist in writing but are hard to find. It does not help when the knowledge is only in people's heads, when the documents disagree with each other, or when the question needs live data from another system, such as today's stock level.

Good first uses have many repeat questions and clear written sources:

• Support teams looking up product details and past solutions while a customer waits.
• New employees asking about policies, tools, and who owns what during their first months.
• Sales teams looking for a past project, a client review, or a technical answer for a proposal.
• Engineering teams searching architecture notes, coding standards, and runbooks.

We use this ourselves. Our sales and marketing teams put plain-language questions to an internal AI search over the case studies on our website and the design work we publish. The index updates every day, so newly published work shows up within a day. Our public reviews on Clutch and GoodFirms and our technology radar can be searched the same way. One question it answers is which of our past projects match a new request.

Buy, extend, or build

Buy a ready-made product when your documents are spread across tools it already connects to and its access rules match yours. Extend what you have with a custom assistant when one wiki or document system is fine and only its search is weak. Build when the knowledge sits in your own software or when the assistant is part of a product you sell. If data cannot leave your servers, run the model and the rest of the system there too, with a self-hosted platform or a custom build.

Ready-made options include Glean, Guru, Notion AI, and Microsoft Copilot. The business plans of general AI assistants now connect to company documents too: ChatGPT Enterprise, Claude Enterprise, and Gemini Enterprise read from tools such as Google Drive, Microsoft 365, and Slack. A ranked list of such tools helps less than it seems, because the right one depends on where your documents already live. If one of them fits, we will tell you.

“Before you sign anything, take the questions your team really asked last month and run them through the product with your own documents. Demo questions are chosen to work. The questions about your exceptions tell you whether it fits.”

Roman Surikov, CEO at Ronas IT

How a custom AI knowledge management system is built

A custom build turns the six steps above into software you run, and most of the work is in a few choices: which sources to connect first, which index and models to use, where people ask, and how you check that answers stay right. The first release can connect one or two sources for one team and add the rest once answers are good. The connectors work as steps 1 and 3 describe, and the other parts are:

• Search index and embedding model: where the passages, their embeddings, keywords, and access lists live, and the model that turns passages into embeddings. Both choices are explained below.
• Language model: a hosted model through an API, such as OpenAI GPT, Anthropic Claude, or Google Gemini, or an open model on your own servers, served with a tool such as vLLM.
• Interface: a chat in the browser, in a mobile app, or inside tools people already use.
• Quality checks: a set of real questions with known answers, run after every change, and traces of each answer, so a wrong one can be followed back to the passage that caused it.
• Admin area and logs: where content owners add sources and read feedback, and a record of every question and answer.

Two of our public projects are built for Open WebUI, a self-hosted AI platform whose knowledge bases let people ask questions across their documents and files. The first, Open MobileUI, is an iOS and Android client, so staff can reach a self-hosted assistant from their phones.

The second is an MCP server for Open WebUI knowledge bases. It uses MCP, the Model Context Protocol, an open-source standard for connecting AI applications to external systems, so assistants such as Cursor and Claude Desktop can search those knowledge bases. Each connection uses its own API token.

We have also built an assistant into a client product: on a plant marketplace for the UAE, we integrated the OpenAI API into an AI assistant. We built it so that buyers could ask about a plant from that plant's page or open a separate chat to find suitable plants, and so that it suggested alternatives when a plant was not available. It matches the last row of the buy, extend, or build table: an assistant that answers customers inside a product.

Choosing the search index

Search by meaning can miss exact terms such as product codes, error messages, and names, so the search in step 4 also matches keywords. This is called hybrid search: Elastic's documentation describes it as full-text and vector search run in one request that returns one ranked list. Three kinds of index can do this:

• PostgreSQL with pgvector. The pgvector extension stores embeddings in the same database as the rest of the system, so there is no extra service to run and back up, and its documentation pairs it with PostgreSQL full-text search for hybrid search. It is marked Adopt on our technology radar, and we choose it by default when the product already runs on PostgreSQL.
• Elasticsearch or OpenSearch. Search engines built for keyword search that also store and search vectors. OpenSearch has hybrid search too. They fit large and varied document collections, cases where keyword search quality matters as much as meaning, and companies that already run one of them.
• A dedicated vector database. A separate service for embeddings, such as Qdrant, whose hybrid queries merge several searches into one result list. It fits when vector search has to grow separately from the main database.

With any of them, a reranking step can sort the best results again before they reach the language model. Elastic describes semantic rerankers as models that “reorder search results based on their semantic similarity to a query.”

Building a knowledge base from PDFs, scans, and Office files

Some of the documents a company wants to search are not plain text: PDF contracts, scanned forms, slide decks, and spreadsheets. Before the indexer can split them into passages, the text has to be extracted. On one client project, we built this with Apache Tika, Elasticsearch, and the Gemini API. Such a pipeline has four steps:

1. Extract text and metadata. Apache Tika detects and extracts metadata and text from over a thousand file types, such as PPT, XLS, and PDF.
2. Read scans. Tika can pass images to the Tesseract OCR engine when it is installed. For scanned PDFs, Tika has to be configured to run OCR on the images inside the file or on each rendered page.
3. Index. Elasticsearch stores each passage with its metadata, access list, and embedding, so one index serves keyword search, search by meaning, and the permission filter. Elasticsearch can also call Tika itself through its attachment processor.
4. Embed and answer. The Gemini API offers embedding models for the passages and language models for the answers. Gemini can also read a PDF directly: Google's documentation says its models use native vision to understand text, images, diagrams, charts, and tables in documents of up to 1,000 pages. This helps with pages where a table or a diagram carries the meaning.

The embedding model is hard to change later

Embeddings made by different models cannot be compared, so the question and every passage must be embedded by the same model. Changing the embedding model means embedding the whole collection again before search works. Google's embeddings documentation says it about gemini-embedding-001 and gemini-embedding-2: their embedding spaces are “incompatible,” and anyone upgrading “must re-embed all of your existing data.”

The change is not always your choice. Providers retire embedding models: OpenAI shut down its first-generation embedding models on January 4, 2024, and Google shut down text-embedding-004 on January 14, 2026 and lists May 14, 2028 for gemini-embedding-001. An open embedding model on your own servers is not retired by anyone, but you still embed everything again when you move to a better one. We design the architecture so the models can be swapped, and moving the assistant to a new model can be part of our support.

The model fee is the easy part to estimate. At OpenAI, text-embedding-3-small costs $0.02 and text-embedding-3-large $0.13 per million tokens, and its batch mode takes 50% off. The costs that are easy to miss sit around the fee: running text extraction and OCR again if the extracted text was not kept, storing two indexes during the switch, testing answer quality again, and the engineering time. A build can keep them small:

• Keep the extracted text and the passage boundaries, so a new embedding run repeats step 2 and nothing before it.
• Store the model name and version with every vector, and never mix two models in one index.
• Build the new index next to the old one and switch when it is ready. Qdrant describes building “a second collection in the background” and then moving an alias to it, and Elasticsearch aliases let you reindex data without downtime.
• Compare the two models on your question set before people see answers from the new index.

Risks and how to handle them

The main risks are wrong answers, answers from documents the person should not see, instructions hidden in documents, outdated content, and a forced change of the embedding model. Each can be reduced, and each needs an owner on your side after launch.

• Wrong answers: test with real questions before launch, show sources with every answer, and have the assistant say it does not know when no passage fits.
• Permission leaks: filter passages by the asker's access before the model sees them, and test with accounts from different teams.
• Instructions hidden in documents: a document can contain text written to change what the model does. Prompt injection is the first entry in the OWASP Top 10 for LLM applications, and OWASP calls the kind that arrives through files indirect prompt injection. The assistant should treat documents as data, and any action beyond answering should need a person to confirm it.
• Outdated content: give every source an owner and a review date, and keep each document in one place where people edit it. Our marketing AI agents read a knowledge base generated every day from the wiki our team already edits, and each file in it links to the wiki page it came from. Nobody updates a second copy for the AI by hand, so the two do not drift apart.
• A forced change of the embedding model: when the provider retires the model, every document has to be embedded again. Plan for it at the start with the steps in the embedding model section above.

“When an answer is wrong, check the passages the search found first and the model second. An old version of a document or a page about a similar tool leads to a wrong answer however good the model is. That is why we test the search on its own before we tune how the answers are worded.”

Alexander Storozhevsky, Lead developer at Ronas IT

When an assistant can also act, such as creating a ticket or changing a record, it becomes an AI agent, and the risks grow. Our guide on how to build a custom AI agent covers that step.

What a custom AI knowledge management system costs

A custom build costs from $3,000, and the price depends on how many sources you connect, how complex their access rules are, and whether you need a new place to store documents. Our starting prices for the parts of such a system:

Each build price is complete: design, project management, testing, and the connections to your sources are already in it, so you pick the one build that matches your case. An analysis phase before it is optional. If the model runs on your own servers, setting them up is part of the build estimate, and keeping them running afterward starts at $200 per month.

After launch you also pay for hosting the index and the app, and for model usage, which depends on how many questions people ask and how long the passages sent with each question are. With a model on your own servers, the usage fee becomes the cost of those servers.

What to do next

1. Collect a month of questions from one team, with the documents that answer them.
2. List the sources those documents live in and who may open each one.
3. Find your row in the buy, extend, or build table above.
4. Name an owner for the content and for the answer quality after launch.

Tell us which team the assistant is for and where its documents live, and get a first estimate and a plan for the analysis phase. We reply within 24 hours.

Get an estimate