How the AI picks an article

The Knowledge Base retrieval engine ranks articles by running two parallel searches — keyword and vector — over article chunks, fusing the results with Reciprocal Rank Fusion, reranking the top candidates with Cohere, and deduplicating to the article level so each article appears at most once. This reference is a precise account of that pipeline.

Every AI agent in Atender — a Capability inside an Agent Stack, a Specialist Agent, Sidekick, the web chat or voice agent — asks this engine for articles, and the same engine powers the public help center’s search box. See Where Knowledge Base retrieval is used for the full list of call sites.

The high-level shape

A query arrives. The engine returns a ranked list of articles. In between:

1. The query is embedded into a vector.
2. Candidate chunks (not articles) are retrieved by two parallel paths — keyword and vector — over the published, non-archived articles in the tenant.
3. Chunks from both paths are fused with Reciprocal Rank Fusion (RRF) so a chunk that ranks high on either path floats up.
4. Cohere reranks the top chunk candidates by rereading the query and candidate text together.
5. Results are deduplicated to the article level — each article appears at most once, represented by its best-scoring chunk.
6. The top N articles are returned, with similarity scores and the matched chunk’s heading trail.

Outcome weighting is being developed as a future signal, but it is currently off and does not affect ranking.

What chunks are

Articles are split into small passages of a few hundred words. Each passage is embedded separately. A 4,000-word reference can have 10–15 chunks. Why: a long article covers many topics; one big embedding dilutes each topic’s signal. Chunk-level embeddings let “How do I refund?” match a refund chunk inside a Billing FAQ without being drowned out by the credit-card chunks.

The chunker has a version. When the chunker is improved, articles re-chunk in the background.

The two retrieval paths

• Keyword (Postgres tsvector) — Literal words, common stems — Catches exact terms — error codes, product names, brand names
• Vector (pgvector cosine similarity) — Semantic meaning — Catches paraphrases — “duplicate charge” matches “charged twice”

Each path returns its own ranked list of chunks. RRF combines them by rank position, not by raw score, which makes the fusion stable even when the two paths use different scoring scales.

Filters applied before ranking

Before any ranking happens, the candidate set is filtered:

• Tenant — Only the calling tenant’s articles.
• Status = published — Drafts, needs-review, and archived articles are invisible to retrieval.
• Embedding present — Articles whose embeddings haven’t finished generating are excluded until indexing completes.
• Category scope (optional) — Some callers pass fullCategoryIds and partialCategoryIds to scope retrieval to a subset of categories.
• Language (optional) — Customers in non-default languages prefer chunks in their language, with fallback to the default.
• Market (optional) — Customers browsing under a market prefer chunks tagged to that market, with fallback to global.

Roles do not appear in this list. KB roles control who sees what when browsing the public help center; the retrieval engine ignores them.

What does not affect ranking

• Article role assignments. Roles are browsing-visibility tags. The retrieval engine doesn’t read them.
• Tags. Tags are filterable in the in-app editor and on the public site, but they don’t change retrieval scores.
• difficulty / estimatedMinutes. Informational metadata; doesn’t change ranking.
• views counter. Tracked but not used as a retrieval signal today.
• lastReviewedAt. Tracked but not used as a retrieval signal today.

If you need to control which articles the AI retrieves, see Can I control which articles the AI uses?.

What boosts a chunk

The signals that do affect ranking, in rough order of impact:

1. Embedding similarity — vector cosine distance between the query and the chunk.
2. Keyword overlap — how well the chunk’s tsvector matches the query terms.
3. RRF position — chunks that rank well in either the vector path or the keyword path move up after fusion.
4. Cohere reranker score — the reranker rereads the top candidates and reorders them based on how well each chunk answers the query.

Outcome weight is a planned/WIP signal. It is intended to boost chunks from articles that have historically resolved similar conversations, but it is currently off and does not affect ranking today.

Title and summary get weighted higher in the keyword path. Keywords field is included in the keyword path, not the vector path — use it for synonyms and error codes you want matched without polluting the prose.

The retrieval mode

Atender now uses the chunked retrieval path for Knowledge Base search: keyword + vector search over chunks, fused with RRF, reranked by Cohere, then deduplicated to the article level before results are returned. There is no separate legacy-vs-chunked mode to choose from when authoring articles — every article is searched by both paths.

For what this pipeline means in practice when writing articles, see Write articles the AI retrieves well.

See also

• Where Knowledge Base retrieval is used
• Write articles the AI retrieves well
• KB search and AI retrieval — concept
• Can I control which articles the AI uses?
• Why is search not finding an article?