Most retrieval problems are not embedding problems. They are schema problems wearing an embedding costume.

This is an unpopular thing to say, because embedding models are interesting and schemas are not. There is a new model on the leaderboard every few weeks, each one with a benchmark chart showing it edging out the last. There is no leaderboard for deciding whether your knowledge articles should be chunked by section or by paragraph. One of those two decisions will determine whether your agent answers correctly. It is not the one with the chart.

We have now debugged enough retrieval systems inside Salesforce orgs to notice that the failures cluster. They cluster in the same three or four places, and none of those places is the model. What follows is what we actually check, in the order we actually check it.

The symptom and the actual cause

The reported symptom is almost always the same sentence: the agent gave a wrong answer even though the right information was in the system. Someone can point to the exact article, the exact field, the exact record. It was there. The agent did not use it.

The first instinct is to reach for the model. Swap the embedding model, raise the top-k from five to twenty, add a reranker. Sometimes this appears to work, which is the worst possible outcome, because it teaches the team that retrieval quality is a dial rather than a design.

What is usually happening instead falls into a small set of causes:

• The chunk boundary cut across the thing being asked about, so no single chunk contains a complete answer.

• The metadata that would have narrowed the search was never written, so the search is ranking across the entire corpus when it should have been looking at eleven records.

• Two documents that mean genuinely different things were flattened into nearly identical text, so the model cannot distinguish them and neither can the index.

• The right document exists but is superseded, and nothing in the system marks it as retired.

Notice that a better embedding model helps with none of these. A reranker helps slightly with the third and not at all with the others. The fixes are upstream, in how the content was prepared and described before it ever reached a vector.

Chunking is a schema decision

Chunking gets treated as a preprocessing detail, something you configure once with a character count and never revisit. In practice it is the single highest-leverage decision in the whole pipeline, because it determines what a complete thought looks like to your system.

Splitting on a fixed character count is convenient and almost always wrong. It is wrong in a specific and predictable way: it severs the relationship between a heading and the paragraph that explains it. The heading carries the topic. The paragraph carries the detail. Split them and you get one chunk that is topically relevant but has no content, and another that has the content but no longer says what it is about. The first ranks well and answers nothing. The second answers the question and never surfaces.

Splitting on semantic boundaries costs more to build and pays for itself the first time someone asks a question that spans two sections. In practice that means:

• Split at structural markers, not character offsets. Headings, list boundaries, and record boundaries are all better cut points than the 512th character.

• Carry the heading path into every chunk. If a chunk lives under Returns and then under International, both of those words should appear in the chunk text, even though a human reading the page would only see them once at the top.

• Allow overlap where the content genuinely flows. A procedure whose steps continue across a boundary should have those steps duplicated into both chunks rather than cleanly divided between them.

• Set a floor as well as a ceiling. A twelve word chunk will match almost any query weakly and answer none of them. Merge it into its neighbour.

The test for whether your chunking is right is simple and does not require any tooling. Take twenty chunks at random and read them cold, with no surrounding context. If you cannot tell what each one is about and what question it would answer, neither can the retrieval system.

Metadata is half the retrieval system

A vector search that cannot filter is guessing politely. This is the part teams skip most often, because embedding a corpus feels like the real work and writing metadata onto records feels like data entry.

Most retrieval failures we see inside Salesforce orgs are not the model failing to understand the question. They are the search returning a technically similar record from the wrong account, or a superseded version of a policy that was never marked as retired, or a draft that should never have been indexed at all. The semantic match was fine. The scope was wrong.

The metadata worth writing is usually small and boring:

• Object type and record type, so a question about a Case never retrieves an Opportunity that happens to use similar language.

• Effective date and expiry, so superseded content can be excluded rather than merely outranked.

• Ownership and visibility, so the search respects the same boundaries the running user does.

• Publication status, so drafts and archived content are filtered out before ranking rather than after.

• Source system, so you can tell where an answer came from when someone disputes it.

Filtering before ranking is not an optimisation, it is a correctness requirement. If a hundred thousand chunks are eligible and only forty are relevant to this user on this record, ranking across all hundred thousand means the top result is competing against a hundred thousand distractors instead of thirty nine. The best embedding model in the world is worse at that job than a WHERE clause.

What breaks specifically inside a Salesforce org

Generic retrieval advice assumes a corpus of documents. A CRM is not that. It is a graph of records with field level security, sharing rules, and a long history of well intentioned customisation. Several things follow from this.

The first is that permissions are part of retrieval. An index built by an administrator sees everything. The agent runs as somebody else. If the index does not carry enough information to reproduce that user's visibility, you will either leak records the user should not see or silently return partial results. Partial results are the more dangerous failure, because nothing appears to be wrong.

The second is that the same concept has several names. Picklist values drift. Free text fields accumulate spellings. A status that reads Closed Won in one record type and Won in another is one concept to a human and two to an index. Normalising these before indexing is unglamorous and removes a whole class of near miss.

The third is that records are not documents. A Case is a shell with a subject, a description, a stream of comments, and a resolution written months later. Indexing that as one blob produces a chunk that is topically about six things. Indexing each comment separately produces chunks with no context. Neither is right, and the answer is usually a composed view: the record summary plus the specific comment, written into a single chunk that stands on its own.

• Compose chunks from a record view rather than a raw field dump.

• Include the parent context in the child chunk, not as a lookup.

• Reindex on change, not on a schedule, or accept that your index is a snapshot of last Tuesday.

• Version the source, so you can tell which revision an answer was drawn from.

A practical order of operations

When a retrieval system is underperforming, the order in which you investigate matters more than the tools you use. Working from cheapest and most likely to most expensive and least likely:

• Confirm the content is actually in the index. This sounds insulting and finds the problem more often than anything else on this list.

• Read the chunk that should have matched, in full, as text. Half the time it is truncated, missing its heading, or contains only boilerplate.

• Check what the filter narrowed to. If the answer is everything, that is the bug.

• Check whether a stale or draft version outranked the current one.

• Only then look at ranking, and only then at the model.

Teams that follow this order fix retrieval problems in an afternoon. Teams that start at the bottom of the list spend a fortnight evaluating embedding models and end up back at chunking anyway, having learned less.

What to measure

Retrieval quality is measurable without an academic benchmark, and the measurements that matter are unfashionable.

Recall at the chunk level tells you whether the right content was retrieved at all. This is the number to fix first, because no amount of downstream cleverness recovers from content that was never returned. If the right chunk is not in the candidate set, the answer is already wrong and everything after that point is decoration.

Precision matters second, and matters mainly because of what it does to the model. Every irrelevant chunk you hand over competes for attention with the one that matters. Retrieving twenty chunks to be safe is not safe. It measurably degrades answer quality compared to retrieving four good ones, and it does so quietly, by making the response vaguer rather than obviously wrong.

Staleness is the one nobody tracks and everybody should. What fraction of answers cite content that has been superseded. This number is usually zero on the day of launch and climbs steadily, which is exactly the shape that escapes notice until someone acts on advice that was retired eight months ago.

• Track recall first, precision second, staleness continuously.

• Freeze a set of real questions with known correct sources and re-run it on every change.

• Log which chunks were retrieved for every production answer, not just the answer.

• Sample disputed answers weekly and trace them back to the chunk.

That last practice is the one that changes teams. When you can point at the exact chunk that produced a wrong answer, the conversation stops being about whether the AI is trustworthy and starts being about whether a specific document was written clearly. That is a solvable problem with an owner.

The short version

Fix the schema and a mediocre embedding model performs well. Leave it broken and the best model on the leaderboard will confidently return the wrong clause, and it will do so in fluent, well structured prose that reads exactly like a correct answer.

The work is in the preparation: how content is split, what it is labelled with, whether the index respects who is asking, and whether anything marks the old version as old. None of that is glamorous and all of it compounds. Every hour spent there keeps paying after the next model release, which is more than can be said for an hour spent tuning around a bad chunk boundary.