Designing a Database Interface for Agents: Lessons from Live-Client Testing

1. The problem

Mdexa is a database of biomedical dataset cards that agents query through an MCP server or an HTTP API. A first version exposed six tools: search, card, license evidence, facet listing, access information and a content search that was not yet available. It passed our own tests (40 checks in our own test suite at that point). Passing our own tests told us little about how a model would actually use it, so we ran a test with a real client.

2. What we tested

Protocol conformance. The official TypeScript SDK client (version 1.32.2) connected to the server over both stdio and Streamable HTTP, negotiated protocol version 2025-11-25, listed the six tools and called each of them. All 13 tool results per transport were well-formed and no warnings were raised. The newer protocol revision (2026-07-28) is implemented on the server but no official client could exercise it, so only our own tests cover it [1].

A real agent. We ran a real command-line agent client against the server, using one low-cost model through a third-party channel, and gave it nine questions (counts, license lists, a commercial-use question, repository versus data license, a source-risk question, a request for access, a dataset that does not exist, and one open task). The 43 tool calls contained no case of calling a link-only dataset downloadable or a restricted one commercially usable. We did not get a stronger-model comparison: the channel for it returned an error.

3. What went wrong, and what we changed

The dataset that does not exist. Asked about a dataset that is not in the database, the model searched with seven different keyword sets (eight calls in total) before concluding. Nothing in the response told it that "no match" was a complete answer. The fix was a structured no-match response: a one-line statement that the database has no matching dataset, the catalogue total, the number of records that each filter alone would have excluded, and an instruction not to retry with reworded queries. The same question then took 1 call.

Evidence one dataset at a time. In the open-ended question the model fetched license evidence in about ten separate calls, one per candidate, to explain risks, because compact result rows carried the license status but not the risk flags. We added risk_flags and a repo_only marker (a repository license exists but nothing says it covers the data) to every compact row. Four of the 43 cards were repo_only at that time. The question fell from 15 to 12 calls; the model still fetched evidence for each recommended dataset, since the task asked for the evidence. Each compact row grew by 48 characters on average (about 11%).

Red lines in tool descriptions. Server-level instructions are not always read, so the description of the search tool now states what the status fields mean and that restricted, conflict, unstated and unverifiable never mean "commercially usable". The access tool's description says that Mdexa serves no dataset files and that the agent should not write download commands from memory (the first test showed a model producing a download command from its own knowledge). The description also says that every word of a query must match, and that statistics should come from the facet listing rather than from paging.

Filters that tell you what is legal. When tag filters were added, an unknown value returns an error that lists the legal values for that facet, and a reserved facet returns an error saying so, instead of an empty list the agent might misread as "no such datasets". Compact rows grew by a further 50 characters on average (472 to 522) to carry four tag keys.

Third-party text. Quoted excerpts come from pages that Mdexa does not control. Fields that carry them are named as untrusted third-party text, the responses say to treat them as data and not as instructions, and the agent guide repeats it.

4. What the benchmark added

The first run (single run, 43 first-release cards, 2026-11-13) of a later benchmark, whose revised report is planned, recorded the 30 tasks of the tool-equipped condition in detail. Across the 30 tasks the agent made 191 Mdexa tool calls (6.4 per task): 95 searches, 34 card reads, 25 evidence reads, 20 facet listings, 9 access look-ups, 4 similar-dataset calls and 4 batch-check calls. Thirteen of the 95 searches (13.7%) returned zero results: five were multi-word queries in which every word had to match, and most of the rest asked for something the database does not contain, as some tasks require. All four batch-check calls were rejected because the tool required an API key in the anonymous setup, so the model wasted a call or two and then fell back to one search per item. In the five most call-heavy tasks (11 to 14 calls), the first search had usually already returned the right set, and later calls checked that nothing had been missed. The current source tree has since added an explicit "complete" note to search results; we have not re-run the benchmark to measure its effect. (These are records of that first run, not conclusions. An earlier internal analysis gave a different call breakdown that did not add up to the per-task mean; the figures here are recomputed from the raw records and the server log.)

5. Principles we took from this

1. Make an empty result a first-class, final answer, with enough context (catalogue size, what each filter excluded) for the agent to believe it.
2. Return little by default, and put into each compact row the fields the agent needs to decide whether to ask for more (here, risk flags and a repository-only marker).
3. Put the rules that matter into the tool description, not only into server instructions.
4. Make errors teach: list the legal values.
5. Label untrusted text and say how to treat it.
6. Do not offer a tool that cannot work for the caller (a key-only tool visible to anonymous clients).

6. What is not solved

• The model still re-searches after a complete answer; a note saying results are complete reduces but may not remove this, and its effect is unmeasured.
• Multi-word queries that require all words to match still produce empty results; the benchmark analysis suggests ranking by number of matched words instead.
• The anonymous-visible batch tool remains a suggested fix, not a verified one.
• Compact rows do not yet say why a dataset matched; models fetch whole cards (about 3.5 thousand characters each) to find out.
• All findings come from single runs of one low-cost model; a stronger model, other clients and the newer protocol revision were not tested.

How to cite

Mdexa Engineering. Designing a Database Interface for Agents: Lessons from Live-Client Testing. Engineering note, not peer reviewed. 2026-11-13. Data version: catalogue of 43 cards; interface tests of 2026-11-12/13.