# A Controlled Vocabulary for Tagging Biomedical Datasets

Engineering note · Mdexa Engineering · 2026-11-13

Data version: tag vocabulary 1.2; first backfill on the frozen first-release catalogue of 43 cards (public fingerprint ea894e8472bc, releases/first-release/ in the public repository; identical to the internal archive 9ff1d63a933d except change-log wording)

https://mdexa.com/research/N3-controlled-vocabulary-tags/

**Mdexa Engineering** · Engineering note, not peer reviewed · 2026-11-13 · Data version: tag vocabulary 1.2; first backfill on the 43 cards of the frozen first-release catalogue (statistics of 2026-11-13; public fingerprint `ea894e8472bc` in `releases/first-release/` of the public repository, with a verify script. The public version is identical field by field to the internal archive, except that the change logs are worded for the public; the internal archive's fingerprint is 9ff1d63a933d.)

## Abstract

Finding a dataset by what it contains, rather than by name, needs facet tags, and tags are only useful if they are few, controlled and honest about their evidence. A vocabulary of 12 facets and 99 values was drawn up, each value with a definition and each stored tag carrying a source grade (author-stated with a verbatim excerpt checked against a snapshot, inferred, human-confirmed or derived). Backfilled on 43 cards, it produced 182 tags, 99 author-stated and 83 inferred. On 12 real retrieval questions, filtering by existing card fields alone found 38 of 61 relevant datasets, and adding tags found 57 (58 after later adjustment); on 17 questions including specialty, 43 of 76 against 72 of 76. The gain came from four facets that existing fields could not express, and several facets gave none and were demoted or removed. The reference answers and the tags were produced by the same model-assisted process, so the figures are directional only and need a second rater.

**Keywords:** controlled vocabulary; facet tagging; dataset retrieval; provenance of annotations; ICD-11; medical specialties

## 1. The problem

A database of dataset cards can be searched by name, keyword and a few structured fields such as content type and size. Practitioners ask questions those fields do not express: "multiple-choice questions with explanations", "has an official train/test split", "pediatric data", "directly loadable JSON". Free-text keyword matching over descriptions finds some of these and misses others, and tags added without discipline become noise. We wanted a small vocabulary that improves retrieval measurably and that a reader can audit. The vocabulary is published as `schema/tags.json` in the public database repository.

## 2. Design

**Only what helps finding.** A facet exists only if a user or an agent can find a dataset faster or more accurately with it. Facets that fail this test are removed or demoted.

**Controlled vocabulary.** Each facet answers one user question (what does one record look like; what was it intended for; where does the content come from; what else besides the answer is annotated; how was it reviewed; is there an official split; does it have a leaderboard or a human baseline), and each value has a stable identifier, a Chinese and English name, a one-line definition, and examples of when it applies and when it does not. The vocabulary started with 12 facets and 99 values.

**Every tag carries a grade of evidence.** *Author-stated*: the saved page snapshot says it, and the tag stores a verbatim excerpt and the snapshot id, which the validator checks against the snapshot file. *Inferred*: derived from other card fields or the structure of the snapshot, with the basis named and any low confidence noted. *Human-confirmed*: a named person and a date. *Derived*: a value computed from another tag on the same card (see below). No evidence, no tag. Risk-type origins (exam-derived, scraped, real patient text, model-generated, derived data) are not stored as tags; they are computed from the card's risk flags. Size tier is computed from the size fields at publication.

**Only restrict, never include.** A disease or population tag is applied only when the dataset states that it is limited to that range. A dataset that merely contains a pediatric subset is not tagged pediatric; a subset is marked by a `scope` note.

**Specialty and disease category in parallel.** Specialty answers "which department handles this" and disease category (ICD-11 chapter) answers "what disease is it". They are many-to-many and neither can be inferred from the other, so both are facets, each tagged on its own evidence. A tag may be derived from the other facet only when its own facet has no evidence, and it is then graded *derived*, names the tag it was derived from, and is shown as derived with a filter switch. Two cards were derived this way. The specialty vocabulary has 39 entries (30 modern medicine, 8 traditional medicine, one for broad multi-specialty content), each linked to the national clinic-department catalogue, an international board list and ICD-11 chapters, and is meant to be shared by dataset-level and content-level tags by reference and not by copy.

## 3. Backfill and the retrieval test

Backfilling the 43 cards gave 182 tags: 99 author-stated (all excerpts verified against snapshots by the validator), 83 inferred, none human-confirmed. One card has no tags because no usable snapshot exists.

We then tested usefulness. For each of 12 real questions, two filter rules were written: one using only the existing card fields (content type, domain, size values, risk flags, and keyword search over descriptions) and one adding tags. Both were scored against reference answers.

| Questions | Relevant | Existing fields: hits / false / missed | With tags: hits / false / missed |
|---|---|---|---|
| First 12 | 61 | 38 / 5 / 23 | 57 / 2 / 4 (58 / 2 / 3 after adjustments) |
| All 17 (adds specialty and disease category) | 76 | 43 / 5 / 33 | 72 / 4 / 4 |

*Table 1. Figures from the first backfill and its retrieval test (internal test records, not public). Vocabulary version 1.2 has 13 facets because the specialty facet was added; the facet, value and tag counts elsewhere in this note are those of the first backfill unless stated.*

Where the gains came from: four facets (data format, official split, benchmark status and annotation depth) account for most of the improvement on the first 12 questions. For example, "directly loadable JSON/JSONL" found 1 of 11 relevant datasets with existing fields and 10 of 11 with tags; "has an official split" found 4 of 8 against 8 of 8. Questions that existing fields could already express, such as multiple-choice benchmarks or small evaluation sets, improved little or not at all. For specialty, almost the whole gain came from one question about broad multi-specialty benchmarks (1 hit against 10); questions about a single specialty or disease class were answered equally well by keyword search on 43 cards.

## 4. What we cut, and why

On this evidence we removed or demoted facets that did not earn their place. Size tier became a derived value computed from the size fields, so it is no longer stored on cards. Population (one card tagged) and disease category (5 tagged cards at the first backfill, and one question that keyword search handled better) became optional facets, hidden from the default filter bar and shown only on detail pages. Drug category (ATC level one) was kept in the vocabulary but is not enabled because no card is limited to one. The `saturated` value of benchmark status was deleted because no card had it. After the change the first 12 questions re-ran at 58 of 61 hits. A maintenance rule says a new value needs at least one real question that uses it, and any facet or value unused by test questions for a year is demoted or deleted.

## 5. Limits of this evidence

- **Self-grading.** The reference answers and the tags were produced by the same model-assisted process, without independent human annotation. The numbers are directional; a second, human rater should write independent reference answers and the test should be re-run.
- **Small catalogue.** Specialty and disease facets cannot be tested properly on 43 cards in which few datasets are limited to a single specialty. Their value has to be measured again as the catalogue grows.
- **Our own translations.** Chinese names of ICD-11 chapters and ATC first-level classes are our translations and have not been checked against official Chinese editions. The ICD-11 chapter numbers and English names were checked against the WHO reference guide on 2026-11-13.
- **Mapping not reviewed by physicians.** The mapping between specialties and ICD-11 chapters was written by us and has not been reviewed by physicians; the department catalogue's full list of second-level departments was not checked against the source.
- **Low-confidence tags.** Several inferred tags rest on search snippets or on secondary descriptions; they are marked with notes and are not shown on list cards.

## 6. Not solved

Tags describe whole datasets. Questions such as "how many oncology questions are in this dataset" need item-level tags, which need content analysis and term recognition and are not done. A reserved field for the specialty distribution inside a dataset exists and is empty. Independent re-rating, a larger catalogue, and a check of the vocabulary by clinicians remain to be done.

*Engineering note, not peer reviewed. Mdexa Engineering, 2026-11-13. Based on the public vocabulary file `schema/tags.json` and internal test records.*