Four grades, and one outcome that is not a grade.
The grade is read off the highest confirmed severity §11. There is no score, no average and no partial credit: one confirmed finding at high is an F whatever else the package does well. And a package is graded only when every guard of the gate holds §9; otherwise it is not gradeable, with every reason recorded.
- A
Grade A: No confirmed finding.
Only noted context, or nothing. Under 2.9.1 that means every tool, prompt and resource description (and every prompt's message text) was readable and the one measured check, the description analysis, found nothing in it. Findings from checks whose precision is not yet measured are noted at info and cannot move the grade (decision D5).
- B
Grade B: Highest confirmed severity is low.
A property of the publisher record that leaves the operator with no one to hold to account, without by itself changing what the package does.
- C
Grade C: Highest confirmed severity is medium.
A property that materially weakens the operator's ability to know what they are running, or that concentrates control in a way worth knowing about before installing.
- F
Grade F: Any confirmed finding of severity high.
An operator running this subject is exposed to something they would not reasonably expect from its stated purpose: code they cannot inspect, credentials reaching a destination they did not choose, or instructions to the agent that work against the operator.
- —
Not gradeable. A guard failed, or could not be evaluated.
The failure we care most about avoiding is not a wrong grade. It is a passing grade on something nobody looked at. A package we couldn’t examine is published as “not gradeable” with the reasons why. It never looks the same as a package we examined and found clean §9. “We cannot tell you what this does” is a legitimate published outcome, and for a meaningful share of popular packages it is the accurate one.
We don’t run the code; what the code does when called isn’t graded yet.
Under 2.9.1 no measured check assigns low, so no package currently receives B, and C currently comes from one finding only: a tool description replaced at run time (description_reassigned_at_runtime). §11
The description check was admitted on 39 attack spans and 456 clean descriptions, against a published bar of 200 labelled spans and 1,000 clean descriptions. The phase-1 corpus reached that bar on 2026-09-17 and the figures pass the test on it; its labels are two independent model passes with adjudication, and no person has labelled it yet. Until a human pass confirms the labels, “measured” carries this note. §18
Measured on 2026-10-07 over 1,907 labelled descriptions (1,949 spans), under methodology 2.9.1, claude-opus-5, prompt 2026-09-28.10-b64-fewshot-counts-nothink, every proposal read once more by model:claude-opus-4-8@labeller-2026-09-18.1: 87 of 89 real descriptions found (population and research; 245 of 247 synthetic descriptions apart), recall on real descriptions 97.8%, Wilson 95 % 92.2% to 99.4%; precision 98.5% on the found descriptions (332 of 337, Wilson 95 % 96.6% to 99.4%); 0 false positives across 1408 clean descriptions (0.0%, Wilson 95 % 0.0% to 0.3%) and 0 across 93 hard negatives (0.0%, Wilson 95 % 0.0% to 4.0%); 19 description(s) not analysed and excluded from every rate. Before the independent reading, the check alone proposed a finding on 88 of 89 real descriptions (98.9%, Wilson 95 % 93.9% to 99.8%), with precision 94.6% on the descriptions it proposed on (333 of 352, Wilson 95 % 91.7% to 96.5%), 4 false positives across 1408 clean descriptions (0.3%, Wilson 95 % 0.1% to 0.7%) and 2 across 93 hard negatives (2.2%, Wilson 95 % 0.6% to 7.5%); the reading decided the rest. Labelled by two independent model passes (model:claude-opus-5@labeller-2026-09-18.1, model:claude-sonnet-5@labeller-2026-09-18.1; kappa 0.977 on 1949 records); disagreements and byte-identical inconsistencies adjudicated by a third model pass (model:claude-opus-4-8@adjudicator-2026-09-18.1, model:claude-opus-4-8@adjudicator-2026-09-18.2, model:claude-opus-4-8@adjudicator-2026-09-27.3) citing the guide, and by attestari-engineers@2026-09-18 (12 records), and by claude-adjudicator (8 records); a third labeller (model:claude-opus-4-7@labeller-2026-09-18.1) agreed with the gold set on 99 of 100 (99.0%, Wilson 95% 94.6%–99.8%) of a seeded sample (40 positive, 20 hard negative, 40 clean). No person has labelled the corpus. §18
Reason codes, from the gate map the engine exports §9 §7
advertised_tools_not_parsedG3- The README or a bundle manifest advertises tools the extractor did not parse; the mismatch is itself the finding.
archive_unreadableG3- The artifact is not a readable tar or zip.
check_erroredG5- A fetch or a check crashed on this package (a failure with a traceback, or in the fetch or check stage) and the run recorded it.
check_missingG5- A configured check has no result row for this package; a check that did not run cannot have found nothing.
declaration_not_parsedG3- A registration site exists in readable source and the extractor could not read it: our gap, recorded as such.
evidence_unpublishablepublish- A finding's evidence cannot be published without personal data; the finding is withheld, not redacted, and the subject is published as not gradeable.
excerpt_unverifiedG6- A finding's quoted excerpt is not byte for byte inside the cited evidence blob, within the cited lines when lines are cited.
extraction_budget_exceededG3- Reading the package's source passed the per-package budget of wall time or resident memory, so nothing past that point was read; the budget is measured, the measured figures are recorded on the row and published with it, and the grade is refused.
gate_blockedG2- The gate row recorded by the run says the package is not gradeable.
gate_file_missingG1- The run has no gradeable.json, so nothing in it may be read as gradeable; every package in the run is refused.
gate_not_recordedG1- gradeable.json has no row for this package: the gate was never recorded for it.
gate_recomputed_blockedG3- The gate, recomputed at decision time from the run files, refuses the package; its own reason codes are listed with it.
minified_onlyG3- Registration text exists only in a minified or obfuscated bundle: a medium finding, and no grade.
model_check_skippedG3- The description analysis did not run on a package that has parsed tool descriptions, including when the analysis service was unavailable and the retries ran out: an examination incomplete on our side, not a finding about the package.
no_registration_siteG3- Neither a registration site nor a consumption pattern was found in the artifact.
no_surface_examinedG3- No tool, prompt, resource or resource template was parsed from the published source; the package's advertised behaviour was never visible.
not_a_serverD4- The package consumes or defines the tool API rather than serving tools: out of scope, neither graded nor refused, counted apart from the population.
not_analyzedG4- A tool was never analysed: the model provider declined or never finished the call, a replay had no record, or the description is declared but not readable as a literal.
partial_descriptionG3- A tool's description was read with holes and nothing was found in its readable segments; that does not establish a clean tool. Not applied when a high finding already places the package at F.
reading_missingG8- A model_directed_text finding above info has no independent reading recorded (readings.json absent, the finding not in it, or its call not completed), so it can be neither confirmed nor noted and the package is refused.
runtime_generatedG3- The file that registers tools builds them from external data at run time; the tool list exists only when the server runs.
surface_not_recordedG3- surfaces.json has no row for this package, so the gate cannot be recomputed at decision time.
unknownG3- The surface row was written before the surface taxonomy existed and records no outcome; the grade is refused rather than guessed.
verifier_failedG7- A model-derived finding's excerpt is not inside a description span of a tool declared at the cited path, by the same test the verifier applied when the proposal was made.
A package that is not a server at all, a client, an SDK, a CLI, is out_of_scope: neither graded nor refused, and counted apart from the population §10.