Attestari methodology

This document is the complete set of rules by which Attestari examines a published package, decides each finding, grades the package, and publishes the result. It is the only place those rules live in prose, and attestari/policy.py is the only place the decision rules live in code; the two carry the same version number and a test refuses to run the policy when they disagree. Every published decision cites the version that produced it and the SHA-256 of this file as it was when the decision was made, together with the model, the prompt and the verifier that produced the finding (section 12). Nothing in a run is decided by a person; the two things a person still does are listed in section 13.

1. What this document is for

Attestari examines software packages that an AI agent may install and grant access to: MCP servers and agent tooling published to public registries. We report what we find, and we publish the evidence alongside it so a reader can check our work rather than take it on faith.

A finding is an opinion, formed by applying the rules below to material anyone can download. This document states those rules completely. If a finding cannot be traced to a rule in this document and to evidence a reader can independently retrieve, it should not have been published, and we want to hear about it.

The document also states, at length, what we do not examine (section 15). That section is not a disclaimer bolted on at the end. It is the part most likely to matter to someone deciding whether a passing result means anything.

2. Definitions

subject: a specific published version of a package in a public registry, for example @scope/server at version 1.4.2 on npm. Ratings are always of a version, never of a project in general.

artifact: the file a package manager actually downloads and installs: the npm tarball or the PyPI distribution. This, not the source repository, is what we examine.

tool surface: the set of tools a subject registers with an agent, together with the descriptions the agent reads. Determined by parsing the artifact's source.

finding: a statement about a subject, tied to a severity, a quoted span of text, the file and line it came from, and the hash of the stored evidence containing it.

not analyzed: a tool we could not examine. Distinct from a tool we examined and found nothing in. The two are never reported as the same thing.

gradeable: a subject for which every advertised tool was examined, every check ran, and every finding's evidence re-verified: the guards in section 9. Only a gradeable subject can carry a grade.

attested manifest: the hash of each readable tool's name, description bytes and schema, and the hash over all of them, computed as section 7 defines and recorded on every decision row. What a gateway pins to when it wants the tools it was shown to be the tools it examined.

capability inventory: what the artifact's code is able to reach, as declared by the source: network destinations, remote code, files, processes. Recorded as context, never measured for precision, never graded (section 20).

3. We examine the artifact, not the repository

Most software auditing reads the public source repository. We do not, and the distinction is deliberate: the repository is not what runs on your machine. The published artifact is.

A package's repository can be clean while the tarball served by the registry contains code that was never committed there. That gap is how a large share of real registry compromises work, and auditing the repository looks straight past it.

We read repository metadata, such as ownership, contributors, and whether the latest commit carries a signature, as context about who stands behind a subject. We do not treat repository contents as evidence of what a subject does.

What this costs us. We do not currently compare the artifact against the repository at the published tag. That comparison would detect a file shipped with no traceable origin in the source, which is a strong signal we would like to have. It is a known gap, recorded in section 15.

4. How subjects are selected

Selection is mechanical and reproducible. A candidate list records the exact queries used, the ranking rule, the page sizes, the timestamp of every fetch, and the snapshot identifier of any dataset consulted. Anyone re-running the same selection against the same sources gets the same list. What it selects is a sample of each registry, not the registry: every count we report is a count over the packages we selected, and says nothing about the packages we did not.

npm candidates come from the registry's public search API across a fixed set of queries, ranked by monthly download count confirmed against the downloads API for a stated window. PyPI candidates come from the simple index, with download counts drawn from a public dataset derived from PyPI's own download statistics.

Exclusions are recorded, not silent. Every candidate that surfaced and was not selected appears in the same file with the reason it was dropped: ranked below the cut, duplicate of a scoped variant, no download figure available. A reader can see what we passed over.

Conflicts. Packages published by Attestari, The Elite360 Corporation, or any account we control are excluded by a hard rule, checked against package name, scope, maintainers, publisher, author, contact addresses, and repository URLs. We do not rate our own work.

A tagged package is not necessarily a server. Selection casts a wide net using MCP-related keywords and naming patterns, which catches libraries and SDKs alongside actual servers. We do not resolve this by maintaining a hand-curated exclusion list. The tool-surface determination in section 5 decides what is a server, and sample figures always separate confirmed server, examined from confirmed server, could not examine from not a server.

5. The checks

Eight checks are declared in attestari/checks/, and this version states the standing of each. Four run against every subject, and each of those states what it reads and what it is entitled to conclude. Four are declared with their inputs fixed and do not run; a subject is never marked clean on a check that did not run, and a report lists them as not assessed (section 15). None of the eight executes the subject's code.

The set of checks and tags whose findings may carry a gradeable severity is MEASURED_CHECKS in attestari/policy.py. In this version it holds model_directed_text with every tag and nothing else (decision D5, section 10). A check enters that set only by a measurement recorded in section 18.

undetermined_tool_surface: can the tool surface be determined at all?

Reads: source files in the artifact, looking for MCP tool registration sites on the token stream (text inside strings and comments does not count). Concludes: a parsed set of tools with their descriptions, or one of these outcomes:

• not_a_server: no registration site, and the source consumes the tool API (it lists or calls another server's tools, imports an SDK client) or defines it (a method named registerTool or tool with a body, a server class). A client, adapter, proxy, SDK, framework, instrumentation, CLI, mock or scanner. Such a subject is out of scope: it is not a server, so it is neither graded nor refused, and it is excluded from every population count (decision D4).
• minified_only: registration text exists only in minified or obfuscated bundles. The operator cannot inspect what they install; this is a medium finding whose evidence is the bundle's bytes and the length of its longest line, not a coverage gap.
• runtime_generated: a registration site exists and the file that registers builds its tools from external data (an OpenAPI document, a remote catalog). A genuine limit, reported as such.
• declaration_not_parsed: a registration site exists in readable source and the extractor could not read it. This is the only outcome that is honestly our gap; the site is recorded.
• no_registration_site: neither a registration site nor a consumption pattern.
• archive_unreadable: the artifact is not a readable tar or zip.

Every outcome but parsed refuses the grade under its own code (section 9), except not_a_server, which is out of scope. A subject whose surface cannot be read is not thereby condemned. It is a subject about which we will not offer an opinion, because the thing an agent would actually be exposed to was never visible to us.

model_directed_text: does a tool description contain instructions aimed at the model?

Reads: the literal text of each parsed tool description, passed to a language model as clearly delimited evidence, never as free text that could be read as instruction. Concludes: specific quoted spans that direct the reading model's behaviour, each categorised and verified.

Before any description is read, each tool's description is in one of three states, read from the parsed surface, never guessed:

• readable: a single literal, or a concatenation of literals with no substitution ("a" + "b", ["a", "b"].join("\n")). Every byte the client receives is in the source, and the check reads all of it.
• partial: a template with substitutions (`Reads ${what}`, an f-string). Each literal segment is read as its own byte span; each substitution is a hole, recorded with its position, that the verifier never matches inside. A finding in a segment is a valid finding. The absence of findings in a partial description does not make the tool clean: a package with a partial tool that produced no finding is refused a grade (partial_description, section 9) unless a high finding already places it at F, where the floor is established by evidence.
• unreadable: a variable, a property, a call, a configuration passed by reference that does not resolve to a literal declared in the same file, or a minified bundle. The text a client will receive exists and was not examined. Each such tool is recorded as not_analyzed with the reason descriptions_not_statically_readable, and the package is refused a grade by the gate in section 9.

A tool that declares no description at all (absent) has nothing for this check to read, and for this check that is a clean outcome (no_descriptions_present). A surface that does not record the state of a description is treated as unreadable.

A tool description is read by the agent, not by the user. Text placed there can instruct the agent to exfiltrate data, override the user's stated intent, or conceal what it is doing. Those three categories, exfiltrate_data, override_user and conceal_from_user, are the only ones that become findings, and each carries severity high. Routing, ordering and rate guidance ("call X first", "no more than three calls") is never a finding. The check records only a count of such spans per tool; the spans themselves are transcribed only by a separate inventory pass that a run must opt into (attestari run --inventory), and nothing that decides reads it.

Two boundaries of those categories are stated here because real descriptions have tested them. Parameter documentation is not a finding unless the parameter requests data outside the tool's declared purpose: a parameter that must carry the user's prompt, the user's credentials, files, conversation history, or configuration the tool does not need to do what it says is exfiltrate_data, wherever in the description the text appears and however it is phrased. Documenting a parameter does not license routing data through it. Retry and resume guidance about the tool's own calls after an interruption ("keep the run id and resume", "an interrupted call is not a cancellation") is routing guidance, not override_user, unless it directs the model to disregard an explicit stop or cancel from the user.

Every proposed span is checked by a deterministic verification step: the quoted text must occur byte for byte inside the stored file, within the description of the named tool. A proposal that step rejects never becomes a finding. The verifier carries its own version (VERIFIER_VERSION in attestari/checks/model_directed_text.py), recorded on every decision row; a change to what it accepts or rejects bumps that version.

The analysis prompt encodes the evidence so that it cannot be interpreted as instruction by the model reading it. This is both a correctness measure and a safety one: an analyser that can be manipulated by the text it audits is not an analyser. The SHA-256 of the prompt bytes and the name of the model are recorded with every decision (section 12), so a reader can tell which analyser produced a row.

Context tags on exfiltration findings

An exfiltrate_data finding says that a description routes data through the tool that the tool does not need. It does not by itself say where the data goes, whether the publisher said so, or whether the operator can turn it off. Three informational tags answer those questions from the package bytes alone, and each is applied only when the analyser can establish it; a tag that cannot be established is absent, and the decision row records why. None of the three changes a severity, a decision or a grade, and none is part of the finding's identity. They are read by the deterministic analyser in attestari/checks/exfil_context.py, at decision time, from the stored artifact:

• destination_publisher_owned: every network destination the source names is on a domain the package declares as its own, that is the host of its homepage or repository URL, read from the manifest inside the artifact (package.json, pyproject.toml, PKG-INFO) or from the registry record. A shared code host or a registry (github.com, gitlab.com, npmjs.com, pypi.org) is never a publisher's own domain. A network destination is the host of an absolute URL literal in a source file that also makes a network call, or assigned to a name that a network call uses.
• disclosed_in_readme: a line of the README names a transmit host together with a sending verb, or discloses telemetry (telemetry, usage data, instrumentation, analytics, with collect, send, report, transmit, upload or share). A documentation link that merely mentions the domain is not a disclosure.
• opt_out_available: a source file that reads the environment names an opt-out variable (OPT_OUT, DO_NOT_TRACK, DISABLE_TELEMETRY and their variants), or a source file that tests for a file names an opt-out file.

The analyser reads text; it executes nothing and traces no data flow. The tags therefore say what the package declares and names, not what a particular byte of the user's data reaches. A reader weighing a finding should read them as exactly that.

remote_code_fetch: does the package fetch and execute code it did not ship?

Reads: source files and package manifests, for download-then-execute patterns, dynamic evaluation, remote imports, and install-time scripts. Concludes: quoted spans showing code that is obtained at runtime rather than published.

This is the check that catches a package which is mostly a launcher: what the registry serves is small and readable, and the substance arrives later from somewhere else. Neither the registry nor this analysis can tell you anything about code that was never published.

This check carries no severity of its own in this version. Its findings are the remote-code entries of the capability inventory (section 20): recorded with the severity the check assigned, decided at info (decision D5), published as context, never graded. The run of 2026-09-14 showed its patterns matching a chmod of a directory, a git log subprocess and a vendored schema compiler; until its precision is measured (section 18) it states a capability, not a risk.

publisher_provenance: what links this artifact to a person or organisation?

Reads: registry metadata, publisher attestations, maintainer records, repository ownership, commit signatures. Concludes: what is present and what is absent in the chain between a named party and this artifact.

Provenance findings describe absences more often than problems, and absences are common: most registry packages carry no build attestation and no signed commits. We report them because their accumulation is meaningful, not because any one of them is damning. Severity reflects this: a single provenance tag is low, and a finding reaches high only when a second, independent provenance tag on the same package corroborates it (decision D6).

The account-age rule (account_younger_than_package: the account that owns the source repository was created after the package's first release) is informational pending measured precision (section 18). A registry name transferred to a new owner is enough to trigger it, so until its precision is measured it is decided at info (decision D5), as is every other provenance finding.

The four declared checks that do not run

undeclared_network_calls (hosts contacted by the code that neither the README nor the tool descriptions mention), dependency_risk (known advisories, unpinned or deprecated direct dependencies, install-time scripts), abandonment (archived repositories, no commits in a stated period, unanswered issue growth) and fork_divergence (packages published from forks, and the byte-level difference between shipped code and upstream) are declared with the inputs each will consume, so that the evidence a run collects already covers them. They produce no result row, and the gate (section 9) requires a result row only of the four checks that run. A report lists them as not assessed, which means exactly that. The network destinations the capability inventory records (section 20) are the raw material of the first of them, and are published as inventory, not as a finding, until the check runs and is measured.

6. Severity

Severity describes exposure to the person who installs and runs the subject. It is not a measure of intent, and it makes no claim about whether a publisher meant any harm.

High: an operator running this subject is exposed to something they would not reasonably expect from its stated purpose: code they cannot inspect, credentials reaching a destination they did not choose, or instructions to the agent that work against the operator.

Medium: a property that materially weakens the operator's ability to know what they are running, or that concentrates control in a way worth knowing about before installing.

Low: a property of the publisher record that leaves the operator with no one to hold to account, without by itself changing what the package does. A provenance finding with a single tag is decided low; a second, independent tag on the same package raises it to high (decision D6, section 10).

Info: a factual observation, usually an absence, that is common across the registry and meaningful mainly in aggregate or alongside other findings. It is published as context and has no effect on the grade.

We do not use a borrowed scale such as CVSS. These are not vulnerabilities with exploit mechanics; they are properties of what a package does and what can be known about it. The grade (section 11) is read off the highest confirmed severity.

7. Evidence and verification

Every artifact we fetch is stored by content hash. Every finding cites the hash of the evidence it came from, the path within it, and the line range, alongside the quoted span itself.

At decision time, the quoted span is re-verified byte for byte against the stored evidence (guards G6 and G7 in section 9). A span that does not re-verify does not become a published finding, and its subject becomes ungradeable rather than passing quietly.

Every published report carries instructions for reproducing its central claims from public sources in a few minutes, without our software and without trusting us. If you cannot check a finding yourself, we have not finished writing it.

No personal data in published evidence. A published blob never contains an email address, a phone number or a street address. This is a rule, not a judgement. Evidence is published in one of two forms, defined exactly in docs/publish-contract.md (section 9a):

• A finding that cites a structured upstream response (a registry record, a GitHub API response) is published with a projection: a canonical JSON object carrying only the member the finding cites, copied byte for byte, the JSON pointer of the object it was taken from, and the SHA-256 of the whole upstream response as provenance (source_sha256). The projection's own hash is the evidence hash the published finding verifies against; the upstream bytes are never published.
• A finding that cites a file from the package artifact is published with that file whole: it is public code. It is scanned all the same.

Every blob of either kind is scanned for the patterns the contract lists before it is published. A finding whose evidence cannot be published without personal data is withheld, not redacted and not softened, and its subject is published as not_gradeable with the reason evidence_unpublishable. The stored evidence, the decision and the finding id are untouched; only publication is refused.

Attested manifest

For each parsed tool whose description is readable (section 5), the tool hash is SHA-256 over: the tool name in UTF-8, one 0x00 byte, the description bytes exactly as extracted (description_raw), one 0x00 byte, and the schema component. When the declared input schema is a JSON literal, the schema component is its canonical JSON (RFC 8785: sorted keys, no insignificant whitespace, UTF-8). When it is any other expression, a zod schema, a builder call, a reference to a constant, the schema component is the schema's source bytes exactly as extracted, taken the same way description_raw is: a zod schema is hashed as the bytes of its expression, and only a JSON literal is canonicalised. When no schema is declared, the component is empty. The manifest hash is SHA-256 over the tool hashes, as 32-byte digests, sorted bytewise and concatenated; hexadecimal is only how either hash is printed. Both are computed by attestari/manifest.py, and a fixed-vector test pins this definition to that code.

A tool whose description is not readable has no tool hash, and a subject with any such tool has no manifest hash: the manifest is not attested, which is an outcome, not an omission. Both hashes are recorded on every decision row of the subject (tool_hashes, one entry per parsed tool with its hash or the reason it has none, and manifest_hash; section 12), and published with the result. The lookup API serves them for a package version at GET /manifest/:name/:version, so a gateway can pin the tools it was shown to the tools this pass examined.

8. Inputs to a decision

A run directory written by attestari run (see attestari/pipeline.py): findings.json, results.json, gradeable.json, surfaces.json, records.json, not_analyzed.json, summary.json, plus the content-addressed evidence store the run cited (evidence/blobs/<sha256>). The policy reads these files and the blobs. It never calls the network or a model.

9. Package gradeability (fail closed)

The failure we care most about avoiding is not a wrong grade. It is a passing grade on something nobody looked at. A subject that could not be examined must never be indistinguishable from one that was examined and found clean.

So the grading gate fails closed. A package is graded only when every guard below holds. Any guard that fails, or cannot be evaluated, makes the package not_gradeable; every reason found is recorded, in order, and every finding of that package is withheld. There are no exceptions and no severity that is exempt.

• G1 gate recorded. gradeable.json exists and has a row for the package. A missing file blocks every package in the run; a missing row blocks that package.
• G2 gate passes as recorded. The row says ok: true.
• G3 gate passes when recomputed. attestari.grade.gradeability is re-run at decision time from results.json and the package's surfaces.json row (tools with descriptions, parsed tool names, advertised tool names) and must return ok. A package with no surfaces row cannot be recomputed and is blocked.
• G4 no tool left unanalysed. No not_analyzed entry for the package in results.json or in not_analyzed.json. A refused, truncated or unreplayed model call is an outcome, never a clean result.
• G5 every check ran and none errored. Each configured check (publisher_provenance, remote_code_fetch, undetermined_tool_surface, model_directed_text) has a result row for the package, and summary.json records no failure for the package that is a crash: a failure row whose stage is fetch or a check id, or that carries a traceback. Recorded HTTP failures of a source (a 404 repository, an unreachable website) are inputs to the checks, not crashes, and do not block by themselves.
• G6 every excerpt re-verifies byte for byte. For each finding of the package: the cited blob exists in the evidence store, and the UTF-8 bytes of evidence.excerpt occur inside it. When the finding cites lines (line_start..line_end), the bytes must occur within those lines. An empty excerpt fails. One unverifiable finding blocks the whole package.
• G7 model-derived findings passed the verifier. For each model_directed_text finding that quotes a tool description (every one except description_reassigned_at_runtime), the excerpt must lie inside one of the description spans of a tool declared at the same path in the package's surfaces.json row: the same test attestari.checks.model_directed_text.verify_proposal applies when the proposal is made. A proposal the verifier discarded is never a finding; a finding that would not pass the verifier now blocks the package.

The gate recomputed in G3 refuses a package for any of the following reasons, and records each one it finds:

The gate map. Every reason code a decision or a publication can carry, the guard or decision that records it, and one plain-English line for each, is the table GATE_MAP in attestari/policy.py. attestari decide exports it as analysis/gate_map.json beside the code and as gate_map.json inside the run, and the website reads the exported file rather than restating it. A test fails when a reason code known to the policy or to the gate has no entry, and when the exported file differs from the table. Each code also has a fixture under analysis/fixtures/reason_codes/<code>/, a run directory and the evidence it cites, on which attestari decide (or, for evidence_unpublishable, attestari publish) records exactly that code; a test decides every fixture and fails when a code has none.

"We cannot tell you what this does" is a legitimate published outcome, and for a meaningful share of popular packages it is the accurate one.

10. Finding decisions

The policy named severity-ladder decides each finding of a gradeable package:

• D1 confirmed. A verified finding whose decided severity is high, medium or low is confirmed at that severity. The decided severity starts from the severity the check assigned and is changed only by D5 and D6 below; the row records both.
• D2 noted. A verified finding whose decided severity is info is noted: published as context, no effect on the grade.
• D3 withheld. Every finding of a package that is not gradeable is withheld, and its row lists the package's blocking reasons. A withheld finding is never published as a finding.
• D4 out of scope. A package whose surface is not_a_server (section 5) is out of scope: it is published with the verdict out_of_scope and the reason not_a_server, every finding of it is withheld, and it is counted apart from the population rather than as a refusal. Out of scope is not a grade and not a refusal; it says the artifact is not the kind of thing this methodology rates.
• D5 measured checks only. Only a check whose precision has been measured (section 18) may place a finding above info. The policy carries the set of measured (check, tag) pairs, MEASURED_CHECKS; in this version it holds model_directed_text with every tag and nothing else. A finding from any other check or tag is decided at info and noted, whatever severity the check assigned; the assigned severity stays on the row. Every check still runs and every finding is still recorded. Adding a pair to the set is a methodology change that must cite the measurement in section 18.
• D6 provenance corroboration. A publisher_provenance finding is decided high only when a second, independent provenance tag is present on the same package; a single tag is decided low. D6 is applied before D5, so while publisher_provenance is unmeasured the outcome is still info.

There are no suppression rules in this version. A finding that should not count is a methodology change (add a rule here, bump the version) or an appeal (section 16), never an edit to a run.

11. Package grade

From the confirmed findings of a gradeable package:

A package that is not gradeable has no grade; its row carries blocked_by instead.

12. What every decision row records

Every row, package or finding, carries:

• policy, methodology_version, methodology_sha256: the rules it was decided under, by number and by the hash of this document's bytes;
• code_commit, the git commit of the code that decided (code_dirty says whether the tree had uncommitted changes), and decided_at;
• analyzer_model, the model the run's description check called (empty when the run made no model call), prompt_sha256, the SHA-256 of the findings call's system prompt bytes, and verifier_version, the version of the deterministic verifier (section 5). The run header records the prompt layout and version beside them, and says whether the prompt hash was recorded by the run itself or reconstructed from the code at decision time;
• tool_hashes, one entry per parsed tool of the subject with its tool hash or the reason it has none, and manifest_hash, the attested manifest of section 7, or the reason the subject has none;

and every finding row also carries the finding's evidence_sha256 (the blob it was verified against) and excerpt_sha256, the verification outcome, the rule that produced the decision, and, on an exfiltrate_data finding, the context tags of section 5 with the evidence for each. The run-level header repeats the stamps and records the SHA-256 of every input file. The rows are written twice, as one document (decisions.json) and as one JSON object per line (decisions.jsonl: the header, then each package, then each finding, every line carrying the stamps), and summarised for reading in decisions.md. A rerun over unchanged inputs with the same methodology and code produces identical rows apart from decided_at.

13. Automated decision, human responsibility

Findings are decided by the rules in this document, applied by software, without a person reviewing each one. This is a deliberate choice. Rules applied by a machine are applied identically to the two-hundredth subject and the first; a person working through hundreds of findings is not.

What is not automated is the rules themselves. Changing them is an explicit act that produces a new version of this document (section 21), and every finding cites the version that produced it. The methodology, not any individual judgement call, is what stands behind a rating.

Running the rules is not a human act and needs no approval. The two things a person does:

1. Appeals. A vendor disputing a finding files an appeal against its finding id (attestari appeal file). A person resolves it (attestari appeal resolve) as upheld, dismissed or withdrawn. Appeals are an append-only event log beside the run (appeals.json); nothing in decisions.json is edited. Publication applies them: an upheld appeal publishes the finding as withdrawn_on_appeal and recomputes that package's grade without it; an open appeal publishes the finding marked under_appeal, unchanged. What an appeal can be about, and how we answer one, is in section 16.
2. Methodology changes. Editing this document and the matching constant, as described in section 21.

Publication is separate. Producing findings and publishing them about a named third party are distinct steps. Automation covers the first. The second is attestari publish <run>: one command, not a review queue, run only after the disclosure period in section 19. It refuses a run whose decisions were made under a methodology version or document hash other than the current one, and never overwrites an earlier publication.

14. Scheduled operation

attestari cycle --candidates <file> performs one cycle with no human input: it resumes the most recent failed or cancelled run over that candidates file if there is one, otherwise starts a new run; then decides the run under this methodology, diffs it against the previous decided run over the same candidates file, and writes decisions.json, decisions.jsonl, decisions.md and diff.json into the run directory. --every <seconds> repeats on that cadence in-process; a scheduler may instead invoke the one-shot form. The scheduled pass runs weekly.

15. Known limits

These are the things a passing result does not tell you. They are stated here rather than in a footnote because a reader who does not know them will over-read our findings.

We do not run the code. Analysis is static. A subject that behaves differently when executed, against a particular host, after a delay, or only in the presence of certain credentials, presents none of that to us.

We do not examine binaries. Where a package's substance is a compiled executable it downloads, we can describe the launcher and the fetch precisely, and we can say nothing whatsoever about what the binary does.

Text assigned at runtime is invisible to us. A tool description that is benign in source and rewritten when the module loads reaches the agent in its rewritten form. We detect and report that a description is reassigned at runtime, but we cannot read what it becomes.

Tool names built by expression may be missed. Where a tool's name is computed rather than written literally, our extractor can fail to find a tool that genuinely exists. A name, a configuration or a description written as a constant declared in the same file is resolved; anything else is reported as declaration_not_parsed (section 5), and the gate in section 9 catches an advertised tool we did not find as a mismatch rather than passing the subject silently.

A minified bundle is opaque to us and to you. Where a package ships its registration only in a minified or obfuscated bundle, we report that as a medium finding and read nothing from it. We do not reconstruct source from a bundle.

Tools generated at run time are not in the artifact. Where the file that registers tools builds them from an OpenAPI document or a remote catalog, the tool list exists only when the server runs. We report the site and read nothing further.

A publisher who reads these rules can write around the model check. Section 5 publishes the classification rules in full, and the language-model check applies them to prose. A description written with those rules in hand can carry the same instruction in a form the model does not recognise, and the check is measured (section 18) only against cases that were not written against it. The deterministic checks do not share that weakness: the description states, the surface taxonomy, the capability inventory and the reassignment finding read structure, not meaning, and a description that is rewritten when the module loads is reported as such whatever the rewritten text says. A clean model check on a package whose author has read this document is weaker evidence than a clean model check on one whose author has not.

The context tags and the capability inventory read text, not behaviour. A destination is a host the source names; a disclosure is a sentence the README carries; an opt-out is a variable the code reads. Where a destination is computed, a disclosure lives outside the README, or an opt-out is honoured by some code paths and not others, the tags and the inventory do not see it, and they say nothing either way.

We do not diff the artifact against the repository. A file present in the published package with no counterpart in the source tree is a signal we do not currently produce. See section 3.

Four declared checks do not run. Undeclared network destinations, dependency risk, maintenance status, and fork divergence are declared in section 5 and not assessed. Where a report lists these as not assessed, that is what it means, not that they were assessed and found clean.

A finding is about one version, at one time. Registries are mutable. A result describes the artifact we fetched, identified by hash, on the date stated. It says nothing about what the same version number serves tomorrow.

16. Notice, appeals, and corrections

Notice before publication. Before any finding about a named subject is published, we send it to the publisher at the contact address in their registry record or repository, and wait 30 days. The same finding goes to the registry on the same day (section 19). That period is for correcting us, and a publisher is under no obligation to respond.

Appeals. A publisher may dispute any finding at any time, before or after publication, by writing to appeals@attestari.ai. An appeal that identifies a factual error, such as a misquoted span, a wrong line reference, or a claim about code that is not there, is checked against the stored evidence. If we are wrong, we correct it.

An appeal that disputes our interpretation rather than our facts will be considered and may be published alongside the finding. We do not remove an accurate finding because a publisher objects to it.

Corrections. Corrections are published, dated, and describe what was wrong and what changed. We do not silently edit a published finding.

Remediation. When a publisher fixes the thing a finding describes, a re-examination of the new version is free and prompt, and the updated result is published.

17. Independence

A publisher cannot pay us, in money or in kind, for a rating, for a better rating, or for the removal of a finding. There is no paid review, no expedited review, and no arrangement under which a rated party influences its own result.

The subjects we examine are chosen by the mechanical rule in section 4. They are not chosen by request, and they are not chosen by who has paid for anything.

Where Attestari earns revenue, the source is disclosed. Any commercial relationship with a party that is also a subject is disclosed on that subject's published result.

Publisher tools. Attestari may offer a publisher a tool that applies these checks to an artifact that has not been published. Its output is never published, never stored where the scheduled pass can read it, and never affects a grade. The public grade of a subject is computed only from the registry artifact, only by the scheduled pass, only under the methodology in force, whether or not the publisher used such a tool. The same holds for every other thing sold to the people deciding whether to install: a lookup, a manifest, a watch, a badge, an attestation of inspection. Each shows what the scheduled pass published and never changes it.

18. Measured performance

These figures are preliminary. They are measured on a small held-out corpus, and they are restated whenever the corpus or the check changes; they are evidence that the check finds known attacks, not a general detection rate. None of the measured cases was written against the rules in section 5 (the adaptive adversary, section 15).

The corpus. The held-out half of the disclosed-cases set (seeds/heldout/cases.json) records nine cases. Four carry a published sample and are measured: three tool-poisoning disclosures by third-party security researchers and one in-the-wild observation from our own run (blender-mcp 1.9.1), with 39 expected spans between them (23 in the three disclosures, 16 in blender-mcp). The other five are pattern-only records whose disclosure printed no sample; they are counted in the corpus and never measured. Three further disclosed cases are worked examples inside the prompt (seeds/fewshot/) and are never measured. Precision is measured against a control set of 456 tool descriptions in 16 widely-used packages reviewed by hand (runs/twenty/surfaces.json).

Detection and precision, as a range across four independent samples of the same prompt version (the model is sampled once per call, so two runs of the same evaluation can differ):

• 4 of 4 measured cases detected in every sample, both in-the-wild cases among them
• 32 to 33 of 39 expected spans found, every one in its expected category: 16 to 17 of 23 on the three disclosures (17 in one sample of four), 16 of 16 on blender-mcp in every sample
• 0 false positives across 456 clean tools in every sample
• 0 proposals rejected by the verifier in any sample

Six spans were missed in every sample and one further span was found in one sample of four; the other 32 were found in all four. Every missed span sits in a description on which other spans were found; no measured case was missed whole in any sample. The rug-pull case whose poisoned text is assigned when the module loads, and so never appears in the source we read (section 15), is now a worked example in the prompt and is not measured.

Where the figures come from: the four evaluation runs recorded at runs/eval-heldout-2026-09-14.4, runs/eval-var-1, runs/eval-var-2 and runs/eval-var-3 (2026-09-14, claude-opus-5, prompt version 2026-09-14.4, the separated prompt layout, each against an evidence store whose proposal cache was empty so that every call went to the model), scored by attestari eval against the held-out manifest and the clean set above. In those evaluations only, each seed sample's evidence block cited the publication that disclosed it; a production run carries no such citation.

The population pass. The scheduled pass over the 200 most-downloaded candidates of both registries (runs/20260914T234604Z, candidates file top200.json, decided 2026-09-15 under prompt version 2026-09-14.4 with claude-opus-5) records, over the 200 packages selected:

• 21 out of scope (not_a_server), leaving a population of 179 confirmed or presumed servers
• 61 gradeable: 59 graded A and 2 graded F, both on exfiltrate_data findings of the description check (blender-mcp 1.9.1, sixteen findings; @shopify/dev-mcp 1.15.2, one finding)
• 118 not gradeable, every one under not_analyzed recorded by the gate (G2) and confirmed when recomputed (G3); the gate's own codes over the population: no_registration_site 46, advertised_tools_not_parsed 41, not_analyzed 22, declaration_not_parsed 16, partial_description 11, minified_only 6, runtime_generated 2, archive_unreadable 1
• 553 findings recorded: 17 confirmed, 181 noted at info (53 of them decided at info by D5, 34 from publisher_provenance and 19 from remote_code_fetch, the latter now the capability inventory), 355 withheld with their packages

These counts are counts over the selected sample (section 4), not over either registry. A grade of A in that pass means the one measured check found nothing in every readable description of the subject; it does not speak to the four declared checks that do not run, or to anything in section 15. The same run, decided again under this version, produces the same grades: the changes of this version add what a row records and change nothing a row decides.

19. Disclosure

Disclosure runs on a fixed clock, the same for every subject, and the clock is not ours to shorten or extend for anyone.

Day 0, registry and publisher. On the day a finding of severity high or medium is decided against a named subject, the finding, its evidence and the methodology version go to the registry's security contact (npm and PyPI each publish one) and to the publisher at the contact address in their registry record or repository. A finding a registry can act on before publication is worth more to the operators exposed to it than a finding published first.

Day 30, publication. The decision is published thirty days after day 0, by the command in section 13, in the form of docs/publish-contract.md. It is published whether or not the publisher or the registry has responded, with any open appeal marked as such (section 16) and any correction the notice period produced already applied. A fixed version is examined and published on the same clock as any other version.

What is never disclosed early. Nothing goes to anyone before day 0, and nothing goes to any party other than the registry and the publisher before day 30. A commercial customer of Attestari sees a finding when everyone else does.

Noted findings (severity info, decision D2), out-of-scope packages and refused packages are published on the same day-30 clock without a day-0 notice: they state an absence, a scope or a refusal, not a risk that a registry can act on.

20. Capability inventory

A grade says whether the description check found an instruction to the agent that works against the operator. It does not say what the package's code can reach. The capability inventory does, as far as text can: for each examined subject it records what the artifact's source declares it is able to do, published as context beside the grade, never as a finding and never as a severity.

What it holds in this version. The remote-code entries: every remote_code_fetch result (download-then-execute, download-and-mark-executable, evaluation of fetched content, a remote import, an install script that downloads, an artifact downloaded with no execution path found), each citing its line, each carrying the severity the check assigned for the record, each decided at info under D5. The network destinations the context analyser of section 5 names for a subject with an exfiltration finding are recorded on that finding. Nothing else is in the inventory yet.

Declared, not measured. The inventory is produced by deterministic pattern checks whose precision has not been measured against a hand-reviewed corpus (section 18). It is therefore declared: we say what the patterns matched and where, and we do not say what fraction of matches a reader would agree with. Until a measurement is recorded in section 18 and MEASURED_CHECKS names the check, no inventory entry moves a grade, and the severity a pattern assigns is recorded and not decided. This replaces the standing remote_code_fetch had in earlier versions, where its assigned severities were stated as severities with a note that they were pending measurement; in this version they are inventory.

What it will hold. File writes outside the package's own directory, process spawning, and the hosts contacted that neither the README nor the tool descriptions mention, as the declared checks of section 5 come into force. Each enters the inventory the day its check runs, and leaves the inventory for the grade only by a measurement in section 18.

21. Versioning

This document carries a version: the methodology_version: line at the top, MAJOR.MINOR.PATCH. Every published finding cites the version in force when it was produced and the SHA-256 of the document. Superseded versions remain available at their own permanent addresses, so a finding published in 2026 can still be read against the rules that made it in 2026.

A change to what a check examines, what a severity means, or when we refuse to grade is a version change. Correcting a typo is not. A change to what a row records, or to how and when a result is disclosed, is a version change too, because a reader of an older row must be able to tell what it does not carry.

Changing any rule is a deliberate act: edit this document, bump methodology_version, bump METHODOLOGY_VERSION in attestari/methodology.py, add a line to the Versions section, copy the document to a new snapshot under the site's content/methodology/, and commit. A published snapshot is never edited.

Versions

• 1.0.0 (2026-09-11): first automated policy. Guards G1-G7, decisions D1-D3, grade ladder A/B/C/F, appeals and publication as the two human acts.
• 1.1.0 (2026-09-11): the public methodology around the unchanged rules: what the document is for, definitions, artifact not repository, subject selection, the four checks, severity definitions, evidence and verification, known limits, notice, appeals and corrections, independence, measured performance, versioning. Guards, decisions and the grade ladder are unchanged from 1.0.0.
• 1.2.0 (2026-09-11): two corrections. Evidence: no personal data in published evidence; structured responses are published as projections of the cited member with the upstream hash as provenance, files ship whole and scanned, and a finding that cannot be evidenced without personal data is withheld with its subject refused (evidence_unpublishable). Model check: "no literal tool descriptions" is split into no_descriptions_present (clean) and descriptions_not_statically_readable (not analysed, refuses the grade through not_analyzed), read from the parsed surface and failing closed when the surface does not say.
• 1.3.0 (2026-09-11): the surface taxonomy and three description states. A surface that did not parse is not_a_server (out of scope, decision D4), minified_only (a medium finding), runtime_generated, declaration_not_parsed, no_registration_site or archive_unreadable, each refusing the grade under its own code except the first. A description is readable, partial (literal segments read, substitutions are holes; findings valid, absence not clean: partial_description) or unreadable. The extractor resolves names, configurations and descriptions written as constants declared in the same file, reads class-based getName() / getConfig() declarations, accepts schema and args as schema keys, and reads Python call-form registration x.tool(name=...)(fn).
• 1.4.0 (2026-09-14): unmeasured checks decide at info. Decision D5: only checks whose precision has been measured (section 18) may grade; the set MEASURED_CHECKS holds model_directed_text alone, so every remote_code_fetch and publisher_provenance finding is recorded and published as context at info. Decision D6: a provenance finding is high only with a second, independent provenance tag on the same package, otherwise low. Section 5 marks remote_code_fetch and the account-age rule as informational pending measured precision. Section 7 gains "Attested manifest": a tool hash per readable tool and a manifest hash per subject, defined in attestari/manifest.py and pinned by a fixed-vector test. Section 17 gains "Publisher tools": a publisher's own use of these checks never affects a grade. Section 6 no longer names anonymous_publisher as the only low finding (D6 decides any single-tag provenance finding low); section 5 records routing guidance as a count per tool, the inventory being a separate opt-in pass.
• 1.5.0 (2026-09-14): two boundaries of the model check stated in section 5. Parameter documentation is not a finding unless the parameter requests data outside the tool's declared purpose: a parameter that must carry the user's prompt, credentials, files, history or configuration the tool does not need is exfiltrate_data wherever in the description it appears. Guidance about retrying or resuming the tool's own calls after an interruption is routing guidance, not override_user, unless it directs the model to disregard an explicit user stop or cancel. Prompt version 2026-09-14.4 carries both. Section 4 speaks of a sample, not a population. Section 15 names the adaptive adversary: a publisher who reads the published rules can write around the model check; the deterministic checks and the reassignment finding do not share that weakness. Section 18 is marked preliminary, states the held-out corpus size, and is restated from the evaluation run under 2026-09-14.4. Guards, decisions and the grade ladder are unchanged.
• 2.0.0 (2026-09-15): the version in force. Section 5 lists the eight declared checks with the standing of each, names MEASURED_CHECKS, gives the verifier a version, and adds the three context tags on exfiltrate_data findings (destination_publisher_owned, disclosed_in_readme, opt_out_available), informational and read from the package bytes at decision time. Section 7 states that a zod schema is hashed as its source bytes and that both manifest hashes ride on every decision row and are served per package version. Section 9 adds the gate map (every reason code, its gate, one line, exported as gate_map.json) and a fixture per reason code. Section 12 adds analyzer_model, prompt_sha256, verifier_version, tool_hashes and manifest_hash to every row, and decisions.jsonl beside decisions.json. Section 17 extends the publisher-tools rule to everything sold to installers. Section 18 adds the population pass of 2026-09-14 beside the held-out figures. New section 19, Disclosure: registry and publisher on day 0, publication on day 30, on one clock for everyone. New section 20, Capability inventory: the remote_code_fetch findings and the named network destinations are inventory, declared and not measured, in place of the check's earlier severity standing. Versioning moves to section 21. Guards G1-G7, decisions D1-D6, MEASURED_CHECKS, the prompt and the grade ladder are unchanged, and the population pass of 2026-09-14 decides to the same grades under this version.