← Back to Reliability Explorer

Measured selection evidence

DeepWiki MCP

deepwiki-mcp

Measured historical tool-selection reliability from frozen benchmark evidence.

Multi-batchS2-backedObserved missesMixed provenance
Mixed live and version-bound evidence. Historical evidence does not establish current endpoint availability or current production behavior.
6independent benchmark batches
52measured frozen cases
S1, S2evidence level(s)
5observed semantic misses

First measurement: 2026-09-10 13:23:12.077611 UTC · Last measurement: 2026-09-10 19:13:49.842728 UTC · Observed strict accuracy range: 57.1%–100.0%

Batch history

Each immutable benchmark batch is shown separately. No pooled accuracy is calculated here.

Benchmark / batchTypeFrozen casesStrict scoreWilson 95% CIProvider-valid coverageInfrastructure failuresEvidence / measured
selection-evidence-expansion-v1
selection-evidence-expansion-v1		 CONSTRUCTED_FROZEN_SELECTION_BENCHMARK24 24/24
100.0%		 Not recorded–Not recorded 24 / 24 (100.0%)0 S1
2026-09-10 13:23:12.077611 UTC
external-gold-expansion-v2-batch1
external-gold-expansion-v2-batch1		 EXTERNAL_INDEPENDENT_GOLD_BENCHMARK4 3/4
75.0%		 30.1%–95.4% 4 / 4 (100.0%)0 S2
2026-09-10 15:38:03.968297 UTC
external-gold-expansion-v2-batch2
external-gold-expansion-v2-batch2		 EXTERNAL_INDEPENDENT_GOLD_BENCHMARK5 5/5
100.0%		 56.6%–100.0% 5 / 5 (100.0%)0 S2
2026-09-10 16:49:53.346446 UTC
external-gold-expansion-v2-batch3
external-gold-expansion-v2-batch3		 EXTERNAL_INDEPENDENT_GOLD_BENCHMARK6 6/6
100.0%		 61.0%–100.0% 6 / 6 (100.0%)0 S2
2026-09-10 17:45:38.245198 UTC
external-gold-expansion-v2-batch4
external-gold-expansion-v2-batch4		 EXTERNAL_INDEPENDENT_GOLD_BENCHMARK6 5/6
83.3%		 43.6%–97.0% 6 / 6 (100.0%)0 S2
2026-09-10 18:47:01.455700 UTC
external-gold-expansion-v2-batch5
external-gold-expansion-v2-batch5		 EXTERNAL_INDEPENDENT_GOLD_BENCHMARK7 4/7
57.1%		 25.0%–84.2% 7 / 7 (100.0%)0 S2
2026-09-10 19:13:49.842728 UTC

Observed semantic misses

These are preserved adjudicated historical observations, attributable to a benchmark, batch, and case.

Benchmark / batchCase IDExpected toolSelected toolBoundary familyAdjudication
external-gold-expansion-v2-batch1
external-gold-expansion-v2-batch1		 deepwiki-004ask_question read_wiki_structureNOT_RECORDED_IN_SOURCE_EVIDENCE SEMANTIC_MISS
external-gold-expansion-v2-batch4
external-gold-expansion-v2-batch4		 b4-dw-005read_wiki_contents read_wiki_structureCONTENTS_VS_QUESTION VALID_GOLD_RETAINED
external-gold-expansion-v2-batch5
external-gold-expansion-v2-batch5		 b5-dw-003read_wiki_contents read_wiki_structureCONTENTS_VS_QUESTION VALID_GOLD_RETAINED
external-gold-expansion-v2-batch5
external-gold-expansion-v2-batch5		 b5-dw-004ask_question read_wiki_contentsQUESTION_VS_CONTENTS VALID_GOLD_RETAINED
external-gold-expansion-v2-batch5
external-gold-expansion-v2-batch5		 b5-dw-006read_wiki_contents read_wiki_structureCONTENTS_VS_STRUCTURE VALID_GOLD_RETAINED

Evidence provenance

Technical identities are secondary detail for auditability.

DeepWiki MCP
Canonical subject ID
deepwiki-mcp
Provenance type
CONSTRUCTED_FROZEN_SELECTION_BENCHMARK
Identity mode
LIVE_TOOLS_LIST_SNAPSHOT
Identity source
LIVE_TOOLS_LIST
DeepWiki
Canonical subject ID
deepwiki-mcp
Provenance type
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
Identity mode
LIVE_TOOLS_LIST_SNAPSHOT
Identity source
CURRENT_LIVE_TOOLS_LIST_INHERITED_V1_AND_FROZEN_BY_COPY
DeepWiki
Canonical subject ID
deepwiki-mcp
Provenance type
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
Identity mode
LIVE_TOOLS_LIST_SNAPSHOT
Identity source
CURRENT_LIVE_TOOLS_LIST_FROZEN_BY_EXISTING_IMMUTABLE_SNAPSHOT
DeepWiki
Canonical subject ID
deepwiki-mcp
Provenance type
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
Identity mode
VERSION_BOUND_SNAPSHOT
Identity source
CURRENT_OR_VERSION_BOUND_SNAPSHOT_REUSED_FROM_QUALIFIED_CORPUS
selection-evidence-expansion-v1 immutable identities
case_freeze_sha256
5d248e845fc1b52ef53585682a97a592fecd0d3c1b8c9e561c758a4222273aed
freeze_commit_sha
d2937086b419ba0bf77517da772540de1975c45f
measurement_artifact_sha256
27f281b49b50fe5218102274c57db0110f3119091ca5dd53ebe954997f4fe826
model_lock_sha256
a8df78457b76e0dea73f66e2a75907feab3c42885db23658caada6973c5cb478
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
external-gold-expansion-v2-batch1 immutable identities
case_freeze_sha256
17777d2c3983cd444353b8d69dcc98836a1909ee6fde6f5c8cf18cf9b059427e
final_freeze_receipt_commit
b691170
freeze_input_commit
c614ae51750d12ce113b5e3d51429f52e529f08c
measurement_artifact_sha256
01243422aa946296f235810aaee6f065ec21d9e78a01017ee8161e5e61a8b5e8
measurement_commit
b9f358b1a65aba348fa1c0e625eac6da1f31b651
model_lock_sha256
c4330165deb3d8587e224a2ca1b3d886db60801ca436710a0edace55525923d2
policy_evaluation_commit
3e560e5767f181831c65569b893ca789cd3bdff0
policy_evaluation_sha256
4e710a9c5205660cc07603d2fd7538aa063eda4c2574c29308f9f255e4706f82
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
raw_predictions_sha256
ab085cedf13eef5724f4e160b0f49f8402d62bc5b10203de0f8d81e41f2a017e
external-gold-expansion-v2-batch2 immutable identities
case_freeze_sha256
ce21fc085bd9a0185c9e37d0d795221f286397cbc477ee941e876be24f7ba1b4
final_freeze_receipt_commit
d3c55d2f96a4e45bf2577225ea16929877f2ded1
freeze_input_commit
61f6511c41562c24019cd441b37bc6ee07283951
measurement_artifact_sha256
dd13aec8a90b2b5490c934c9ad9650c37733c65f18d824e2ae6066202176ced9
measurement_commit
f0b6cd33be044d8bad800c20b4f2684e6d8431ce
model_lock_sha256
153647f94eed0ee3d3069229545846eef864fd43fa821f2d311dc3fb5d265ba9
policy_evaluation_commit
f0b6cd33be044d8bad800c20b4f2684e6d8431ce
policy_evaluation_sha256
736f154a5d5c3b8f0d77cd58fa0644ca97ad94c7a6b46af98d394c1dc1578f9a
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
raw_predictions_sha256
8c4364ab598241ddbe4adb16453ba2e12b0917a1d1852caedb07884d1a54570b
external-gold-expansion-v2-batch3 immutable identities
case_freeze_sha256
c1595743f48fe8ae6b3f70f2ea401731052b3bd7614297feabaa8172ce190ae5
final_freeze_receipt_commit
a309bda7e8dd2867ccc7894dc7d42efbfc48395e
freeze_input_commit
8450ed889756784a12ca4ebb8d84974d73b31778
immutable_measurement_receipt_commit
9d8290160849a5124d1896fc318d53470585ef41
measurement_artifact_sha256
7dcab9d4b71e7de59094c07cbca830ebf674b601ac6de024db0d3e677e7c0067
measurement_commit
b81cc816158f7c0fbe7f8ac0a6fcf2b3f38f4c86
model_lock_sha256
48b5ed9c98f16cc2b3b76bc0da2e7ec8072ecd89ba1914683181b1c20074a3a8
policy_evaluation_sha256
3ea5b62044b4da6e92812831ebb00205970849accffe763a270248a24a0adef4
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
raw_predictions_sha256
c5cfe765f6b72489220cc73f689acb1c1c2c9284ffd8d093de0e597b645f6fbf
external-gold-expansion-v2-batch4 immutable identities
case_freeze_sha256
a9513c1cb9b800b75d822d75fa8c6e47041e551d86a75699f1858b6c3785204d
final_freeze_receipt_commit
224aeab80db539383018d77cc7adb849976f1db5
freeze_input_commit
003985017fbd8c1e207fb146329b97b38582e606
immutable_measurement_receipt_sha256
7d6d0a1ac4b1d501aa4c5d0044b4a4e50bb319b272bd33eb783aec6499d594c6
measurement_artifact_commit
9b10dc0c99c4eed15a3d370743e8f46439302bae
measurement_artifact_sha256
7d9a5f57e0897f2ca8b2734ac0f38dc5afd099b2edf2495dbfa25e41f088cee9
model_lock_sha256
ffc1ec351c1a62c3692cdaedba0c5a6ec2c6285923eb065e0cd31953f978e3a1
policy_evaluation_sha256
0e9b11360db41731ce139584060b3f1417ce50b31dcf86ef88180e391e56e0d5
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
raw_predictions_sha256
869812a69814f58e73a1c6c3f94a28b6dabc2cdb729285d3a532dd528ad36c89
scanner_freeze_sha256
e31caf3131b4e03f54ff411a386bb72ac70c35fbe9027e3383b7026019c22ae1
external-gold-expansion-v2-batch5 immutable identities
case_freeze_sha256
40847d041af701492505676ae027f824248a94dba8dad19c4c764baec224a2bc
final_freeze_receipt_commit
4c11daed10b52027a8a31130703335e4835873ec
freeze_input_commit
935e16c4cc073d72b1c64c48206196feaf28c170
immutable_measurement_receipt_commit
b064dea6a0ea488826d11fc0fc88c5b4de3d5c26
measurement_artifact_commit
87a78aba4797f79f3c99c22e202eddd1da70f034
measurement_artifact_sha256
9b5f0c7f94783f2b25d28d8ecdaebe3a5a11b9db00dd84f677f95ac008f32eeb
model_lock_sha256
60c5653e461180e44e9621a54bb77f3f474e6889ddc29d1ff30d3cb9398013fa
policy_evaluation_sha256
2ef31f2a44a722fb862425dc11755b118c3ffb712672471d10d4cd0bbcee9af1
promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d
raw_predictions_sha256
612c995839a9ad1a32d87320055ada3201c511b626a0f3e8778a8bbe5201bc68
scanner_freeze_sha256
80b3b0d60a5ec2e3b5019d2c45838dd06517453ce64c9a32ed0dbda6be718795
validation_criteria_sha256
0cd5821683b9f85ae37140bfc73d25064012d69aabbbbb4363e56b058cb240a6