{"schemaVersion":"1.0.0","benchmark":{"id":"conyr-solana-wallet-intelligence-benchmark","name":"Solana Wallet Intelligence Benchmark","version":"0.1.0","status":"protocol-only","network":"solana-mainnet","publishedAt":"2026-08-19","modifiedAt":"2026-08-19","canonicalUrl":"https://conyr.ai/research/solana-wallet-intelligence-benchmark","manifestUrl":"https://conyr.ai/research/solana-wallet-intelligence-benchmark/manifest.json","maintainer":{"name":"Conyr","url":"https://conyr.ai"}},"releaseState":{"comparativeResultsPublished":false,"providersRegistered":0,"frozenCasesPublished":0,"protocolTestsDefined":30,"minimumCasesForComparativeRelease":25,"maximumCasesForComparativeRelease":50,"explanation":"Version 0.1 publishes the evaluation contract before subjects, provider outputs or scores. Repository observations and curated Conyr labels were not promoted to independent benchmark ground truth."},"resultClasses":{"measured":"Produced by an executed, timestamped benchmark run against a frozen case manifest under the published run conditions.","documentationObservation":"A capability described in a provider's public documentation. It can inform test design but is never converted into a benchmark score.","internalFinding":"A result produced during Conyr development or research. It is excluded until independently adjudicated and admitted as a frozen public case."},"cohortContract":{"status":"pending-independent-adjudication","requiredCaseCount":{"minimum":25,"maximum":50},"publicIdentifiersOnly":true,"requiredEvidence":["Public Solana wallet or token identifiers","Frozen slot or time boundary","Transaction-signature-backed adjudication","Positive and negative cases","Written admission and exclusion rationale","Ground truth produced independently from every provider response being scored"],"exclusions":["Private customer or user data","Cases selected only because Conyr already detects them","Unverified identity, ownership or intent allegations","Cases whose required history cannot be frozen or cited","Cases discovered after inspecting a tested provider's answer"]},"axes":[{"id":"position-accounting","name":"Position accounting","question":"Does the provider reconstruct completed positions instead of treating profitable exits as independent wins?","unitOfAnalysis":"One wallet and token mint over a frozen slot range","groundTruth":"An independently reconstructed inventory ledger containing buys, sells, transfers, fees, lot allocation, remaining inventory and source transaction signatures.","measures":["Completed-position outcome accuracy","Realized PnL absolute and percentage error","Open inventory quantity error","Partial-history and unsupported-inventory abstention accuracy"],"abstentionRule":"A documented unknown is correct when the frozen ledger cannot establish cost basis. Coercing unknown inventory to zero or profit is an error."},{"id":"provenance-depth","name":"Funding provenance depth","question":"How far can a provider follow capital without inventing an origin?","unitOfAnalysis":"One wallet and its pre-cutoff native-SOL funding path","groundTruth":"A transaction-signature-backed path adjudicated hop by hop, with service and infrastructure roots verified separately from the provider under test.","measures":["Direct-funder accuracy","Valid path depth before termination","Recognized-root accuracy","False attribution rate"],"abstentionRule":"Unknown is preferable to an unsupported entity attribution. Unresolved paths are not scored as self-funded wallets."},{"id":"coordination-discovery","name":"Coordination discovery","question":"Can the provider find repeated on-chain relationships without equating co-occurrence with common ownership?","unitOfAnalysis":"One token episode and an adjudicated wallet cohort","groundTruth":"Positive and negative wallet relationships reviewed from funding, execution, timing, size, recurrence and infrastructure evidence; intent is never used as ground truth.","measures":["Relationship precision and recall","False grouping of shared infrastructure users","Evidence-domain coverage","Calibrated confidence or explicit abstention"],"abstentionRule":"The benchmark grades detected on-chain relationships, not claims about legal ownership, identity or malicious intent."},{"id":"leader-follower-inference","name":"Leader/follower inference","question":"Can the provider recover a repeat, directed trading relationship and support its direction?","unitOfAnalysis":"One wallet pair across multiple shared token episodes","groundTruth":"A blinded pair adjudication using repeated event order, lag distribution, shared-mint support, size resemblance and counterexamples where the apparent direction reverses.","measures":["Directed-edge precision and recall","Direction accuracy","Lag and support-count error","Single-event false-positive rate"],"abstentionRule":"One coincident trade is insufficient. A provider can abstain when repeat support does not clear its published threshold."},{"id":"freshness","name":"Freshness","question":"How long after an eligible on-chain event does the tested result become available?","unitOfAnalysis":"One live, pre-registered event observed from a common Solana slot reference","groundTruth":"The chain slot and event signature, an external observation timestamp and the first successful provider response containing the eligible update.","measures":["Availability latency p50 and p95","Timeout and stale-response rate","Event-to-derived-intelligence latency","Observation-window sample count"],"abstentionRule":"Documentation language such as real time is not a measurement. Freshness is reported only from timestamped live trials."},{"id":"null-honesty","name":"Null honesty","question":"Does the provider preserve the difference between zero, empty, unavailable, unsupported and unknown?","unitOfAnalysis":"One deliberately selected boundary or incomplete-evidence case","groundTruth":"An adjudicated answer state paired with the evidence that makes a numeric zero, empty set, unknown value or unsupported request appropriate.","measures":["Answer-state classification accuracy","Unsupported-as-zero rate","Unknown-as-fact rate","Machine-readable partial or error semantics"],"abstentionRule":"A clear unknown or unsupported answer can receive full credit. A plausible-looking value without sufficient evidence cannot."},{"id":"evidence-traceability","name":"Evidence traceability","question":"Can a user inspect the chain evidence behind the answer?","unitOfAnalysis":"One normalized provider answer from any other axis","groundTruth":"The frozen case record and its public transaction, slot, address, mint and time references.","measures":["Transaction-signature coverage","Slot and timestamp coverage","Method or data-version disclosure","Stable evidence-reference coverage"],"abstentionRule":"A result is not made correct merely by attaching a link. References must resolve to evidence relevant to the normalized answer."}],"tests":[{"id":"PA-01","axisId":"position-accounting","name":"Closed round trip","fixtureRequirement":"A wallet/mint ledger with complete acquisition and disposal evidence inside the frozen range.","expectedObservation":"Closed-position outcome, cost basis and realized PnL can be compared to the adjudicated ledger.","status":"definition-only","scoreable":false},{"id":"PA-02","axisId":"position-accounting","name":"Partial exits","fixtureRequirement":"A wallet/mint ledger containing multiple disposals before inventory reaches zero.","expectedObservation":"Profitable fragments are not counted as independent won positions and remaining inventory stays open.","status":"definition-only","scoreable":false},{"id":"PA-03","axisId":"position-accounting","name":"Open inventory","fixtureRequirement":"A wallet/mint ledger with acquired inventory remaining at the cutoff.","expectedObservation":"The provider separates realized outcome from remaining quantity and unrealized state.","status":"definition-only","scoreable":false},{"id":"PA-04","axisId":"position-accounting","name":"Transfer-acquired inventory","fixtureRequirement":"A wallet/mint case where token inventory arrives without an observable in-range purchase cost.","expectedObservation":"Unknown or partial cost basis is preserved rather than fabricated as zero-cost profit.","status":"definition-only","scoreable":false},{"id":"PA-05","axisId":"position-accounting","name":"Fee-aware lot allocation","fixtureRequirement":"A multi-lot position with independently reconstructed fees and lot-consumption order.","expectedObservation":"The disclosed accounting method reproduces realized PnL and lot state within the published tolerance.","status":"definition-only","scoreable":false},{"id":"PV-01","axisId":"provenance-depth","name":"Direct funder","fixtureRequirement":"A wallet with one unambiguous qualifying native-SOL funding transaction before its first eligible activity.","expectedObservation":"The provider returns the correct immediate sender and supporting transaction signature.","status":"definition-only","scoreable":false},{"id":"PV-02","axisId":"provenance-depth","name":"Multi-hop path","fixtureRequirement":"A wallet with a public, adjudicated funding chain spanning at least three intermediary wallets.","expectedObservation":"Every returned hop is valid, ordered and supported; depth is reported before termination.","status":"definition-only","scoreable":false},{"id":"PV-03","axisId":"provenance-depth","name":"Recognized infrastructure root","fixtureRequirement":"A funding path ending at an independently verified exchange, custody, payment or service wallet.","expectedObservation":"The root attribution and on-chain root address agree with the adjudicated evidence.","status":"definition-only","scoreable":false},{"id":"PV-04","axisId":"provenance-depth","name":"Unresolved root","fixtureRequirement":"A complete visible path that does not reach an independently supported entity within the test depth.","expectedObservation":"The provider returns unknown or unresolved instead of inventing an identity or self-funded state.","status":"definition-only","scoreable":false},{"id":"PV-05","axisId":"provenance-depth","name":"Ambiguous funding event","fixtureRequirement":"A wallet with multiple plausible inbound transfers around activation and a written adjudication boundary.","expectedObservation":"The provider discloses its selection semantics or abstains when a unique funding path is not supportable.","status":"definition-only","scoreable":false},{"id":"CD-01","axisId":"coordination-discovery","name":"Repeated timing relationship","fixtureRequirement":"A wallet cohort with repeated, tightly timed participation across multiple token episodes.","expectedObservation":"Detected relationships retain recurrence and timing evidence instead of relying on one co-occurrence.","status":"definition-only","scoreable":false},{"id":"CD-02","axisId":"coordination-discovery","name":"Shared private funder","fixtureRequirement":"A cohort linked by an adjudicated, non-infrastructure funding source and repeat execution evidence.","expectedObservation":"The provider finds the supported relationship without asserting unobservable identity or intent.","status":"definition-only","scoreable":false},{"id":"CD-03","axisId":"coordination-discovery","name":"Shared infrastructure negative","fixtureRequirement":"Independent wallets whose only common feature is funding or service use through public infrastructure.","expectedObservation":"The provider avoids grouping ordinary infrastructure users into a coordinated actor.","status":"definition-only","scoreable":false},{"id":"CD-04","axisId":"coordination-discovery","name":"Hot-token co-occurrence negative","fixtureRequirement":"Independent wallets that buy the same high-activity token without repeat relationship evidence.","expectedObservation":"Token popularity alone does not produce a high-confidence coordination verdict.","status":"definition-only","scoreable":false},{"id":"LF-01","axisId":"leader-follower-inference","name":"Repeated directed edge","fixtureRequirement":"A wallet pair with the same temporal direction across multiple independently adjudicated shared mints.","expectedObservation":"The provider returns the supported leader, follower, repeat support and direction.","status":"definition-only","scoreable":false},{"id":"LF-02","axisId":"leader-follower-inference","name":"Direction reversal negative","fixtureRequirement":"A wallet pair whose order reverses materially across shared token episodes.","expectedObservation":"The provider abstains or lowers confidence instead of forcing a stable leader/follower direction.","status":"definition-only","scoreable":false},{"id":"LF-03","axisId":"leader-follower-inference","name":"Single-event negative","fixtureRequirement":"A wallet pair observed close together once and never again inside the frozen range.","expectedObservation":"A single event does not become a repeat follow edge.","status":"definition-only","scoreable":false},{"id":"LF-04","axisId":"leader-follower-inference","name":"Lag and support disclosure","fixtureRequirement":"A repeated pair with adjudicated event ordering and lag values for every qualifying observation.","expectedObservation":"Reported lag distribution and support count can be checked against public event evidence.","status":"definition-only","scoreable":false},{"id":"FR-01","axisId":"freshness","name":"Raw-event availability","fixtureRequirement":"A preregistered live swap observed from a common Solana slot reference.","expectedObservation":"Event availability latency is measured from the external observation timestamp, not documentation.","status":"definition-only","scoreable":false},{"id":"FR-02","axisId":"freshness","name":"Position-state update","fixtureRequirement":"A preregistered live trade that changes a known wallet/mint position.","expectedObservation":"Time to the first provider response containing the eligible position update is recorded.","status":"definition-only","scoreable":false},{"id":"FR-03","axisId":"freshness","name":"Provenance update","fixtureRequirement":"A preregistered qualifying funding event for a public benchmark wallet.","expectedObservation":"Time to a correct, evidence-linked direct-funder or path update is recorded.","status":"definition-only","scoreable":false},{"id":"FR-04","axisId":"freshness","name":"Stale-response detection","fixtureRequirement":"Repeated reads around a known state transition with a fixed timeout and retry policy.","expectedObservation":"Stale, timed-out and updated responses are counted separately and reported with p50 and p95 latency.","status":"definition-only","scoreable":false},{"id":"NH-01","axisId":"null-honesty","name":"Measured zero","fixtureRequirement":"A case where the requested quantity is in scope, fully observed and adjudicated as zero.","expectedObservation":"The provider returns numeric zero without conflating it with missing or unsupported data.","status":"definition-only","scoreable":false},{"id":"NH-02","axisId":"null-honesty","name":"Valid empty set","fixtureRequirement":"A successful in-scope query whose frozen evidence contains no matching relationships or records.","expectedObservation":"An empty collection remains distinct from an error, null value and zero-valued measurement.","status":"definition-only","scoreable":false},{"id":"NH-03","axisId":"null-honesty","name":"Unknown value","fixtureRequirement":"A case where missing history or ambiguous evidence prevents a supported answer.","expectedObservation":"The response communicates unknown and does not emit a definitive value.","status":"definition-only","scoreable":false},{"id":"NH-04","axisId":"null-honesty","name":"Unsupported or partial response","fixtureRequirement":"A normalized request containing at least one capability unavailable on the registered provider surface.","expectedObservation":"Unsupported, partial and failed states are machine-readable and are not silently removed from reporting.","status":"definition-only","scoreable":false},{"id":"ET-01","axisId":"evidence-traceability","name":"Transaction references","fixtureRequirement":"A normalized answer whose adjudication depends on one or more public transactions.","expectedObservation":"Relevant transaction signatures are present or resolvable from stable evidence references.","status":"definition-only","scoreable":false},{"id":"ET-02","axisId":"evidence-traceability","name":"Slot and event time","fixtureRequirement":"A timing-sensitive answer with adjudicated slot and timestamp evidence.","expectedObservation":"The answer exposes enough time context to reproduce its ordering and cutoff.","status":"definition-only","scoreable":false},{"id":"ET-03","axisId":"evidence-traceability","name":"Method and data version","fixtureRequirement":"A derived answer for which method or dataset changes can alter interpretation.","expectedObservation":"The response or registered surface identifies the applicable method, API or data version.","status":"definition-only","scoreable":false},{"id":"ET-04","axisId":"evidence-traceability","name":"Stable evidence resolution","fixtureRequirement":"A provider evidence reference captured during the run and rechecked after the publication delay.","expectedObservation":"The reference resolves to relevant evidence and does not require unavailable private session state.","status":"definition-only","scoreable":false}],"runProtocol":{"providerRegistration":["Record the provider, product, API version, paid tier and test date before a run.","Use a documented public interface available to ordinary customers on the disclosed tier.","Disclose any provider-specific configuration needed to request the normalized answer."],"execution":["Freeze the case manifest and its content hash before collecting provider responses.","Run providers against the same case identifiers, cutoff and normalized question.","Record request start, response end, status, retry count and raw-response hash.","Store raw outputs for audit; publish them only when provider terms and data rights permit.","Normalize after capture without silently filling missing fields or converting unknown to zero."],"adjudication":["Blind adjudicators to provider identity wherever the answer format allows.","Resolve disagreements against public chain evidence and record the decision rationale.","Version ground-truth corrections; never rewrite a previously published run in place."],"reporting":["Report numerator, denominator, missing count and confidence interval for every rate.","Publish per-axis measurements before any optional weighted summary.","Keep measured results, documentation observations and internal findings visibly separate.","Report unavailable or untestable capabilities without fabricating a numeric score."]},"scoringPolicy":{"overallWinnerPublished":false,"rationale":"Version 0.1 defines no composite winner because axis weights are use-case dependent and no comparative run has been executed.","requiredPerAxisFields":["sample_size","eligible_count","missing_count","metric_definition","point_estimate","uncertainty_interval","run_timestamp","provider_tier","evidence_coverage"]},"cases":[],"providers":[],"runs":[],"results":[],"nextReleaseGate":{"targetVersion":"0.2.0","requirements":["Publish 25 to 50 frozen public wallet or token cases.","Complete independent adjudication and case-level evidence references.","Publish the manifest hash before querying any tested provider.","Invite providers to review normalization and report factual errors before scoring."]}}