Semantic search: independent spend and abuse audit

Audit date: 2026-10-12. Reviewer: independent automated search/spend review. Human approval: unsigned. Scope: actual account/search/abuse adapters and proposed Workers AI + Vectorize integration; no production deployment, network inference, invoice reconciliation or provider deletion verification is implied. This document supplements security gates, search specification, and the decision ledger in DEVLOG.

Conclusion

A per-account allowance cannot bound a service's total bill. Enabling paid semantic search requires durable global admission before every inference and vector operation, durable owner allowances, and a ceiling on retained vectors/namespaces. Cloudflare billing alerts and free included allocations on a paid account do not supply these controls. Implementation reconciliation below distinguishes new local safeguards from remaining production gates.

The initial snapshot had lexical search in the real account runtime, with semantic provider ports unused and embeddingAllowed:false. No paid semantic work occurred through that runtime. Creating an empty remote index establishes a resource, not a functioning or cost-verified deployment.

Initial findings and required controls

The billing plan must come from verified entitlement, not a requested tier. Paid storage does not implicitly authorize an unlimited semantic allowance. Missing or stale billing should never relax spending limits.

Account farming, network identity and contact bearers

Signup attempts are separately limited to five/minute/IP and 100/hour/subnet. The reviewed stricter admission policy requires one signup per three hours/IP, two/day/IP, 15/day/subnet, five/hour/country, 25/day/country, ten/hour globally and 200/day globally. Region is the two-letter country from trusted request.cf.country; a caller-supplied country header is ignored, and Node uses the shared unknown region. These limits run after challenge/terms/method/handle checks and before expensive account creation. Sequential windows conservatively retain earlier debits when a later limit rejects. Local test configuration deliberately relaxes the IP cooldown; it is not the production setting. These are anchored fixed windows, not precise rolling windows; work can cluster across boundaries. Distributed addresses evade local limits, while the global counter still bounds admission rate. The new lifetime admission counter additionally stops after 1,000 new signup reservations; it never resets with daily windows. This counts new admissions after the counter is introduced, including conservatively charged failed creation, not existing accounts or proven active users. Inventory existing accounts separately. A thousand additional accounts represent up to 500 GB of free payload allowance, before physical copies/index/metadata; maintaining them remains a recurring liability. Without the lifetime gate, the daily rate alone would permit 73,000 admissions over 365 days (36.5 TB of logical allowance). These are allowance scenarios, not observed usage.

Node's actual HTTP adapter strips CF-Connecting-IP, X-Forwarded-For and X-Real-IP, then uses the socket peer. Thus direct client spoofing does not bypass its limiter. Behind a reverse proxy, the socket identity collapses clients into the proxy unless an explicitly trusted adapter is added. Direct calls to the test fetch helper are not evidence about HTTP trust. Worker entry relies on Cloudflare's supplied client-IP header; production ingress, same-zone Worker subrequests and any alternate entry path need a deployment-specific trust test. HMAC pseudonyms reduce retention of raw addresses but neither prove a unique human nor stop address rotation. IPv4 /24 and IPv6 /48 normalization are coarse abuse groupings, with shared-network false positives; they are not an entitlement boundary.

Public contact deliveries require cryptographic bearer verification and intersect recorded grant constraints. Contact message/byte allowances and account blob reservations protect those resources. HTTP contact/message throttles are sender-IP based; one token or many valid tokens from distributed senders can still consume a recipient's resources. A sender must never authorize remote embedding of another account's content. Semantic opt-in and limits must be owner-controlled, and message-triggered background embedding must spend the same owner/global budgets as explicit requests. When indexing reaches a semantic limit, the agreed behavior should preserve lexical indexing and readable authoritative content. Search queue backpressure and message rejection remain separate product policy.

Provider facts checked against official documentation

Prices and limits below were read on the audit date; they must be revalidated when enabling or changing the provider.

Illustrative gross price calculations exclude included allocations, SQL/Worker/R2 costs, fees and operational work. A 1,000-document + 500-query monthly owner allowance at 512 charged tokens/attempt uses at most 768,000 budget tokens. At the conservative 11 micro-USD/attempt, inference admission reserves $0.0165. At 384 dimensions, 1,000 stored vectors and 500 queries give approximately $0.005952/month using Vectorize's published formula. These are estimates under fixed assumptions, not a guaranteed Cloudflare invoice or an economically complete product margin.

At the original lexical paid limit, 100,000 documents × 32 KiB is 3.2768 GB of body text before titles. The implementation stores content in both search_documents and the contentful FTS table: approximately 6.5536 GB of duplicated body text alone. Postings, catalogs, tags, JSON records and tombstones are additional. The 20-million lexical-token ceiling does not prevent this case: punctuation or a very long unbroken word consumes bytes with few lexical tokens. A 100 GB blob entitlement therefore cannot establish safe SQL capacity. Deletes retain revision markers to reject replay, so churn needs its own retention/compaction proof.

Required production checks

1. Demonstrate atomic concurrent admission at owner and global ceilings. Once a limit is reached, count actual mocked provider invocations and prove none start. Repeat after object/process restart, period rollover, provider timeout and malformed accounting/configuration.
2. Charge operations before sending; never refund ambiguous timeouts. Every additional remote attempt needs a fresh durable charge. Count query embeddings, document embeddings, Vectorize reads/writes and relevant cleanup/migration work. Keep cleanup possible when ordinary ingestion is disabled, within a separate bounded operations allowance.
3. Keep stored-capacity reservations across billing periods. Namespace/slot ceilings must not reset each month. Reclaim only after provider deletion completion and local reconciliation; otherwise use conservative unreclaimed capacity. A current-active-vector counter does not count leaked or pending vectors.
4. Bound retries/backoff and give poison records a durable exhausted/deferred state. A finite monthly inference cap limits paid inference but does not cap limitless local alarm/read costs. Prove lexical indexing progresses around semantically deferred items.
5. Verify returned vectors, namespace isolation, live ACL/revision checks, consent revocation during in-flight work and cached cursor use. Demonstrate provider cleanup after revocation and deletion; queued deletion acknowledgment is not erasure evidence.
6. Measure source extraction, SQL bytes/rows, vector churn and request costs with realistic small and large accounts. The implemented budget path uses sixteen fixed partitions and prepaid owner leases; load-test partition contention, owner SQL debit overhead and lease turnover before scale. Shards must share a finite allowance, never each inherit the entire global ceiling.
7. Reconcile provider usage to reservation estimates and document model/pricing drift response. Turn off new paid work on unexplained divergence. Record provider acceptance, persistence and eventual deletion separately. Add alerts as backup, not substitutes for admission.
8. Preserve production release/semantic opt-in gates until privacy, relevance, lifecycle deletion and operational approval are signed. A configured key/binding/index or passing injected tests does not resolve them.

Implementation reconciliation and evidence

The initial high-risk gaps now have application-level controls in the local implementation:

• DurableSemanticBudget requires owner consent and configured provider/global admission. Each owner has 1,000 document embedding attempts, 500 query embedding attempts, 768,000 pessimistically charged input tokens, 576,000 vector operation dimensions and 25,000 micro-USD of estimated operation admission per calendar month. Both free and paid accounts use this launch envelope. It is deliberately narrower than the provisional lexical search plan allowances.
• GlobalSemanticBudget now operates behind sixteen fixed budget partitions. The sum of their defaults is $0.50/day and $5/month of estimated inference/query/upsert admission, 1,000 lifetime admitted owners and two million lifetime reserved upsert slots. These are aggregate limits, not a complete allowance assigned to each partition. Lifetime stock counters survive calendar changes. Prices supplied by callers are ignored; embedded fixed operation costs are 11 micro-USD/inference and four/query or upsert. Accounting corruption now rejects work: independent fault injection found and caused correction of a partial-counter bypass.
• Workers AI receives at most a 480-byte UTF-8-safe excerpt and returns exactly 384 finite dimensions. Charging 512 tokens per attempt is a conservative local estimate, not provider-measured usage. Identical fresh queries still spend admission; cursor continuation avoids repeated remote querying but rechecks consent.
• Vector intent is persisted before remote upsert. Owner lifetime reservations stop after 1,000 upsert attempts, including revisions and ambiguous retries; deletion does not reclaim those reservations. Thus this is 1,000 lifetime mutation slots, not 1,000 freely replaceable active vectors. This conservative usability limitation stays until remote reconciliation can safely reclaim capacity. Successful same-revision replays skip embedding; cleanup-only retry is attempted before deciding whether embedding is needed.
• Owner revocation clears cached sessions and schedules a bounded purge; reenabling waits for local purge work to finish. Semantic enrichment errors are deferred in durable backfill work so ordinary upsert failures do not block later lexical indexing. Backfill/purge markers participate in alarm scheduling. Revocation/deletion still requires eventual provider verification.

Independent node --test tests/semantic-spend-audit.test.mjs passed 13/13 after the accounting and lease fixes. It tests concurrent global ceiling admission, controller reconstruction over the same SQLite state, permanent stock ceilings across monthly rollover, corrupted owner/global accounting, nonrefunded provider timeouts, global denial before inference, UTF-8 excerpt boundaries, ambiguous-upsert cleanup intent with an unreclaimed lifetime slot, exact aggregate conservation across sixteen uneven partitions, one selected vector binding without fan-out, durable lease reuse, midnight response rejection, and concurrent stock-only grant merging without restoring spent money. The last test demonstrated and caused correction of a stale lease overwrite race. This is injected local evidence; it does not show Cloudflare billing, latency, deletion completion or reboot/disk failure behavior. The coordinator owns the combined npm test, actual Worker and browser verification record in DEVLOG.

Remaining production gates: global dollar figures cover admitted inference/vector operation estimates, not ongoing vector storage or all Cloudflare charges. At two million lifetime 384-dimensional remote records, gross stored-dimension cost alone is approximately $0.384/month at the current published rate, even when new paid work stops. If that corpus is queried, the stored-vector component in the published queried-dimensions formula adds another $7.68/month. Thus $5 of admitted operation estimates can coexist with approximately $13.064/month of direct semantic exposure before shared allowances and other infrastructure; at one million vectors the corresponding comparison is $9.032. Owner ceilings may make actual reachable storage lower. Source SQL/FTS, retained tombstones, abuse guard objects, ingress, R2 and alarm costs remain outside these estimates. Cleanup calls bypass ordinary consent/spend admission so revocation can proceed. The later work spend audit adds persisted backoff and an eight-failure automatic pause; operator recovery and operational cost proof remain gates. Tombstone failures now use the semantic failure callback too, allowing lexical queue progress while durable vector cleanup receipts remain; provider deletion completion remains unproven. No hard cap on the entire provider invoice is claimed. Full account erasure, measured SQL metadata ceilings, funded retained-storage liability and production provider reconciliation remain unresolved release work.

Independent economics cross-review reproduced the calculator's account tables, fleet totals, taxes/card assumptions and 23-Plus/4-Pro small-fleet break-even result under its stated assumptions. It identified a memory-unit conversion discrepancy: Cloudflare's own duration example uses 128 MB as 0.128 GB, while the first calculator used 0.125. The economics author corrected the model and regenerated derived totals. Large unrestricted semantic workloads in that model are sensitivity cases, not the smaller launch runtime allowance. Official duration example

Final sharding and lease review

Semantic layout version 1 selects one of four Vectorize bindings by the first SHA-256 digest word of the owner ID modulo four. The account persists its layout/shard and refuses a conflicting route. Each query, write and delete uses that one binding, with the owner namespace applied inside it; ordinary requests do not query every index or consult a global routing directory. The adapter retains an explicit version-zero single-binding path for injection/legacy configuration; changing an account's stored route requires migration. Creating the four indices and configuring bindings is provisioning evidence only. It does not prove a deployed application or migrated corpus.

Budget routing uses the corresponding stable hash modulo sixteen and the fixed names semantic-spend-v1-<partition>. Division plus remainder distributes every configured aggregate limit exactly. For example, 1,000 owner slots become eight partitions of 63 and eight of 62. A budget object verifies the owner maps to its partition and persists an immutable layout/partition marker. The caller's durable lease layout also records version/count. This removes the single per-operation global budget object: each account spends prepaid local balances until it needs another grant.

The final lease size is 128 micro-USD for money and 16 vector upsert slots, requested independently. Query-only claims do not reserve vector stock. Authority admission durably charges a grant before returning it. A local transaction merges that grant into the latest persisted balance, then another durable transaction debits the actual operation before the provider starts. Grant responses must match owner, partition, layout version, UTC day/month and requested amounts. Controller reconstruction reuses persisted balances. A crash after authority debit but before local persistence loses capacity rather than creating extra allowance; ambiguous outcomes receive no refund. A response arriving after its quoted UTC day is rejected. Counters bound admission within UTC periods, not the provider's eventual invoice date for in-flight work.

Unused money and stock leases expire locally at UTC day change; their authority reservations remain consumed, including permanent stock slots. This is conservative and can reduce usable capacity. With $0.50/day divided across sixteen partitions, fixed 128-micro-USD grants can issue at most 3,904 full money grants/day; the leftover 18 micro-USD per partition cannot fund another full grant. One isolated query/day reserves 128 micro-USD while spending 15 locally, so $5/month can admit only about 39,056 full grants, not $5 divided by the query price. Uneven owner/traffic distribution may exhaust one partition while another has spare allowance. There is no automatic borrowing or redistribution and no refund on restart/expiry. These are availability/fairness limitations, not additional spending permission.

Global stock includes leased-but-unused slots; actual stored vectors and successful upserts may be lower. Owner lifetime admission still separately restricts attempts. A migration must preserve both spent and outstanding reservations and avoid two active layouts each allocating the full ceiling. Fixed partition count, different bindings or a fresh namespace must never be treated as a free reset of existing obligations. No automatic migration from the prior singleton budget or identity layout is claimed.

The cost of local per-operation accounting remains: owner budget writes, lease debits, metadata lookups, and occasional remote grant transactions. Leases amortize remote budget RPCs; they do not eliminate accounting SQL/DO costs or demonstrate horizontal throughput. Measure this overhead and retry churn before using the economics model as a launch budget.

Independent identity cross-review examined identity-shards.mjs, its recovery protocol and thirteen passing tests. The versioned 64-bucket-per-family scheme preserves token-checked uniqueness claims and authoritative CAS, with bounded interrupted-operation recovery and conditional cleanup. Ordinary account session reads/CAS remain on the account bucket. No demonstrated critical uniqueness/CAS defect remained in this review. Explicit fresh-layout/migration gating, fixed partition capacity, unscheduled full maintenance, expired-device/failed-operation retention and actual provider load remain operational gates. The combined independent run of semantic spending and identity sharding passed 26/26. This is automated evidence, not human release sign-off.