Sample stores

A sample store persists labelled crops — the input + output of a recognition, with optional human/oracle correction. It is to labelled crops what Cache is to recognised crops: a swappable protocol whose adapters back onto filesystem / sqlite / pgvector / S3.

Four things in the library use sample stores:

.export.samples(store) writes one sample per recognised region as an Orchestrator export step.
Worker.few_shot(store) retrieves nearest-neighbour exemplars to splice into the prompt before the engine runs.
.reconstruct.dictionary.from_samples(store) derives a correction dictionary from labelled samples whose label differs from recognition.text, applied during the Orchestrator’s reconstruct phase.
The embedding classifier reads samples to fit a region classifier used by .classify.ml(...).

SampleStore is opt-in. A recipe without one has no on-disk side effects.

Two roads from a SampleStore

A SampleStore is the canonical “user-environment” surface for accuracy improvement. Once you keep one — typically ./.scriva_samples/ checked in alongside the project, or a shared pgvector index for a team — corrections you make today raise accuracy on every run after.

From the same store, scriva derives two complementary improvements, one on each pillar:

                  ┌──────────────────────────────────────────────────┐
                  │             SampleStore (your env)               │
                  │   crop + recognition.text + label (correction)   │
                  └────────────────┬──────────────┬──────────────────┘
                                   │              │
        few-shot exemplars  ◄──────┘              └──────►  derived dictionary
        (Worker side, before                                (Orchestrator side, after
        the engine runs)                                    recognition completes)
                  │                                                  │
                  ▼                                                  ▼
        Worker.few_shot(store)               .reconstruct.dictionary.from_samples(store)

Few-shot road (Worker side). For each region the Worker is about to OCR, the store retrieves its nearest labelled neighbours and splices them into the prompt as exemplars. The model sees “here’s what cells that look like this should read as.” Best when crops are visually distinctive (handwriting styles, stamps, language-specific glyphs).
Dictionary road (Orchestrator side). Pairs (recognition.text → label) observed in the store become a supervised dictionary. The reconstruct step rewrites the same OCR errors on future runs without the Worker ever knowing. Best when errors are systematic (ENCL0SURE → ENCLOSURE, Acme Ind. → Acme Industries, Inc.).

The two roads are complementary, not competing. Use them together and the store earns its keep twice per run:

from scriva import Orchestrator, Worker, SampleStore

store = SampleStore.layered(
    fs=SampleStore.fs(".scriva_samples"),
    index=SampleStore.pgvector("postgresql://localhost/scriva", dim=1536),
)

worker = (
    Worker.openai("gpt-4o")
    .cache(".scriva_cache")
    .few_shot(store)                                           # input road
    .score(method="rendering")
)

recipe = (
    Orchestrator()
    .deskew()
    .split.grid()
    .classify(blank=True, merged=True)
    .recognize(worker)
    .reconstruct.grid()
    .reconstruct.dictionary.from_samples(store, min_observations=2)   # output road
    .export.xlsx("out.xlsx")
    .export.samples(store)                                     # feed the loop
)

Every correction a reviewer makes against the store strengthens both roads. That is the loop the annotation domain pack automates.

The shortest path

from scriva import SampleStore

store = SampleStore.fs(".scriva_samples")        # filesystem
store = SampleStore.sqlite("samples.db")         # sqlite with embedding BLOB
store = SampleStore.pgvector(dsn, dim=1536)      # Postgres + pgvector
store = SampleStore.layered(                     # storage + nearest-neighbour split
    fs=SampleStore.fs(".scriva_samples"),
    index=SampleStore.pgvector(dsn, dim=1536),
)

When a step or decorator takes a store= argument, a string path is shorthand for SampleStore.fs(path) — same convention as cache=.

What a sample carries

class Sample(BaseModel):
    id: SampleId
    crop: bytes                       # PNG bytes of the region
    label: str | None                 # human/oracle text; None until annotated
    recognition: Recognition          # what the Worker said
    embedding: np.ndarray | None      # filled by the store on put() if it has one
    source: SampleSource              # document, page index, region_id, run_id, timestamp
    attrs: dict[str, Any] = {}        # open bag — role, kind, language, app-specific tags

label is intentionally separate from recognition.text: the Worker’s guess and the ground-truth correction are different things, and most workflows want both. A sample with label is None is unlabelled raw training data; setting label to a string promotes it to a labelled example.

Sample is immutable. Derive new ones with .with_label(...) / .replace(**kwargs).

Protocol

class SampleStore(Engine, Protocol):
    name: str

    async def put(self, sample: Sample) -> SampleId: ...
    async def get(self, id: SampleId) -> Sample | None: ...
    async def find(
        self,
        *,
        where: Callable[[Sample], bool] | None = None,
        near: bytes | Region | None = None,   # nearest-neighbour over embedding
        limit: int = 50,
    ) -> Sequence[Sample]: ...
    async def remove(self, id: SampleId) -> None: ...

The same Engine machinery used for Workers and caches gives sample stores a capabilities set. A store that does not implement embedding search declares without Capability.NEAREST_NEIGHBOUR, and construction reports an actionable error when Worker.few_shot(store) or anything else asks for find(near=...) against it — before you load anything.

Capability	Meaning
`NEAREST_NEIGHBOUR`	Supports `find(near=...)`
`PERSISTENT`	Survives process restart
`EMBEDDED_INDEX`	Computes embeddings on `put` without an external call

Built-in adapters

Factory	Class	Backend	Extra	Capabilities
`SampleStore.fs(path)`	`FileSystemSampleStore`	crops on disk + `samples.jsonl`	core	`PERSISTENT`
`SampleStore.sqlite(path)`	`SqliteSampleStore`	one row per sample, BLOB crops	core	`PERSISTENT`
`SampleStore.pgvector(dsn, dim=...)`	`PgvectorSampleStore`	Postgres + pgvector	`pgvector`	`PERSISTENT`, `NEAREST_NEIGHBOUR`
`SampleStore.s3(bucket, prefix=...)`	`S3SampleStore`	S3 for crops, JSONL manifest for the index	`s3`	`PERSISTENT`
`SampleStore.layered(fs=, index=)`	`LayeredSampleStore`	one store for bytes, another for embeddings	core	union of children
`SampleStore.memory()`	`InMemorySampleStore`	dict + in-memory NumPy index	core	`NEAREST_NEIGHBOUR` (no persistence)

SampleStore.layered(...) is the common production shape: filesystem or S3 for the crops, pgvector for nearest-neighbour. Lookups dispatch to whichever child has the relevant capability.

Computing embeddings

If your store has EMBEDDED_INDEX, it embeds on put. Otherwise pass an embedder when constructing the store, or write sample.embedding yourself before put:

from scriva.embedders import OpenAIEmbedder

store = SampleStore.pgvector(
    dsn,
    dim=1536,
    embedder=OpenAIEmbedder(model="text-embedding-3-small"),
)

The embedder protocol is the same one used by Cache.vector and .classify.ml(...) — any ImageEmbedder works.

Active learning loop

The store closes a loop between production runs and reviewer corrections:

Capture. .export.samples(store) writes every region as a Sample with label=None and the Worker’s text on sample.recognition.text.
Triage. .review.hitl(when=lambda r: (r.confidence or 0) < 0.7) surfaces the low-confidence subset to a human. The reviewer sets sample.label on the ones they correct.
Compound. The next run picks the labelled samples up two ways: Worker.few_shot(store) retrieves them as exemplars before the engine runs, and .reconstruct.dictionary.from_samples(store) rewrites recurring errors after recognition.
Audit. scriva.eval(recipe=..., ground_truth=store) measures precision / recall against the labelled subset; the same store is both training data and held-out test set if you partition by attrs.

The store grows monotonically. Track its size in your event stream (export/finished with format="samples"), and rotate by deleting samples whose source.timestamp is older than your retention window.

What not to store

Cached recognitions. That is what Cache is for. A cache keys on the crop hash; a sample store keys on identity and embedding. They look similar but solve different problems.
Orchestrator events. Subscribe to the event stream and write your own log; events are not regions.
Raw pages without a region. Samples are crops. Persist whole pages through Document plugins or your own application layer.

Writing your own backend

Implement the protocol. Two common patterns:

Redis + S3. S3 for crops, Redis for the JSONL index. Five lines on top of redis.asyncio + aioboto3.
MinIO + DuckDB. Same shape, fully on-prem. DuckDB’s vector extension handles find(near=...) once you set Capability.NEAREST_NEIGHBOUR.

The protocol is intentionally narrow so the surface stays writeable.

Sample stores

Sample stores

Two roads from a SampleStore

The shortest path

What a sample carries

Protocol

Built-in adapters

Computing embeddings

Active learning loop

What not to store

Writing your own backend

What to read next