scribasedocs

scribase judge

Points at a table and a column, asks one plain-language question once, and labels every matching row with a typed answer: a yes/no probability, a choice from a closed set, or a score along a described scale — plus a confidence. Each answer is recorded in an auto-created <table>_judgments table, so there is no ALTER TABLE, no webhook, and no second service to stand up.

This needs an adjacent AI-judgment product, not just a database — Supabase and Neon do not have one. Scribase's own portfolio does (TypeSafe, the same System One model, jev-latest, that powers Jev), so this ships as a CLI command that calls it directly: set TYPESAFE_API_KEY and run it against any table in any project.

Terminal
scribase judge acme shop production --table tickets --columns subject,body \
  --question "Is this message about a billing problem?"

or against a bare connection string, including a local or non-Scribase Postgres database:

Terminal
scribase judge --db-url "$DATABASE_URL" --table tickets --columns subject,body \
  --question "Is this message about a billing problem?"

How the answer is typed

Omit both --choices and --levels for a yes/no probability (noul). Add one of the other two for a different shape:

Flag Answer shape Example
(neither) A probability from 0 to 1 "Is this about billing?" → 0.99
--choices <name[:description],...> One of a named, described set "What should support do?" → escalate
--levels <name[:description],...> (ascending) A position on an ordered scale "How urgent is this?" → 0.0–1.0 across the levels

A name with no :description uses the name as its own description.

Options

Flag Description
--table <t> Required. table (defaults to the public schema) or schema.table
--columns <a,b,c> Required. Columns sent to the model as the row's state
--question "<text>" Required. The judgment, in plain language
--id-column <col> Primary key column read and recorded (default id)
--choices <name[:desc],...> Closed set of options; mutually exclusive with --levels
--levels <name[:desc],...> Ordered scale, low to high; mutually exclusive with --choices
--where <sql> Row filter (default: every row)
--limit N Rows judged per run, 1–500 (default 25)
--out-table <name> Override the default <table>_judgments table name
--dry-run Judge and print every answer; write nothing

Example

Terminal
scribase judge acme shop production --table tickets --columns subject,body \
  --question "What should support do with this ticket?" \
  --choices "refund:A billing charge needs to be reversed,\
escalate:The product is broken for users right now,\
backlog:A feature request with no urgency" \
  --where "resolved_at is null" --limit 50
Text
1 -> refund (confidence 1.00)
3 -> escalate (confidence 1.00)
Wrote 2 judgment(s) to "public"."tickets_judgments".

The <table>_judgments table

Created once, on first use, with every answer column present regardless of question type (so a table can accumulate different kinds of judgments over time):

SQL
create table tickets_judgments (
  id text not null,
  input jsonb not null,
  question text not null,
  answer_type text not null,
  choice text,
  noul double precision,
  score double precision,
  confidence double precision,
  judged_at timestamptz not null default now()
);

Notes

  • Needs TYPESAFE_API_KEY, a TypeSafe System One API key — this talks to TypeSafe's API directly, not the Scribase management API, aside from resolving <org> <project> <env> to a connection string.
  • --where is sent as a raw SQL predicate against the table you named; it runs with the same credentials the connection string already carries.
  • A column value containing a newline cannot be read safely by this command's row parser; it reports that row's fetch as a failure rather than silently misreading it.
  • NULL and an empty string are not distinguished in the state sent to the model (both read back from Postgres as an empty field).