Get LLM Judge Evaluation

Returns HTTP 200 in every state, including while still running. Branch on status, not on the status code. The response shape is identical across statuses — the same seven fields are always present.

GET /v2/judge/llm/{evaluation_id}

Retrieve an LLM Judge evaluation

cURL

curl --request GET \
  --url https://api.eu.linqalpha.com/v2/judge/llm/{evaluation_id} \
  --header 'X-API-KEY: <api-key>'

200

{
  "error": null,
  "payload": {
    "evaluation_id": "2fde560a-e7eb-45e0-8cd3-04e57c50d1d3",
    "status": "completed",
    "verdict": {
      "scores": {
        "Factuality": 4,
        "Completeness": 4,
        "Relevance": 5,
        "Grounding": 5
      },
      "reasoning": "The answer's core claims align with the supplied sources...",
      "overall_score": 4.5
    },
    "judge_model": "gpt-5.4-mini",
    "input_reference_count": 2,
    "verified_reference_count": 2
  }
}

What it does

Returns the current state of one LLM Judge run, using the evaluation_id from Execute LLM Judge. The response shape is identical in every state — the same six fields are always present — so a client reads status and never has to branch on the body’s shape.

This returns HTTP 200 while the judge is still running. Branch on the status field, not on the status code.

Statuses

status	Meaning	verdict	judge_model	input_reference_count / verified_reference_count	Retry?
pending	Queued or running.	null	null	null	Keep polling
completed	Finished.	Fixed-rubric verdict — see below	The model that produced it	Both non-null	No
failed	Something broke on our side.	Same shape as completed, but scores / overall_score are null and reasoning carries a customer-safe sentence	null	null	Yes

A GET immediately after a POST returns pending. That is expected, not an error.

Failure detection: verdict !== null && verdict.overall_score === null is the failure marker. reasoning on such a verdict is the customer-safe sentence that used to be surfaced as a top-level reason field — that field was removed to keep the shape uniform.

Unlike Get Agent Judge Evaluation, this endpoint does not currently return excluded. The LLM Judge has no exclusion pre-filter — the same status enum is exposed for consistency, but only pending / completed / failed are produced today.

Polling

Poll no more than once every 10 seconds. Typical runs settle in a few minutes; a sensible client gives up after around 30 minutes and treats the run as failed.

import time
import requests

url = f"https://api.eu.linqalpha.com/v2/judge/llm/{evaluation_id}"
headers = {"X-API-KEY": "<your-api-key>"}

deadline = time.time() + 30 * 60          # give up after ~30 minutes
while time.time() < deadline:
    payload = requests.get(url, headers=headers).json()["payload"]

    if payload["status"] == "completed":
        print(payload["verdict"])
        # Coverage signal — see the note below.
        print(f"verified {payload['verified_reference_count']}/{payload['input_reference_count']}")
        break
    if payload["status"] == "failed":
        # `verdict.overall_score is None` is the failure marker; `reasoning` is the
        # customer-safe reason that used to live under a separate `reason` field.
        print("failed:", payload["verdict"]["reasoning"])   # safe to retry
        break

    time.sleep(10)                        # still pending

Response

verdict uses one shape on every non-null run — four dimensions on a 1–5 scale, plus reasoning and a server-computed overall_score. completed populates all three; failed uses the same shape with scores / overall_score set to null and the customer-safe reason on reasoning. Every successful LLM Judge run therefore returns scores directly comparable across callers and time, and a caller reads one field to tell success from failure.

completed

{
  "error": null,
  "payload": {
    "evaluation_id": "2fde560a-e7eb-45e0-8cd3-04e57c50d1d3",
    "status": "completed",
    "verdict": {
      "scores": {
        "Factuality": 4,
        "Completeness": 4,
        "Relevance": 5,
        "Grounding": 5
      },
      "reasoning": "The answer's core claims align with the supplied sources...",
      "overall_score": 4.5
    },
    "judge_model": "gpt-5.4-mini",
    "input_reference_count": 2,
    "verified_reference_count": 2
  }
}

failed

{
  "error": null,
  "payload": {
    "evaluation_id": "2fde560a-e7eb-45e0-8cd3-04e57c50d1d3",
    "status": "failed",
    "verdict": {
      "scores": null,
      "reasoning": "The evaluation could not be completed.",
      "overall_score": null
    },
    "judge_model": null,
    "input_reference_count": null,
    "verified_reference_count": null
  }
}

The response fields

Field	Type	Notes
evaluation_id	UUID	The id from POST /v2/judge/llm.
status	enum	pending / completed / failed.
verdict	object | null	Non-null on completed and failed. See the two response examples above.
judge_model	string | null	The model that produced the verdict. Non-null only on completed.
input_reference_count	int | null	How many references you submitted. Non-null only on completed.
verified_reference_count	int | null	How many references the verifier actually produced a verdict for. Non-null only on completed. See below.

verified_reference_count vs input_reference_count — read them together

On a clean run the two are equal.

When they differ (verified_reference_count < input_reference_count), a chunk of the verifier failed and only the surviving verdicts reached the judge — the verdict is still valid, but partly graded on incomplete verification. Downstream you may want to weigh those runs differently or resubmit.

On non-completed statuses both are null.

Empty references — both counts are 0

Submitting references: [] is a valid request; the verifier is skipped and the judge grades on prompt / query / answer only. In this case input_reference_count and verified_reference_count are both 0, not null.

Per-reference verification verdicts, the evidence behind them, and internal cost/latency accounting are used to produce the verdict but are not part of this response.

Isolation

Judge runs are scoped to the organization that submitted them. An evaluation_id belonging to another organization returns 404, exactly as an id that does not exist — the two are indistinguishable by design.

The two endpoints — this one and Get Agent Judge Evaluation — read from disjoint id spaces. An id from POST /v2/judge/agent returns 404 here, and vice versa. Idempotency keys are scoped the same way, so the same key on the two endpoints yields two separate evaluations.

A malformed evaluation_id is rejected with 400 before any lookup.

Authorizations

X-API-KEY · string · header · required

Path Parameters

evaluation_id · string<uuid> · required

The id returned by POST /v2/judge/llm. Ids from POST /v2/judge/agent return 404 here — the two endpoints read from disjoint id spaces.

Response

200

application/json

The evaluation's current state. Same shape in every status.

error · unknown

payload · object

Show child attributes