> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pnbr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Extraction

> Turns what Files say into proposed Graph changes, with their sources cited and their cost known first.

Concept · extraction

An extraction reads pinned Files through one Shape or typeset, costs AI credits, and opens its own new workspace.

> **Use when**
>
> You turn the text of Files into entities and relationships, or you check on, stop or finish an extraction that already started.

> **Looking for something else?**
>
> Facts a person tells you or you write by hand (recipe [Save facts a person tells you as entities](https://docs.pnbr.io/guides/capture-by-hand.md)). Changing entities that are already in the Graph (card [Workspace](https://docs.pnbr.io/concepts/workspace.md)). Reading a File (card [File](https://docs.pnbr.io/concepts/file.md)).

## For

An extraction reads the text of Files and finds entities and relationships that
fit a Shape. It opens its own new workspace and stages them there. Nobody has
approved them yet. A submit of that workspace puts them
in the Graph. An extraction costs AI credits and takes time, so you plan it,
start it once, follow it, and then review what it staged.

## Use when

* A person asks for the knowledge in one File, several Files or a collection.
* You must know the cost of a run before you start it.
* A run is slow, has failed, or ended with some Files unread.
* A run costs more than you want, and you must stop it.

## Not this

* Facts you already know: stage them yourself (recipe [capture-by-hand](https://docs.pnbr.io/guides/capture-by-hand.md)).
* An entity that is already in the Graph and needs a change: use card
  workspace.
* Whether a File is ready to read: use card [Collection](https://docs.pnbr.io/concepts/collection.md) and its readiness check.

## Rules

1. `extraction.1` Plan before you start. **Your job:** run
   `penumbra extractions plan` with the Files or the collection, a Shape and an
   extraction level. It lists the Files and versions the run would read, the
   types, and the credit range. It holds nothing and charges nothing. Give the
   person the range before you start.
2. `extraction.2` A run takes one perspective: one Shape or one typeset.
   **Enforced:** `penumbra graph extract` refuses both or neither with
   `invalid_arguments`. **Your job:** to use several Shapes, start one
   extraction for each Shape. Each one opens its own workspace.
3. `extraction.3` A run reads Files by id or reads one collection, never both.
   **Enforced:** the CLI refuses both or neither with `invalid_arguments`. One
   run takes 1 to 256 Files, and a larger collection is refused (card rule
   [collection.5](https://docs.pnbr.io/concepts/collection.md#rule-collection-5)).
4. `extraction.4` A run reads the versions that are current when it starts.
   **Your job:** a File that you replace, or add to the collection, after the
   start is not read by that run. Start another run for it (problem
   [new-version-not-extracted](https://docs.pnbr.io/troubleshooting/new-version-not-extracted.md)).
5. `extraction.5` Starting a run costs AI credits and, with an API key or as a
   connected agent, waits for approval (card rules [approval.2](https://docs.pnbr.io/concepts/approval.md#rule-approval-2) and
   [approval.3](https://docs.pnbr.io/concepts/approval.md#rule-approval-3)). **Your job:** pass `--request-id <key>` and keep it.
   **Enforced when you ask:** the same key returns the same extraction and does
   not charge again. Without the flag, the CLI makes a new key, and a second
   run charges a second time.
6. `extraction.6` Penumbra charges for work done. A failed run is not charged.
   A stopped run is charged for the work done, and a queued run that is
   stopped is not charged. **Your job:** tell the person what a stop costs
   before you stop a run that is already reading.
7. `extraction.7` What an extraction finds waits in the workspace that the
   extraction opened. **Your job:** review it and submit it (recipe
   [review-and-submit](https://docs.pnbr.io/guides/review-and-submit.md)). Nothing is in the Graph until the submit returns
   `applied` (card rule [workspace.2](https://docs.pnbr.io/concepts/workspace.md#rule-workspace-2)).
8. `extraction.8` Leave that workspace alone until the extraction ends.
   **Your job:** do not modify it while the run is working. When the run ends in
   part, decide on Try again before you read or edit the workspace (rule
   [extraction.13](https://docs.pnbr.io/concepts/extraction.md#rule-extraction-13)). Then review the items with
   `penumbra graph workspace show`.
9. `extraction.9` An extraction compares each item with the entities that the
   Graph already has. The comparison runs the first time anyone reads,
   validates, edits or submits the workspace. When it matches or flags an item,
   it saves that as a change to the workspace. An item with the same Shape type
   and the same name as exactly one live entity becomes an update to that
   entity. The workspace shows it as `updates existing`. Every other item stays
   a new entity. **Your job:** check each `updates existing` item before you
   submit. Run `penumbra graph workspace keep-new` for one that is a different
   thing (problem [extraction-duplicates-existing-entity](https://docs.pnbr.io/troubleshooting/extraction-duplicates-existing-entity.md)). Search for an item
   whose entity has another name, because only the exact name matches.
10. `extraction.10` Follow a run with its extraction id.
    **Your job:** run `penumbra graph extractions "<workspace>" --run <id> --watch`, with the workspace name that `penumbra graph extract` printed. The count of items so far can go down as well as up. Ctrl-C ends
    the watch and does not stop the run. Use the id exactly as
    `penumbra graph extract` printed it. Do not build one.
11. `extraction.11` Stop keeps what the run has found. **Enforced:**
    `penumbra graph extractions stop` does not wait for approval and is safe to
    repeat. A stopped run is not offered Try again: it is refused with
    `retry_not_offered`. **Your job:** after a stop, the rest of the Files are
    not read, and the stop message does not say which Files were read. To read
    the rest, start a new extraction for those Files (recipe [extract-files](https://docs.pnbr.io/guides/extract-files.md)).
    A new extraction charges for the Files it reads.
12. `extraction.12` Try again finishes a run that failed, or that ended in part
    because the engine could not finish some work (the CLI says
    `Partly finished`). **Your job:** run
    `penumbra extractions try-again <run-id>`. It reads the same Files at the
    same versions with the same Shape, reuses the work that finished, and
    charges only for the rest. It costs credits and waits for approval under
    the same rules as a start. It closes the old workspace and opens a new one
    under a new name. Follow the workspace name and the run id that it prints
    (rule [extraction.10](https://docs.pnbr.io/concepts/extraction.md#rule-extraction-10)), and not the old ones.
13. `extraction.13` Try again is offered only on the newest run of a lineage,
    while its workspace is still open and has not been changed. **Enforced:** it
    is refused with `retry_not_offered`, `retry_in_progress`,
    `retry_review_edited` or `retry_stale`. **Your job:** decide on Try again
    before you read or edit a workspace that ended in part. The first read can
    change it (rule [extraction.9](https://docs.pnbr.io/concepts/extraction.md#rule-extraction-9)). If the workspace changed, submit it or
    close it (rule [workspace.11](https://docs.pnbr.io/concepts/workspace.md#rule-workspace-11)), then start a new extraction.
14. `extraction.14` The extraction level sets how thorough a run is, `standard`
    or `fast`. **Enforced:** a level that the project does not offer is refused
    with `extraction_level_unavailable`, and the reply names the levels it
    offers.
    Without `--level`, the run uses the project's default.
15. `extraction.15` Every extraction opens its own new workspace, and you do not
    name one. **Enforced:** the start takes no workspace argument, and the
    answer names the workspace it opened. That workspace holds only the items of
    that run. **Your job:** use that name to follow, review and submit the run.
    Do not capture into it.

## Failure signs

* A start returns `pending` or exit code 4: problem [change-waits-for-approval](https://docs.pnbr.io/troubleshooting/change-waits-for-approval.md).
* A start is refused with `collection_too_large`: rule [collection.5](https://docs.pnbr.io/concepts/collection.md#rule-collection-5).
* A start is refused with `review_not_empty`: problem [extraction-incomplete](https://docs.pnbr.io/troubleshooting/extraction-incomplete.md).
* A run ended and some Files were not read: problem [extraction-incomplete](https://docs.pnbr.io/troubleshooting/extraction-incomplete.md).
* After the owner approved the start, you cannot follow the run: problem
  [cannot-follow-approved-extraction](https://docs.pnbr.io/troubleshooting/cannot-follow-approved-extraction.md).
* The workspace holds items that the Graph already has: problem
  [extraction-duplicates-existing-entity](https://docs.pnbr.io/troubleshooting/extraction-duplicates-existing-entity.md).
* A File has a new version and the Graph shows the old text: problem
  [new-version-not-extracted](https://docs.pnbr.io/troubleshooting/new-version-not-extracted.md).

## Doors

| Door | Call |
| - | - |
| CLI | `penumbra extractions plan`, `penumbra graph extract`, `penumbra graph extractions`, `penumbra graph extractions stop`, `penumbra extractions try-again` |
| MCP (default) | None. The default catalog has no call that starts or follows an extraction. Use the CLI or the full catalog. |
| MCP (full catalog) | `extraction_start` starts a run, opens its own workspace and returns its cost estimate. `extraction_progress`, `extraction_findings`, `extraction_stop` and `extraction_try_again` follow and finish it. No MCP tool plans one. |
| Program | None. `execute` offers no method that starts or follows an extraction. |
| SDK | `ProjectGraph.extract`, `GraphWorkspace.extractions`, `semanticTasks.fileExtraction.plan`, `semanticTasks.fileExtraction.tryAgain` |

A Kit with `extraction_start` also has the four tools that follow and finish a
run.

## HTTP reference

* [Get extraction capabilities](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-capabilities.md)
* [Plan file extraction](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-plan.md)
* [Start extraction](https://docs.pnbr.io/api/http-graph-extractions-start.md)
* [List extractions](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-list.md)
* [Get extraction](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-get.md)
* [Get extraction results](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-results.md)
* [Stop extraction](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-stop.md)
* [Retry extraction](https://docs.pnbr.io/api/http-semantic-tasks-file-extractions-try-again.md)

## Keep going

[Workspace](https://docs.pnbr.io/concepts/workspace.md) · [Approval](https://docs.pnbr.io/concepts/approval.md) · [Collection](https://docs.pnbr.io/concepts/collection.md) · [Shape](https://docs.pnbr.io/concepts/shape.md) · [File](https://docs.pnbr.io/concepts/file.md) · [Entity](https://docs.pnbr.io/concepts/entity.md) · [An extraction did not read every File](https://docs.pnbr.io/troubleshooting/extraction-incomplete.md) · [The owner approved an extraction and you cannot follow it](https://docs.pnbr.io/troubleshooting/cannot-follow-approved-extraction.md) · [An extraction found an entity the Graph already has](https://docs.pnbr.io/troubleshooting/extraction-duplicates-existing-entity.md)

Penumbra Grammar 0.6.26


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.