Skip to main content
Troubleshooting · extraction

Use when

A run ended with Partly finished or Stopped early, you must know whether a run read every File, or a start is refused with review_not_empty.

Looking for something else?

Files whose text arrived wrong (recipe Check that a File’s text arrived intact). A start that waits for approval (problem A change returned pending). Items that repeat an entity the Graph has (problem An extraction found an entity the Graph already has).

​
Signs

  • The CLI prints Partly finished: N Files weren't fully read and the line next: penumbra extractions try-again <run-id>.
  • The CLI prints Stopped early. and says that the Files were not fully read.
  • penumbra graph extractions "<workspace>" --run <run-id> --json has an incomplete object. It holds parts_delivered, parts_total and files_not_fully_read.
  • A new extraction is refused with review_not_empty and the status 409.

​
Why it happens

A run reads its Files in parts. If the engine cannot finish some parts after its retries, Penumbra delivers the parts that finished and ends the run as partly finished. A stop works the other way. The run delivers what it has found so far and reads no more. In both cases the workspace holds only what was read, so the Graph that you build from it has gaps. A start gets the name of its workspace from the Shape, the first File and the request key. If a workspace with that name already holds items, Penumbra refuses the start before it admits the run.

​
What Penumbra does

By itself:
  • Delivers the parts that finished when the engine cannot finish the rest, and ends the run as partly finished.
  • Lists the Files that were not fully read in the run’s status by File ID, and marks the ones that were partly read. The CLI text prints only their count; --json lists them.
  • Charges only for the work that finished.
  • Delivers what a stopped run had found and marks the run as stopped early, without naming the Files it did not read.
  • Offers Try again for a failed run. It offers it for a partly finished run only while its workspace is open and unchanged. It offers it on the newest run only.
  • Does not offer Try again for a stopped run, and refuses it with retry_not_offered.
  • Continues a tried-again run from the parts that finished, closes the old workspace and opens a new one under a new name.
  • Counts the first read of a workspace as a change when the comparison with the Graph matches or flags an item. Try again is then refused with retry_review_edited.
  • Refuses a start with review_not_empty and the status 409 when the workspace named for that start already holds items.
  • Refuses that start before it admits the run, so it holds no credits and charges nothing.
  • Does not raise review_not_empty on Try again.
You must:
  • Read how the run ended before you say that the Graph is complete.
  • Decide on Try again before you read or edit a partly finished workspace.
  • Tell the person which Files were not read, or that Penumbra does not say.
Not handled:
  • After a stop, nothing lists the Files that were read.
  • The refusal review_not_empty says “Start it again with a new idempotency_key”. It does not say that the old key admitted nothing.

​
Do

  1. Read how the run ended: penumbra graph extractions "<workspace>" --run <run-id>. Add --json to read incomplete.
  2. If the run is partly finished or failed, decide on Try again now, before you read or edit the workspace (card rule extraction.13). Tell the person that it costs credits, and that a key or a connected agent waits for approval (card rule extraction.12). Run penumbra extractions try-again <run-id> --request-id <key> and keep the key.
  3. Follow the workspace name and the run id that Try again prints, and not the old ones (card rule extraction.10).
  4. If Try again is refused with retry_review_edited, retry_not_offered or retry_stale, do not edit the workspace further. Submit what it holds (recipe review-and-submit), or close it (card rule workspace.11). Then start a new extraction for the Files that were not read (recipe extract-files).
  5. If the run was stopped, start a new extraction for the rest (card rule extraction.11). Compare the Files in the plan with the Files that the items cite, because the stop does not say which Files were read.
  6. If a start is refused with review_not_empty, nothing was admitted or charged. If an earlier start with that key had been admitted, the same key would return that run with replayed and no refusal. Start again with a new --request-id.
  7. Tell the person which Files were not read, and what you did about them.

​
Don’t

  • Don’t tell the person that the Graph is complete after a run that ended in part.
  • Don’t read or edit a partly finished workspace before you decide on Try again.
  • Don’t start a new extraction of all the Files to finish a partly finished run. Try again reads the rest and charges only for the rest.
  • Don’t use a new key to repeat a start whose reply was lost (recipe recover-after-a-lost-reply). A new key is right only after a refusal such as review_not_empty, because that refusal admitted nothing and a key repeats the same start only.

​
Check

  • The run that finished shows Ready to review and no Partly finished line, or the person has decided to leave the rest unread.
  • You can name the Files that were read, the Files that were not, and the cost.
Card extraction, with rules extraction.10, extraction.11, extraction.12 and extraction.13. Card workspace, with rule workspace.11. Card collection, with rule collection.5. Recipes extract-files and check-file-text. Problems change-waits-for-approval and extraction-duplicates-existing-entity.

​
Keep going

Extraction · Extract entities from Files · Workspace · Collection · File · Check that a File’s text arrived intact · A change returned pending · An extraction found an entity the Graph already has