Skip to main content
Troubleshooting · extraction

Use when

An extraction finished and its workspace holds items. The Graph may hold the same things, or an item says updates existing and you doubt it.

Looking for something else?

Two entities that are already in the Graph (problem Two entities in the Graph are one thing). A File with a new version (problem A new version was never extracted).

​
Signs

  • An item reads updates existing <Type> "<name>", and you are not sure it is the same thing.
  • An item stays new and reads (new: ...), and validate warns existing_match_not_linked with the ids of the entities that have its name.
  • The workspace holds an item whose name differs from the name of an entity that penumbra graph search returns.
  • Validate warns existing_match_unchecked: the workspace has not been compared with the Graph yet.

​
Why it happens

An extraction reads Files. The comparison with the Graph runs afterward, when someone first reads, validates, edits or submits the workspace. It uses the exact Shape type and the exact name, with case and spacing ignored. It does not use similarity, so it cannot see an entity that has another name. An item stays new when it cannot be matched to one entity.

​
What Penumbra does

By itself:
  • Compares each extracted item with the live entities of its Shape type, the first time the workspace is read, validated, edited or submitted.
  • Turns an item into an update when exactly one entity has the same Shape type and name. It keeps the item’s handle and adds its citations to the entity’s own.
  • Shows that update as updates existing, and shows a changed property as a proposed change, the same as any update.
  • Keeps an item new, flags it possible_duplicate, and warns existing_match_not_linked in four cases. Several entities have its name. The Graph holds too many entities of that type to compare. Another item already updates that entity. The entity was written under another version of the Shape.
  • Merges repeated items inside one extraction result, and flags a possible duplicate between items of one extraction result.
  • Never compares an entity that you staged by hand with the Graph.
You must:
  • Check every item that says updates existing before you submit.
  • Search the Graph for each item that stays new, and for names that the exact match cannot see.
Not handled:
  • An item whose entity has another name, such as an abbreviation or a legal suffix, stays new.
  • Two entities that are one thing are not merged (recipe resolve-duplicates).

​
Do

  1. After the extraction ends, list the workspace: penumbra graph workspace show "<workspace>". Note the items that say updates existing and the items with (new: ...).
  2. For each updates existing item, compare it with the entity: penumbra context entity <id> --json. The patch shows what the submit changes. If they are one thing, go on. If not, run penumbra graph workspace keep-new "<workspace>" <handle>. The item becomes a new entity again and keeps its relationships. A later read does not match it again.
  3. For each item that stays new, read the warning. It names the entities that have the name. If one of them is the same thing, check it out with penumbra graph workspace hydrate "<workspace>" --node <id>. Stage the change as a patch, and cite the passage that supports it (card rule entity.6). Then remove the new item with penumbra graph workspace modify "<workspace>" <file>, where the file is {"remove":{"entities":["<handle>"]}}.
  4. For each name, search the Graph: penumbra graph search "<name>" --type <Type> --kind node. Search again with --mode hybrid for a spelling that differs. Skip a mode the project cannot run (card rule search.2).
  5. Validate and submit the workspace (recipe review-and-submit). Submit the revision that validate printed, because the first read can change the revision.

​
Don’t

  • Don’t submit first and tidy up later. A repeat in the Graph needs recipe resolve-duplicates, and its delete can wait for approval.
  • Don’t assume that an item without updates existing is new to the Graph. Only an exact name matches.
  • Don’t keep an updates existing item without a look. A namesake of another thing gets the new properties and citations.
  • Don’t search only by the item’s name. The Graph may hold the same thing under another name.

​
Check

  • Every updates existing item is the entity it names.
  • For every item that stays new, penumbra graph search finds no entity that is the same thing.
  • After the submit, penumbra graph search "<name>" --type <Type> returns one entity for each name you checked.
Card extraction, rule extraction.9. Card rule relationship.3. Recipe review-and-submit. Recipe resolve-duplicates. Problem merge-duplicates. Problem same-name-two-entities.

​
Keep going

Extraction · Workspace · Entity · Relationship · Two entities have the same name · Two entities in the Graph are one thing · Review a workspace and submit it