Skip to main content
Concept · workflows
A workflow is a written job. Its steps name operations Penumbra declares, and it is checked for free and priced by a plan. Saving or publishing it runs nothing; a signed-in person starts a run of a published version.

Use when

You design a job as steps, check a draft, save or publish it, or price a version for a person. You also start a run, follow it or stop it.

Looking for something else?

A one-time extraction or crawl, which you run directly (card Extraction). Reading or changing Files and the Graph (cards file and workspace). Who must approve a change (card Approval).

​
For

A person describes a job in one sentence. A workflow writes that job as a JSON document. It holds a purpose, the inputs it takes, a trigger and the steps. Each step names one operation, gives it its inputs, and says what to do when it ends in each way it can. You design the document against the catalog (penumbra workflows catalog). You check it, which costs nothing. You save it as a draft, publish a draft as a numbered version, and plan a version to see what it should cost. A draft may have problems. Only a publish refuses them. None of these acts runs a step or spends a credit. A run does: the person starts a run of a published version with its inputs, and each step then charges as its own operation charges. The words a person sees: A step that can end in a way the document does not handle is a problem to fix, and the document says which way. A workflow can limit its spending per run with guards.max_credits_per_run. A person sees that as the spending limit per run.

​
Use when

  • A person asks you to turn a job they repeat into steps.
  • You must find out why a draft is not ready to publish.
  • You must tell a person what a published version should cost, and which steps have no figure.

​
Not this

  • A one-time extraction or crawl: run it directly (card Extraction).
  • Reading or changing Files or the Graph by hand (cards file and workspace).
  • Deciding who may approve a delete, a publish or a spend (card Approval).

​
Rules

  1. workflow.1 A step names only an operation that declares its outcomes. Enforced: check and publish refuse any other step with unknown_module, unknown_operation or outcomes_undeclared. Your job: read penumbra workflows catalog first, and use only the steps it lists (problem workflow-step-unavailable).
  2. workflow.2 Every way a step can end needs a route. A success continues, a step with no effect skips what depends on it, a refusal stops, and a failure stops without a retry. A partial result has no default. Enforced: a step that can end partial and has no route in on.partial is the problem unhandled_outcome. Your job: ask the person whether what landed is enough, then route it (problem workflow-unhandled-outcome).
  3. workflow.3 A step reads only the workflow’s inputs and the steps before it. A value is a literal, ${inputs.name}, ${steps.id.outputs.path}, ${item} inside a map, or ${size(reference)}. Enforced: check refuses a later step, an unknown input, a missing input and a value of the wrong type, and names each in step.field form.
  4. workflow.4 A check costs nothing and stores nothing, and a draft may have problems. Enforced: penumbra workflows check lists every problem and exits with code 3 while there is one. create and update save the draft and list its problems. Your job: check the file until it is clean before you publish.
  5. workflow.5 A workflow id is a lowercase letter followed by lowercase letters, digits, dots, colons, dashes or underscores, at most 128 characters. Enforced: the document’s workflow_id must be the id it is saved under. The ids catalog, check and runs are reserved, and a taken id is refused with workflow_id_taken (409).
  6. workflow.6 A draft changes at the revision you read. Enforced: an update at any other revision is refused with workflow_revision_conflict (409), and the refusal gives the current revision. Your job: run penumbra workflows show <workflow>, apply your change to that draft, and update with a new --request-id.
  7. workflow.7 A publish makes the next numbered version, and a version never changes. Enforced: the version’s number becomes the document’s workflow_version, so the digest of a published version can differ from the digest check printed. A draft that is already the newest version answers that version. A draft with a problem is refused with workflow_invalid (422), and the refusal lists each problem. Your job: change a workflow by updating the draft and publishing again.
  8. workflow.8 A plan shows what each step should cost and runs nothing. Enforced: only an extraction start with a literal scope, Shape and level gets a figure. Any other paid step has no figure, and its basis begins not_estimated. A step that costs nothing has the basis no_cost. Your job: read every basis. Tell the person which paid steps have no figure, and never present the total of a run as its whole cost.
  9. workflow.9 Each plan has a digest and is kept. Enforced: a plan is kept by its digest on the version and never replaced. The same version with the same inputs gives the same digest. Your job: plan again after a Shape or a collection changes, and give the person the new figures (problem workflow-plan-changed).
  10. workflow.10 Reading takes less authority than changing. Enforced: a project member reads, and a key needs workflows.read or workflows.write. A project editor, admin or owner changes, and a key needs workflows.write. A person below editor is refused with workflow_change_requires_editor and a key without the scope with workflow_change_requires_scope (403). Access limited to a View is refused with view_operation_unavailable.
  11. workflow.11 Saving, publishing and planning run no step and send nothing outward. Enforced: no step runs, no credit is spent, and the approval floor holds none of these acts. Your job: tell the person that a published version does nothing until a person starts a run.
  12. workflow.12 A person starts a run, and only a person. Enforced: penumbra workflows run <workflow> needs a signed-in person who is a project editor, admin or owner. A key, an assistant connection and the in-app Assistant are refused with workflow_run_requires_person (403). The run acts for that person, with the permissions its steps need and no more. A workflow with no published version is refused with workflow_not_published (409). Inputs that do not fit the workflow’s inputs are refused with a diagnostic for each field (422). Your job: ask the person to start the run, give them the command and the inputs it needs, and never start one with a key.
  13. workflow.13 A run is followed by its id. Enforced: a start answers the run’s id, with the status pending until Penumbra starts it. The command penumbra workflows runs get <run> shows the status and what the run has charged. For each step it shows every attempt with how it ended, what it charged and what it produced. The same --request-id starts a run once. Your job: report a run by its status and its charge, and say which attempts failed or were refused and why.
  14. workflow.14 A run stops when it is asked to. Enforced: penumbra workflows runs cancel <run> needs a project editor, admin or owner (or a key with workflows.write). A step already sent finishes, nothing new starts, and a run that has already ended is refused with workflow_run_finished (409). Your job: say that cancelling does not undo a step that finished.

​
Failure signs

​
Doors

​
Keep going

Extraction · Shape · Collection · Approval · Credential · View · Write a job as a workflow · A step can end in a way the workflow does not handle · A step names an operation this project cannot use · A plan gives a different digest than the one the person saw · A run cannot start because the workflow has no published version