> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pnbr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Check that a File's text arrived intact

> You know whether each File's text arrived whole before anyone extracts from it, and the person knows which Files you cannot trust.

Guide · files

> **Use when**
>
> You are about to extract from or cite Files, and one is a PDF, an Office file or a scan. Or a search for its words finds too little.

> **Looking for something else?**
>
> A search that finds nothing for a File that reads well (problem [search-returns-nothing](https://docs.pnbr.io/troubleshooting/search-returns-nothing.md)). A new File that is not searchable yet (problem [new-file-not-searchable-yet](https://docs.pnbr.io/troubleshooting/new-file-not-searchable-yet.md)).

## Outcome

For each File you name one verdict. The text is intact. Part of it is missing
or changed. Or there is no text. The person knows which Files you will not
extract from, and why.

## Before you start

* Have the File IDs, or the collection name. List a collection with
  `penumbra files collections files "<collection>"`.
* Know what each document should contain. Ask the person for a page, a table or
  a number that must be there. Without it you can check only that some text
  arrived.

## Steps

1. Check that Penumbra finished with the File. A PDF can take up to about 10
   minutes longer when Penumbra reads a page again. No action is needed; wait
   with `penumbra files wait <operation-id>` (problem [new-file-not-searchable-yet](https://docs.pnbr.io/troubleshooting/new-file-not-searchable-yet.md)).
   * CLI: `penumbra files show <file>`. Expect `status: ready` and a version
     line that ends `(lexical ready, semantic ready)`. For a whole collection,
     run `penumbra files collections readiness "<collection>"`. Expect a
     `Ready to use:` line and four counts: searchable, still processing, no
     text to search and couldn’t be read. The four add up to the number of
     Files (card rule [collection.3](https://docs.pnbr.io/concepts/collection.md#rule-collection-3)).
   * MCP: `list_files` with the collection and the current version expanded in
     the full catalog (card [collection](https://docs.pnbr.io/concepts/collection.md)). The default catalog has no grouped
     report.
2. Read the processing of one version.
   * CLI: `penumbra files inspect <file>`. Expect `processing: ready`,
     `search_eligible=true` and `content operation: succeeded`. A File with no
     text says `search_eligible=false` and `succeeded_with_warnings`.
   * Read every `warning:` line below it. Each one says which part of the
     text may be incomplete, in the File's own terms, such as a page, a table
     or a picture. The code in brackets, such as `table_grid_unavailable`, names
     the case. A File with a warning is not intact until you have checked that
     part in step 4. No `warning:` line does not prove that the text is intact; step 4 still applies.
   * A File that failed has no version. `penumbra files show` says
     `status: failed`, and `penumbra files text` answers `resource_unavailable`.
     The collection readiness names the reason, such as a damaged file, and says
     that trying again will not help.
3. Read the extracted text.
   * CLI: `penumbra files text <file>`. A PDF shows a line before each page,
     such as `--- page 3, read from the PDF text ---`. It says whether Penumbra
     read the page from the PDF text, from the page image, or from both. The
     last line says `end of text`, or says where it stopped and the `--start`
     that continues. Read all of a short document. For a long one, read the
     first and the last parts with `--start` and `--max-bytes`.
   * To read one page, use `penumbra files text <file> --page <n>`. The last
     line says `end of page <n>`.
   * MCP: `read` with `file:<id>`. It pages with `next_cursor`. `read_file_text`
     lists the pages of each window, and takes `page`.
4. Compare the text with the document.
   * Look for the headings, the table rows, the numbers, the names and the
     special characters that you were told to expect. A table that became lines
     of words, a heading that became body text and a missing page are the usual
     losses.
   * For a PDF, render the pages and look at them next to the text:
     `penumbra files pages <file> --out <folder>`. Expect `pages: N of N
     rendered` and one image for each page. Compare one page at a time: read it
     with `--page <n>`, and open the image of that page. Check first the pages that
     a `warning:` line names and the pages read from the page image. If you
     cannot view images, ask the person to open the File in the App.
   * MCP: `open_file` shows the source to the person.
5. Check that search finds the words.
   * CLI: `penumbra files search "<a phrase from the document>" --mode lexical --file-id <file-id>`.
     Expect one result with the passage and its `location`, such as the page.
   * The hit must contain the phrase, not only a near word.
6. Give each File a verdict, and write down what you saw.
   * Intact: the text, the numbers and the structure that you checked match.
   * Partial: some pages, rows or characters are missing or wrong. Say which.
   * None: no text. A scan or a picture may need reading by a person.
     Do not extract from a File that is partial or has no text until the person
     decides (recipe [extract-files](https://docs.pnbr.io/guides/extract-files.md)).
7. If the source is the problem, ask the person for a better copy, such as a
   PDF exported from the original or the text itself. Replace the File with it
   (recipe [new-edition](https://docs.pnbr.io/guides/new-edition.md)). A copy with the same bytes makes no new version
   (problem [same-bytes-no-new-version](https://docs.pnbr.io/troubleshooting/same-bytes-no-new-version.md)).

## Check

* Every File has a verdict, and a reason for each one that is not intact.
* For each File that you call intact, you compared at least one heading, one
  number and one table row or special character with the document.
* `penumbra files collections readiness "<collection>"` has no File in "still
  processing". If one is, wait and check again (problem
  [new-file-not-searchable-yet](https://docs.pnbr.io/troubleshooting/new-file-not-searchable-yet.md)).

## If it goes wrong

| You see | Read |
| - | - |
| The text lost a table, a heading or characters | steps 4 and 7 |
| A `warning:` line names a part of the text | step 4: compare that part |
| A new File is not searchable yet | problem [new-file-not-searchable-yet](https://docs.pnbr.io/troubleshooting/new-file-not-searchable-yet.md) |
| A File says `status: failed` and has no version | step 2, then ask for a better copy |
| Search finds no passage of a File whose text reads well | problem [search-returns-nothing](https://docs.pnbr.io/troubleshooting/search-returns-nothing.md) |
| A replace with the same bytes made no new version | problem [same-bytes-no-new-version](https://docs.pnbr.io/troubleshooting/same-bytes-no-new-version.md) |
| A text read stops before the end | step 3: continue from the `--start` |

## Worth offering

* For a large set, offer to check one File of each kind, such as a scan, a
  table and a form. Extend the check if the sample shows losses.
* Offer to list the Files you would extract from and the ones you would leave
  out. The person can then choose before any credits are spent.
* Offer to extract from the Files that are intact first, and to leave the
  partial ones for after the person decides.

## Report

List each File with its verdict. For each File that is partial or has no text,
say what is missing, how you saw it, and what you suggest. Say what you
compared with the document. Say which Files are safe to extract from now.

## Keep going

[File](https://docs.pnbr.io/concepts/file.md) · [Collection](https://docs.pnbr.io/concepts/collection.md) · [Passage and citation](https://docs.pnbr.io/concepts/passage.md) · [Search](https://docs.pnbr.io/concepts/search.md) · [Bring a set of documents into the project](https://docs.pnbr.io/guides/bring-in-a-corpus.md) · [Extract entities from Files](https://docs.pnbr.io/guides/extract-files.md) · [A new File is not searchable yet](https://docs.pnbr.io/troubleshooting/new-file-not-searchable-yet.md) · [Identical bytes make no new version](https://docs.pnbr.io/troubleshooting/same-bytes-no-new-version.md)

Penumbra Grammar 0.6.26


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.