> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pnbr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# A new File is not searchable yet

> Penumbra makes a new File searchable by exact words first and by meaning later, and by nothing while it reads the text.

Troubleshooting · files

> **Use when**
>
> You added or replaced a File, and a search does not find its content yet, or finds it only by exact words.

> **Looking for something else?**
>
> A search that finds nothing for a File that is ready (problem [Search returns nothing](https://docs.pnbr.io/troubleshooting/search-returns-nothing.md)). A File whose text is wrong (recipe [Check that a File's text arrived intact](https://docs.pnbr.io/guides/check-file-text.md)).

## Signs

* `penumbra files list` shows the File with a status in brackets after its id: `[processing]` for a new File, `[updating]` for a File that Penumbra replaces.
* `penumbra files show <file>` prints `(none yet)` on its `version:` line, or a line that ends with `(lexical ready, semantic pending)`.
* A search by exact words finds the File. A search by meaning does not.
* `penumbra files search` prints `semantic coverage: partial` with two counts, and a warning `semantic_coverage_partial`.
* The coverage reads `full`, and the new File is still missing from the results.

## Why it happens

A File goes through steps. Penumbra stores the bytes, reads the text, splits the
text into passages, and then builds the data for search by meaning. Each step
ends at a different time.

Search reads only versions that are ready. A version that Penumbra still reads
is not in any search, and the coverage count does not include it. So the
coverage can read `full` while your new File is missing. When the text step ends,
the version becomes current and the File answers exact words. The version joins
search by meaning only when every passage has its data.

A PDF page can fail to read for a few minutes. Penumbra then reads that page
again, up to about 10 minutes, before it finishes the version. The pages that
Penumbra already read stay as they are. If the page reads in time, the File
comes out whole, with no warning. If it does not, the File finishes without that
page's text, and `penumbra files inspect` shows a `warning:` line that names the
page.

A replace follows the same steps. The old version stays current until the new
version has its text. Then the old passages leave search. If the new version has
no data for meaning yet, the File drops out of search by meaning until that data
is ready.

## What Penumbra does

By itself:

* Reads the text of a new version in the background, and makes the version current only when exact-word search is ready for it.
* Leaves a version out of search until it is ready. A version that is still processing is not counted in the coverage.
* Reads a failed PDF page again for up to about 10 minutes, so the first version of the File is whole.
* Adds a version to search by meaning only when all its passages have their data.
* Reports the coverage as `partial` or `none`, with a warning, when some versions wait for that data.
* Keeps the old version of a replaced File current until the new version is ready.
* Gives a File with no readable text no passages. Such a File never appears in search.

You must:

* Wait until the File is ready, or search by exact words in the meantime.
* Tell the person which kind of search covers the File now.
* Treat a full coverage count as true only for the versions that were ready.

Not handled:

* The coverage count does not include versions that are still processing, so a full count does not prove that your new File is searchable.

## Do

1. Read the state of the File: `penumbra files show <file>`. For a version that
   is not current yet, run `penumbra files versions <file>`, then
   `penumbra files inspect <file> --version <version-id>`. The second command
   prints `processing:` with `lexical=` and `semantic=`.
2. To wait, add `--wait` to `penumbra files upload` or `penumbra files replace`.
   Or run `penumbra files wait <operation-id>`. The upload printed the
   operation id. Read the `processing:` line when the wait ends. A PDF can take
   up to about 10 minutes longer when Penumbra reads a page again. `--wait`
   waits up to 15 minutes, so it covers that. No other action is needed.
3. If the status is `[processing]` or `[updating]`, or the lexical state is
   `pending`, wait. Do not draw a conclusion from a search yet.
4. If the lexical state is `ready` and the semantic state is `pending`, search by
   exact words now:
   `penumbra files search "<words>" --mode lexical --file-id <file-id>`. MCP:
   call `omnisearch` in lexical mode. Search by meaning after the semantic
   state is `ready`.
5. If the status is `[failed]`, run `penumbra files wait <operation-id>`. It
   prints `error:` with a code. Send the content again with
   `penumbra files replace <file> <local-path>`. If the text is empty, the
   document may be a scan with no readable text (recipe [check-file-text](https://docs.pnbr.io/guides/check-file-text.md)).
6. For a batch of Files, run `penumbra files collections readiness <collection>`.
   It counts the Files that are ready, the Files that Penumbra could not read,
   and the Files that are searchable but not by meaning.
7. If a View limits your credential, the new File stays outside the View until
   it joins a collection that the View includes (problem
   [hidden-reads-as-missing](https://docs.pnbr.io/troubleshooting/hidden-reads-as-missing.md)).

## Don't

* Don't tell the person that a document is missing because one search did not
  find it. The File may still be processing.
* Don't upload the File again to make it appear. The same bytes give the same
  File and change nothing (problem [same-bytes-no-new-version](https://docs.pnbr.io/troubleshooting/same-bytes-no-new-version.md)).
* Don't use search by meaning only (`--mode semantic`) while the File waits for
  its data. That search skips the File and can return nothing.
* Don't start an extraction right after a replace. The extraction reads the
  version that is current when it starts, and that is still the old version
  until the new one is ready.

## Check

* `penumbra files inspect <file>` prints `lexical=ready` and `semantic=ready`.
* `penumbra files search "<phrase from the File>" --file-id <file-id>` returns a passage of the File, and its first line shows `mode: hybrid -> hybrid`.

## Related

Card [file](https://docs.pnbr.io/concepts/file.md). Card [search](https://docs.pnbr.io/concepts/search.md), with rule [search.2](https://docs.pnbr.io/concepts/search.md#rule-search-2). Recipes [bring-in-a-corpus](https://docs.pnbr.io/guides/bring-in-a-corpus.md) and
[check-file-text](https://docs.pnbr.io/guides/check-file-text.md). Problems [search-returns-nothing](https://docs.pnbr.io/troubleshooting/search-returns-nothing.md) and
[same-bytes-no-new-version](https://docs.pnbr.io/troubleshooting/same-bytes-no-new-version.md).

## Keep going

[File](https://docs.pnbr.io/concepts/file.md) · [Search](https://docs.pnbr.io/concepts/search.md) · [Passage and citation](https://docs.pnbr.io/concepts/passage.md) · [Search returns nothing](https://docs.pnbr.io/troubleshooting/search-returns-nothing.md)

Penumbra Grammar 0.6.26


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.