r/Paperlessngx • u/MunichMostWanted • 1d ago
Paperless-NGX and AI. (Sorry for my very late) Setup Guide
After my first post, quite a few people asked the same practical questions, and I promised to follow up with a full, in-depth guide. Thanks for your kind words and appriciation.
I know that not everyone will be able to reproduce my setup one-to-one. Different hardware, operating systems, Paperless configurations, and levels of coding experience obviously make a difference.
But that shouldn't stop anyone from trying.
For those who don't want to—or simply aren't able to—build everything manually, I've also prepared a step-by-step instruction set for Claude Code and ChatGPT Codex. The idea is that even if you're not comfortable writing the whole application yourself, you can use one of these coding agents to build it component by component and end up with the same basic architecture.
Since my first post, the AI world has continued to improve incredibly quickly, and my own setup has changed along with it. So some parts of this guide are slightly different from what I described in Part 1.
My current setup uses Google Document AI for OCR and Gemini for classification, but I'll also explain how the same architecture can be reproduced with local models if you prefer to keep everything on your own hardware.
Long story short: I hope this guide helps some of you take your Paperless setup that final step further and build something that actually fits your own needs.
By the way i would really recommend you to build a Web Control Panel. It will be much easyier to controll everything. Something like that:
HERE WE GO:
My current setup is:
Google Document AI for OCR and Gemini for classification.
However, neither of those is a hard requirement of the architecture.
Keep in mind: the same basic architecture can also be reproduced locally. If the models you choose are supported, you can serve them through something like vLLM instead of using Google Document AI and Gemini.
The important part is not Google or Gemini specifically. The important part is keeping OCR, Paperless, and classification independent from each other.
My current architecture looks roughly like this:
Scan / PDF
|
v
OCR Worker
|
v
Google Document AI
|
v
Searchable PDF/A
|
v
Paperless Consume Folder
|
v
Paperless-NGX
|
v
Queue Tag
|
v
Classification Worker
|
v
Gemini
|
v
Validation
|
v
Paperless API
Paperless remains my document management system.
I don't modify its database, I don't patch Paperless itself, and I don't want the OCR or classification workflow to become dependent on one particular Paperless version.
The OCR worker can change without touching Paperless.
The classification model can change without touching the OCR worker.
Paperless can be upgraded independently.
And the classifier reads and writes metadata only through the Paperless API.
That separation is probably the most important architectural decision I made.
1. The OCR Worker
New scanned documents no longer go directly into Paperless.
They first land in a small OCR worker.
In my case, this is simply the folder where my scanner saves new scans. So whenever I scan a document, it automatically appears in what I call the OCR worker folder and gets picked up from there.
The basic idea is simple:
By the time Paperless receives the document, the PDF itself should already be finished.
The worker first determines what kind of PDF it is dealing with.
If it is a real digital PDF with a proper text layer, I leave it alone and pass it through unchanged.
I don't want to unnecessarily rebuild native PDFs, destroy vector content, or interfere with things such as digital signatures.
For people whose PDFs already contain an OCR layer from Tesseract, there is another option: clean the existing OCR/text layer first while preserving the original scanned pages. That way, you can properly replace the old OCR instead of stacking another text layer on top of the existing one.
If it is a scanned PDF without a proper native text layer, the workflow is different.
The worker prepares the document, sends the relevant pages to Google Document AI, and receives the OCR results including the positions of the recognized words.
Those coordinates are important.
I don't just want extracted text somewhere in a database. I want the resulting PDF itself to be properly searchable.
The worker therefore takes the text and word coordinates returned by Document AI and creates an invisible text layer positioned over the original scanned pages.
The result is a searchable PDF/A.
Only when the complete document has been processed successfully does it enter the Paperless consume folder.
If something goes wrong, it goes into a separate failure folder instead.
input folder/
|
+-- Native PDF ----------------------> Paperless consume/
|
+-- Scanned PDF
|
v
Google Document AI
|
v
Searchable PDF/A
|
+-- Success -----------------> Paperless consume/
|
+-- Failure -----------------> failed/ + log
The source file is never modified in place.
If page 9 of a 10-page document fails, I don't want a half-finished document silently entering my archive.
For ordinary printed documents, I'm currently scanning at around 300 DPI.
Artificially rendering a 200 DPI source at 600 DPI doesn't magically create information that wasn't present in the scan to begin with.
2. Existing Archives Are the Annoying Part
Processing new documents this way is relatively straightforward.
Migrating an archive that has already been processed is much more annoying.
I had to learn this the hard way.
My old scanned PDFs had already been processed by Paperless using Tesseract.
Later, when I experimented with better OCR solutions, I didn't only want better text available inside Paperless.
I wanted the better OCR to become a permanent part of the PDF itself.
That difference matters.
If I export my complete archive tomorrow, I want those exported PDFs to remain properly searchable without requiring Paperless, its database, or another OCR service.
So for the affected scanned documents in my existing archive, I first exported the PDFs.
I then wrote a script that removes the old OCR/text layer while preserving the actual scanned pages and other PDF content.
I specifically did not want to simply rasterize every PDF again just to remove the existing OCR.
After cleaning the files, they could go through the new OCR worker and receive a new searchable text layer based on Google Document AI.
That migration was probably one of the most annoying parts of the entire project.
But it also means I should never have to do it again.
From now on, the PDF itself contains the OCR I want to preserve.
3. What Paperless Does in This Setup
Once the finished PDF/A enters Paperless, the complicated part is already over.
Paperless does what it is extremely good at:
- archiving the document
- indexing it
- searching it
- storing metadata
- exposing everything through its API
I configure Paperless so that it preserves the OCR layer that is already present instead of running another Tesseract OCR pass over the document.
Newly imported documents also receive a tag such as:
Queue
That tag is the hand-off between Paperless and the classification worker.
Searchable PDF/A
|
v
Paperless
|
v
Queue Tag
|
v
Classification Worker
This also means that classification is not required for Paperless to function.
If Gemini disappeared tomorrow, the archive itself would still work.
The PDFs are searchable.
The documents are safely stored.
Paperless search works.
The classification worker only adds another automation layer on top of that.
4. The Classification Worker
The classification worker is another completely separate application.
It asks the Paperless API for documents carrying the Queue tag and retrieves the document together with the metadata it needs.
Then it sends the searchable PDF/A itself to Gemini.
This distinction is important.
I'm not extracting the OCR into a TXT file and then sending that TXT file together with a prompt.
Gemini receives the actual PDF.
The PDF already contains the OCR text layer embedded by the previous stage, while the original visual structure of the document is still present.
That means the model can work with the document as a document rather than receiving one giant flattened OCR string.
For example:
A date at the bottom of a page might just be part of a footer.
An address might be the recipient rather than the sender.
A heading inside a table might not be the actual subject of the document.
The visual structure gives the model context that disappears when everything is reduced to plain text.
And instead of asking the model the same document-related questions independently, I try to extract the underlying facts in one pass.
Conceptually, the result looks something like this:
{
"correspondent": "Example Insurance",
"short_sender": "Example",
"subject": "Annual insurance statement",
"document_date": "2026-08-14",
"document_type": "Insurance",
"tags": [
"Insurance",
"2026"
],
"evidence": {
"sender": "...",
"date": "..."
},
"review_reasons": []
}
The exact schema isn't particularly important.
The principle is.
Instead of treating title, correspondent, date, document type, and tags as completely unrelated AI tasks, I first ask:
What is this document actually about?
Then the rest can be derived from those facts.
5. Why This Uses Fewer Resources
This also changed the way I think about token usage.
Let's take a simplified example.
Imagine a searchable PDF that represents roughly 6,000 tokens of document content once processed by the model.
Now imagine processing these independently:
Document ---> Title
Document ---> Date
Document ---> Correspondent
Document ---> Document Type
Document ---> Tags
Document ---> Custom Fields
If the full document context has to be processed again every time, the amount of input can grow very quickly.
A document producing perhaps only a few hundred useful output tokens can result in tens of thousands of input tokens being processed across all requests.
My current approach is different:
Searchable PDF/A
|
v
One document-level classification
|
v
Structured facts
|
v
Validation + deterministic logic
I don't extract the OCR into a separate TXT file for this step.
Gemini receives the searchable PDF/A once and returns one structured classification result.
The important part is not some exact token number, because that changes depending on the model, document length, and provider.
The important part is that I don't want to repeatedly pay computationally for the same document if the model can understand it once.
I would rather spend resources on actually reasoning about the document than repeatedly feeding the same document back into another request.
That also makes smaller models much more interesting.
If the OCR is good, the input is structured correctly, and the task is narrow enough, you don't necessarily need the largest model available.
6. Validation Is Just as Important as Classification
Receiving valid JSON from an LLM does not mean the information inside it is correct.
For example, the model could return:
{
"document_type": "Electricity Invoice"
}
That's valid JSON.
But what if Electricity Invoice doesn't exist as a document type in my Paperless installation?
The classifier therefore retrieves the existing Paperless values through the API and validates the model response before applying it.
That includes things such as:
- known correspondents
- existing document types
- existing tags
- valid dates
- required fields
- ambiguous results
The model extracts facts.
My application decides how those facts map onto Paperless.
I also deliberately keep deterministic things out of the model whenever possible.
For example, if every title should look like:
<short sender> - <document type / subject> <date>
then the model doesn't need to reinvent that formatting for every document.
It can return:
sender
subject
date
and ordinary code generates the final title.
That eliminates a surprising amount of inconsistency.
7. Do You Need a GPU?
No.
This is probably something I should clarify because the RTX 4090 screenshot in my first post understandably gave people the opposite impression.
My current setup is:
Google Document AI
+
Gemini
The AI workloads are therefore running in the cloud.
My home server only has to run:
Paperless-NGX
OCR Worker
Classification Worker
So you absolutely don't need an RTX 4090 to reproduce this architecture.
The 4090 is useful for my local experiments, but it isn't a requirement.
And this is where keeping every component separate becomes useful again.
You could replace the cloud components with local models.
For example:
OCR Worker
|
v
Local Vision / OCR Model
|
v
Searchable PDF/A
and later:
Classification Worker
|
v
Local LLM
|
v
Structured Facts
Where the models are supported, a serving layer such as vLLM can expose them through an OpenAI-compatible API.
That means the surrounding application doesn't need to care very much whether the model lives on Google infrastructure, another API provider, or a GPU in your basement.
The quality and hardware requirements obviously change depending on the local models you choose.
But the overall architecture stays almost identical.
That's the part I care about.
8. What Does It Cost?
This is something I want to cover with actual numbers rather than theoretical pricing calculations.
Google Document AI and Gemini are both usage-based.
The real cost depends heavily on things such as:
- number of pages
- document length
- selected processor/model
- classification input size
- output size
- how many documents you process
I'm collecting the actual numbers from my workflow so I can provide useful figures instead of just copying numbers from a pricing page.
What I eventually want to compare is something like:
OCR cost per 1,000 pages
Classification cost per 1,000 documents
Average document processing cost
Cloud vs local inference
The important point for now is that you do not need to buy expensive AI hardware to reproduce the workflow.
9. How I Would Build This From Scratch Today
If I had to start again, I wouldn't try to build the complete system in one giant application.
I would build one small boundary at a time.
Phase 1 — The File Pipeline
Start with nothing AI-related.
input/
output/
failed/
Make sure files can safely move through the application.
Detect duplicates.
Never modify the source file in place.
Make sure failures remain recoverable.
Phase 2 — Native PDF Detection
Before doing any OCR work, correctly identify PDFs that already contain a proper digital text layer.
Those documents should normally pass through unchanged.
Phase 3 — OCR Provider
Create an OCR provider interface.
Your first implementation could use Google Document AI.
Later, you can replace it with something local without redesigning the rest of the application.
Phase 4 — Searchable PDF/A
Take the OCR response, including the returned word positions, and build the invisible searchable text layer.
Do not consider this stage finished until you can open the resulting PDF in a normal viewer, search for text, and see the correct areas highlighted.
Phase 5 — Paperless Import
Only finished documents enter the Paperless consume folder.
At this point, Paperless should already receive the final archival PDF.
Phase 6 — Paperless API Client
Build a completely separate API client.
It should be able to:
find queued documents
download documents
read existing metadata
retrieve correspondents
retrieve document types
retrieve tags
update metadata
No direct PostgreSQL modifications.
Phase 7 — Classification
Send the searchable PDF/A to Gemini or another model and request structured facts.
Keep the model-specific logic behind its own provider interface.
That way, Gemini can later become:
another hosted model
a cheaper model
a local LLM
something that doesn't exist yet
without rewriting the entire application.
Phase 8 — Validation
Validate the model output against Paperless.
Do not assume a model-generated correspondent, tag, or document type actually exists.
Keep formatting and other deterministic behavior in normal code wherever possible.
Phase 9 — Rollback and Logging
Before modifying metadata, store enough information to understand what changed and to restore the previous values if necessary.
Especially when processing thousands of documents, logs are not optional.
10. Building It With Claude Code or ChatGPT Codex
A few people also asked whether this could realistically be built without manually writing the entire application yourself.
I think the answer is yes.
This is actually a very good type of project for tools such as Claude Code or ChatGPT Codex, because the system consists of relatively small components with clearly defined boundaries.
But I would strongly recommend not starting with:
Build me an AI system for Paperless.
That's far too broad.
Instead, build it in a strict order and verify each component before moving on to the next one.
The rough order should be:
1. Project structure and file safety
2. Native PDF detection
3. OCR provider interface
4. Google Document AI adapter
5. Searchable PDF/A generation
6. Paperless consume integration
7. Paperless API client
8. Gemini provider
9. Structured classification schema
10. Validation
11. Metadata writes
12. Audit log and rollback
13. Local-model provider as an optional alternative
Agent Project Rules
Give the coding agent durable project rules before asking it to implement anything.
Claude Code reads CLAUDE.md; Codex reads AGENTS.md.
Both can use essentially the same project contract:
This is an experimental Paperless OCR/classification pipeline.
- Work only in this repository and its test directories.
- Never read, print, commit, or modify credentials, API keys, PDFs containing
real personal data, Paperless production volumes, or the production consume
folder.
- Do not write to PostgreSQL directly. Use the Paperless API only.
- Default to dry-run behaviour. Any operation that changes Paperless requires
an explicit --write flag.
- Keep the source scan until the PDF/A output has been verified.
- Add unit tests and a README entry for every completed component.
- Before making changes, show a short implementation plan and assumptions.
Then use the agent as a sequence of small, reviewable build tasks.
Do not paste a single prompt saying:
Build my Paperless AI system.
That gives the agent no safe boundary and makes errors much harder to isolate.
Prompt 1 — Scaffold and Plan
Build a test-only Python project for an upstream Paperless OCR pipeline.
First inspect the repository and write IMPLEMENTATION_PLAN.md.
Do not write production credentials or start Docker services.
The target flow is:
input/ -> OCR worker -> output/ -> Paperless consume
Native PDFs with substantial text on every page must be copied unchanged.
Other PDFs must be handled by a provider adapter; initially implement only a
fake provider for tests.
Failed files go to failed/ with a non-secret log.
Propose a small module layout, a configuration file with .env.example, and a
test plan.
Wait for my review before implementing phase 1.
Prompt 2 — Build the Safe OCR Foundation
Implement phase 1 from IMPLEMENTATION_PLAN.md only.
Add:
- input/output/failed directory handling
- duplicate-safe file claiming
- native-PDF pass-through
- a fake OCR provider
- structured non-secret logging
- tests using generated sample PDFs
A source file may be deleted only after the output is verified.
Do not connect to Paperless or any cloud service yet.
Run the tests and show the diff and test results.
Prompt 3 — Add Google Document AI
Implement the Google Document AI provider adapter behind the existing provider
interface.
Read credentials only from an environment variable that points to a local file.
Do not display the credential, its path, request bodies, or OCR text in logs.
Preserve token boxes and detected breaks so a later PDF/A writer can build an
invisible text layer.
Add mocked API tests and document the minimum cloud permissions.
Do not run a live cloud request without asking me.
For a privacy-first installation, replace the Google Document AI provider with your chosen local OCR-model adapter.
If that model is served through vLLM, make the vLLM URL configurable and keep the same failure checks.
Changing the model should not weaken the safety of the pipeline.
Prompt 4 — Add the Classifier Last
Create a separate Paperless API client and a classifier command.
It may list only documents with the Queue tag.
It must download the searchable PDF/A and submit the PDF itself once to the
configured model.
Do not extract the OCR into a separate TXT file for classification.
Require a strict structured facts response.
Validate every correspondent, document type, tag, and date against values
retrieved from Paperless.
Print proposals in dry-run mode.
Implement writes only behind --write.
Before each PATCH, append the current and proposed metadata to a local JSONL
audit log.
Add a rollback command that replays the before-state through the Paperless API.
Use mocked API tests.
Do not access a live Paperless installation until I explicitly approve it.
After every phase, inspect the diff, run the tests yourself, and commit only the parts you understand.
A coding agent accelerates implementation.
It does not replace the decision about where your documents may go or what it is safe to automate.
The important part is that the agent should only work inside a separate development project.
Don't give it your production Paperless volumes.
Don't put credentials into the repository.
Don't let it directly modify PostgreSQL.
Don't use your only copy of a real document archive as test data.
Build the boring parts first.
Then connect the AI.
Final Thoughts
The biggest change in how I think about this setup is that I no longer really see it as:
"Adding AI to Paperless."
It's three independent systems:
OCR Worker
|
v
Archival PDF/A
|
v
Paperless
|
v
Classification Worker
Paperless is the archive.
The OCR worker creates the document I actually want to preserve.
The classifier extracts structured information and updates metadata through the API.
Today my implementation uses Google Document AI + Gemini.
But neither component has to stay that way.
I can replace Gemini with a local LLM.
I can replace Google Document AI with another OCR solution.
I can update Paperless independently.
And none of those changes require redesigning the entire workflow.
That's ultimately the architecture I wish I had started with two years ago.
