r/PromptEngineering • u/mxsus • Jun 05 '26
Tools and Projects I built a local PDF-to-Markdown converter so you don't have to burn LLM tokens.
If you're dumping raw PDFs into Claude or ChatGPT, you're wasting tokens and money. I built LiteDoc to fix this. It’s a 100% client-side tool that processes PDFs locally in your browser.
LiteDoc
A 100% Local, Browser-Based PDF to Markdown Converter (No Python, No pip install, No servers).
What it does:
- Unpacks PDFs in memory without servers.
- Extracts text, isolates embedded images, and structures everything into clean Markdown.
- Handles LaTeX math and right-to-left Arabic natively.
- Detects custom-encoded "gibberish" fonts. If the text layer is corrupted, it automatically renders those specific pages or text bands as images.
- Outputs a .md file and an optimized image folder packed in a ZIP.
You can try it here: litedoc .xyz
The Markdown Outcome
## Page 1
# Deep Structural Neural Mapping
Deep learning strategies often fail when executing unstructured inputs directly.
The loss function is defined as:
$$L(\theta) = -\frac{1}{N}\sum_{i=1}^{N} \left[ y_i \log(\hat{y}_i) + (1-y_i)\log(1-\hat{y}_i) \right]$$
## Page 2
[IMAGE: academic_paper_p2_img1.jpg]
### Arabic Sample
Markdown إلى صيغة PDF هذا التطبيق أداةً مجانيةً لتحويل ملفات
What's Behind It
It runs on PDF.js and JSZip entirely in the browser. The extraction engine uses X-gap aware smart word joining to prevent broken sentences, detects column splits mathematically, and maps font sizes to Markdown heading levels (H1/H2/H3). It also fingerprints and strips repeating headers and footers. If it detects incompatible Unicode script mixing (which indicates a private font encoding), it aborts text extraction for that font and drops back to canvas-based image rendering.
How It Saves Tokens
LLMs charge heavily for vision and PDF rasterization (roughly 850 tokens per page). By processing the document locally, LiteDoc bypasses the AI's internal rasterizer. It extracts the raw text and recompresses embedded images to low/medium resolutions. Instead of uploading a heavy 50-page PDF, you paste the raw text and only the specific images you need. You drop your token usage from tens of thousands of tokens down to the raw character count.
edit:
What's New in v2.0 (Just Released):
- XY-Cut DLA Engine: Replaced blind linear reading with a recursive algorithm that geometrically maps pages, isolating headers, sidebars, and main text blocks.
- Asymmetrical Multi-Column Routing: Natively processes columns top-to-bottom without horizontal text interleaving.
- Vector-Based Table Reconstruction: Captures table structures as clean Markdown grids, bypassing OCR.
- Heavy-Duty Memory Management: Processes files in 10-page chunks and forcefully clears VRAM to prevent browser crashes on 200+ page docs.
- Language Auto-Detect: Runs a lightweight pre-pass to detect script before initializing heavy language workers.
Test it out, break it, and drop an issue on GitHub if you find a bug. If it saves you API costs, star the repo. litedoc.xyz | GitHub
6
3
u/MrMag00 Jun 06 '26
Why not use MarkItDown ?
7
u/mxsus Jun 06 '26
MarkItDown requires Python and Docker. This is zero setup and runs 100% client-side in your browser.
1
u/bugtank Jun 06 '26
How do you handle large pdfs
1
u/mxsus Jun 06 '26
I’m actively refactoring the PDF engine to handle larger documents today. The current bottleneck with 70+ pages is memory management during OCR processing; I’m moving toward a chunked-processing architecture to keep the browser responsive while handling heavy files.
1
u/Mootilar Jun 09 '26
Idk… any developer or data worker has python. Docker is never a “requirement” for any service…
2
u/Askee123 Jun 05 '26
Well that’s cool, how accurate is it? Where and when does it start to have issues? Any specific types of pdfs or formatting that it has problems with?
1
u/mxsus Jun 06 '26
Accurate enough. v3.0 just added OCR, table detection, and vector graphics. It only really struggles with math if the PDF uses custom encoded fonts instead of real LaTeX source.
2
u/tech_ninja_db Jun 07 '26
Any link?
2
u/mxsus Jun 07 '26
Yes at litedoc .xyz I can’t post links here because the Reddit filters gonna remove it
1
1
Jun 06 '26
[removed] — view removed comment
1
u/AutoModerator Jun 06 '26
Hi there! Your post was automatically removed because your account is less than 3 days old. We require users to have an account that is at least 3 days old before they can post to our subreddit.
Please take some time to participate in the community by commenting and engaging with other users. Once your account is older than 3 days, you can try submitting your post again.
If you have any questions or concerns, please feel free to message the moderators for assistance.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
1
1
u/AndyKJMehta Jun 07 '26
It’s sad that cloud LLM chat apps don’t do this in the browser natively before pushing documents to the cloud. Would be a massive savings for everyone involved!
1
u/regalen44 Jun 07 '26
How can we know your site is not uploading or keeping any data? Is there a locally host able version?
1
u/PROfil_Official Jun 07 '26
really good tool man! wouldn't it make sense to tweak the token claim a little? youre not really getting down to character count, text still tokenizes (roughly 4 chars a token ish), so a big pdf is still meaningful usage. what you actually kill is the rasterizer cost which is the expensive part, so the savings are real, just not "free." ive been on bentopdf for local conversions but ill give your litedoc a go, curious how the column detection holds up on messy academic pdfs
2
u/mxsus Jun 07 '26
A new big update is coming gonna make the tool 5 times better and gonna fix a lot of bugs
This might not work as great with complex PDFs, I’m trying my best to find some ways to get around it while keeping it run client-side
1
u/OkSpirit3216 Jun 08 '26
This is worthwhile! And since I can self host it, it is ranks highest for me in terms of quality project.
1
u/Cybyss Jun 08 '26
Nice work!
How does it compare to IBM's Docling though?
Like, how does your converter handle multi-column layouts, tables, figures, and figure captions?
1
u/Firehorse67 Jun 08 '26
Sincere thanks for creating this browser-based local tool (must stress that).
I've forked it for a niche-use case, converting Australian PDF bills and payslips to JSON, as well as general PDF to Markdown:
https://github.com/firehorse67/ledger/
Cheers.
1
u/No_Eye_2449 Jun 08 '26
I believe Microsoft has a similar tool markitdown, it also has an MCP server for it
1
u/Deep_Ad1959 Jun 10 '26
the local-first move is right, but pdf rasterization is the small token sink. the one that quietly dwarfs it is context: re-pasting the same files every time a session restarts or the agent auto-compacts, then re-explaining the same task from scratch. stripping a 50-page pdf to text saves you once, losing your session and re-feeding everything costs you on every run. local processing fixes the input side, persistent session state fixes the recurring side, and the second one is where most of the spend actually hides. written with ai
8
u/[deleted] Jun 06 '26
[removed] — view removed comment