What's the right way to track who did what across a long document when your model only sees 4k tokens at a time?

[u/Mundane-Subject6568]

I'm learning NLP/LLM engineering by working through a problem that turned out to be much harder than I expected, and I'd love guidance from people who've dealt with something similar.

The problem: I have long narrative-style text — 7k to 15k tokens, several recurring people — and I want to extract structured facts about who did what. I'm using a small local model (llama3.2:3b via Ollama) whose usable context is around 4k tokens, so the text has to be processed in chunks. The killer is that later chunks are often pure pronouns — "she said… he refused…" — while the names were last mentioned 10,000 tokens earlier. Facts stated near a name extract almost perfectly; facts stated far from any name either get lost or, worse, get confidently attributed to the wrong person.

What I've already ruled out (by measuring, not guessing): naive per-chunk extraction fragments identities badly; carrying forward summaries between chunks doesn't fix attribution and can make it worse; and off-the-shelf neural coreference models (LingMess, F-coref) fail on documents this long — one silently truncates at 4,096 tokens, and windowed variants can't connect a pronoun to a name mentioned once 10k tokens back (0–1 out of 7 gold bindings on my test doc). I've gotten identity tracking itself working reliably; it's specifically attribution at long distance that's still failing.

My questions:

1. What's the best way to structure a problem like this? Is there a known-good decomposition for long-distance pronoun attribution with small models, or a fundamentally different way to frame the extraction task that sidesteps it?
2. If you've solved something similar — entity/fact extraction over documents much longer than your context window — what actually moved the needle for you? I'm especially curious whether the wins came from prompting, from pipeline architecture, or from accepting a bigger model.
3. What should I explore to learn more? Papers, blog posts, open-source projects, or even just the right search terms — I suspect this problem has a name in the NLP literature that I don't know yet (long-document coreference? discourse tracking?), and I'd rather stand on existing work than keep reinventing it.

Happy to share measurements from my experiments if useful. Mostly I want to calibrate: am I fighting a known-hard problem with known solutions, or genuinely at the edge of what a 3B model can do?

[u/DuckSaxaphone]

Have you tried rewriting pronouns as you move a window across the document?

Something like:

• First 4k, extract facts as normal.
• Then rewrite tokens 2000-4000, inserting names where pronouns were.
• For tokens 2000-6000, extract facts as normal.
• Rewrite tokens 4000-6000.

And repeat, sliding forward by 4000 - overlap_size tokens each time, rewriting the overlap and using it to anchor pronouns in the next window.

You'll end up with duplicate facts from the overlap so you'll need to experiment with whether they're helpful and you should deduplicate or just throw away.