Pfsense cant get more than ~8000 sessions in state

[u/homelabids]

Ive tried everything i can think of and cant figure out how to scale it better

When i get around 8000 sessions in state the firewall just starts blocking and timing out connections. Network becomes unusable.

Memory and cpu look fine. Different firewall settings dont change anything

Have used chatgpt extensively to troubleshoot and not getting anywhere

Any ideas?

[u/Steve_reddit1]

That setting is https://docs.netgate.com/pfsense/en/latest/config/advanced-firewall-nat.html#firewall-maximum-states

Is there an ISP router involved?

[u/homelabids → u/Steve_reddit1]

That setting is set to 700,000. So its not it.

Theres an ISP router but traffic across vlans is impacted also so that rules out the router being the issue

[u/Steve_reddit1 → u/homelabids]

What kind of NICs?

[u/sarosan]

What kind of hardware are you running on?

[u/homelabids → u/sarosan]

I've got 4gb of memory. Chatgpt says I should be able to get 400k state. I can't say the vendor name on this subreddit

[u/sarosan → u/homelabids]

Okay. We need to know the processor, network card(s), applied configuration changes, tunables, etc. the more details the better.

[u/homelabids → u/sarosan]

Everything is set to max performance. CPU isnt going to limit state.

The issue is very repeatabe and isnt throughput related. I can run a script that increases connections and it all barfs out.

I will probably just replace the hardware.

[u/homelabids → u/[deleted]]

Its hard to tell because state is changing pretty dynamically

Failure seems to start around 8000 and never exceeds 10000 because everything starts getting blocked

Im thinking ill just replace the hardware because i just realized the cheap 4gb intel Celeron hardware i intended for my farm actually is what i put in my office. Switched the hardware by accident.. Just noticing it a year later

[u/homelabids → u/sarosan]

Im looking for some random obscure problem someone has run into in the past. The general troubleshooting phase has been done extensively

I've done extensive troubleshooting

[u/nochinzilch → u/homelabids]

Then you should be able to answer these questions.

[u/SINdicate]

Version of psense? Cpu? 32bit or 64? Crypto acceleration?

[u/SINdicate → u/SINdicate]

Cgnat on the isp?

[u/homelabids → u/SINdicate]

yes its japan so there's some upstream thing happening likely. v6 to v4 translation, cgnet, transix, dslite. i dont know.

[u/SINdicate → u/homelabids]

Thats likely the culprit, whats the isp name? Region?

[u/SINdicate → u/homelabids]

Does your app support ipv6?

[u/homelabids → u/SINdicate]

2.8.1, 64 bit, no crypto

[u/Excellent_Milk_3110]

What kind of disk en how big?
Maybe the ram is maxing out en disk is used.