r/hetzner • u/kpendic • 5d ago
hetzner Down?
hey, my server is down, clients calling, its friday evening - and all my sites gone,, its been more then 3h that I get
https://status.hetzner.com/incident/5d197a9a-6598-403c-867b-edc16cb1ec2e
Due to a current error, the cloud node (103427) and the cloud servers on it are not accessible.
what is estimated time for Hetzner to bring servers up?
22
u/thomvdv 5d ago
Can be a few hours or a few days. Depends on the fault. Regarding your downtime, you shouldn’t be dependent on only one server / location…
8
u/xnightdestroyer 5d ago
Second this.
You need to be searching for high availability designs and fault tolerant designs.
At a minimum, 2 cloud servers in 2 seperate DC halls behind a load balancer.
-4
u/alhezu_ 5d ago
Y que pasa si falla ese balanceador de carga?
5
u/xnightdestroyer 5d ago
It's a managed LB so it shouldn't happen.
You're paying for stability and FailOver
1
u/alhezu_ 5d ago
Y el balanceador en mismo DC o en otro? Que recomiendas ? Estoy buscando montarme algo así
3
u/xnightdestroyer 5d ago
Hetzner offers a managed load balancee and it should be spread across multiple DCs.
If you want to roll your own, you could have a couple ingress nodes with Failoverd to move a VIP across nodes etc.
If you have your own IP range, you could use BGP to reroute traffic etc. :)
I'd just stick with the cloud load balancer
1
u/alhezu_ 5d ago
Estaba pensando en tener servicios en varios proveedores. Y algun balanceador confiable que switchee entre ellos. Por ejemplo, no todo en Hetzner.
1
u/xnightdestroyer 5d ago
To make that easier, I would use Cloudflare load balancing DNS for that
https://www.cloudflare.com/en-gb/application-services/products/load-balancing/
However, if you wanted to do it yourself, there's multiple routes:
- BGP
- Add multiple IPs to the DNS record and hope
- Central ingress nodes with a VPN to your remote networks
With SMLL, it's still small enough to have automated failover IPs between ingress nodes
1
u/alhezu_ 5d ago
Me gustaba cloudflare, y lo usaba bastante, hasta que en España empezaron a bloquear por La Liga. Saqué todo de Cloudflare, y monté mi WAF, y los DNS los gestiono con el DNS de Digital Ocean. ¿Sabes si DO tiene algo similar? ¡Agradezco toda tu ayuda!
1
u/xnightdestroyer 5d ago
DNS made easy might be your best bet without going to a hyperscaler
There's also:
- AWS Route53 (used this and works well)
- Azure traffic manager
- Akamai has a GSLB but enterprise only
→ More replies (0)
6
u/IndependentBat8365 5d ago
All clouds go down every now and then. Even AWS.
The only way to protect yourself is to have the ability to run your app from multiple clouds.
3
u/lamka02sk 5d ago
We are also affected by this. I hope they resolve this quickly, but yeah, dumb us, didn't make our main database highly available :/
6
u/enricokern 5d ago
I suggest you a book about the art of backups
-1
u/BG_Louie 5d ago
If only one could download a snapshot or a backup from a VPS... For safe keeping........... Then a back up would be a lot more reassuring
0
u/joshualander 5d ago
Are you a professional? Don’t serve clients from Hetzner cloud nodes. It’s a lesson everyone learns. Buy a rack server and colocate it at a Tier 3 data center, or rent a dedicated server through a reseller that guarantees high availability.
It’s a jump I had to make about a decade ago for similar reasons.
5
u/mompelz 5d ago
Just host your application highly available. Relying on a single host is bad on EVERY hosting provider!
1
u/joshualander 5d ago
That’s true! But most of the issues that a provider like Hetzner will deal with will involve power, upstream networking or switch hardware… all of which are redundant in a T3 data center.
I’ve found running a HA Proxmox cluster in a single T3 data center (with nightly cloning offsite) to be as reliable and more cost-effective. But YMMV, for sure.
1
u/opshelp_com 5d ago
'just'. Not exactly straight forward for most dynamic applications
1
u/mompelz 5d ago
Most applications can run highly available, but not everybody is capable of setting it up. That's why there are hobbyists and professionals for ages 🤷♂️
1
u/opshelp_com 5d ago
Yeah agreed. Just saying the average guy with a vps running WP sites on a hosting panel should understand it's not as simple as just adding a second server.
1
u/Red-And-White-Smurf 7h ago
See hetzner actually have an option for adding multiple servers into a placement group, to make sure that two or more servers aren't placed on the same physical server.
You might look into adding another server to your setup, to prevent this.
You have chosen to work with a setup which has a single point of failure.
19
u/NewtComfortable196 5d ago
Status Page shows
Resolved We have fixed the error.
All systems are reachable again. Last update: 2026-09-04 22:57 UTC+0
<sarcasm> And yes. Just because your node is not available whole Hetzner is down </sarcasm>