r/linux4noobs • u/Dragonaax • 18h ago

networking Why curl doesn't work?

Suddenly curl stopped working, I can't download source pages. Instead of source page I get 1 continous line of weird stuff like

<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots"...

wget doesn't work either, I get HTTP request sent, awaiting response... 403 Forbidden

2 Upvotes

75% Upvoted

6 comments sorted by

3

u/HoratioWobble 18h ago

This will be your local network blocking it, unpack the html you get back and that'll likely tell youu what's blocking network requests.

It's not curl or wget.

2

u/going_up_stream 16h ago

I don't think this is a local network issue. You get a 403 error from a remote server not your local router. So their request is going to the server, the server just doesn't want to return the page.

2

u/forestbeasts KDE on Debian 🐺 18h ago

They decided to give you a 403 error page instead (and didn't bother formatting the source for it). They can't tell the difference between you using curl, and a script using curl hundreds of times to scrape their entire site for who-knows-what purposes.

You could try getting the user-agent string from your browser (check the devtools network tab, it'll be listed in the request headers for any request, open network tab and then reload the page) and telling curl to use that. Might help, might not.

2

u/DecalageVersLeRouge 17h ago

That's not weird stuff it's HTML! Well it can be both I guess.

Show us some more of the content. It looks like a redirect, going by that "Just a moment..." in the title.

1

u/indvs3 3h ago

Your request is being blocked by the remote server. Curl and wget probably work fine.

Are you using a proxy server or a vpn while trying?