mjhjmtu

I am trying this curl -I zomato.com | head -n 1

and I am not getting any response.

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 
0 0 0 0 0 0 0 0 --:- -:-- 0:05:29 --:- -:-- 0 

Is the site protected by firewalls?

Even wget is not working on the site as well. Other sites like google.com are returning 200 response as expected.

$ curl -I --max-redirs 1 "https://www.zomato.com/" 2>/dev/null | grep "^location:" | awk 'print $2'
https://www.zomato.com/some-location-in-mongolia

so...

$ newURL=$(curl -I --max-redirs 1 "https://www.zomato.com/" 2>/dev/null | grep "^location:" | awk 'print $2'| grep -o ".*[a-z]"); curl -I "$newURL" | head -n 1
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
HTTP/2 200 

not sure assisting with scrapping is allowed on this forum, but;

wget --header "User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0" --header "Host: www.zomato.com" --header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" "https://www.zomato.com/bangalore/restaurants/biryani"

did it work for you? It is working from my local machine but when I try from aws serves it gets stuck and does not return anything. I am running below command:

curl -v 'https://www.zomato.com/' -H 'accept-encoding: gzip, deflate, sdch, br' -H 'accept-language: en-US,en;q=0.8' -H 'upgrade-insecure-requests: 1' -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0' -H 'accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'cache-control: max-age=0' -H 'authority: www.zomato.com' --compressed