Keyslinger Keyslinger - 3 months ago 14
HTTP Question

How to make a cURL request that produces the same response headers as Firefox

When I browse to a page with Firefox and click a download link, the following headers are shown when I inspect the request in network inspector:


Connection: keep-alive
Content-Disposition: attachment; filename="example_file.mp3"
Content-Length: 35181829
Content-Transfer-Encoding: binary
Content-Type: audio/mpeg
Date: Fri, 19 Aug 2016 18:19:02 GMT
Keep-Alive: timeout=60
Server: nginx
X-Powered-By: PHP/5.4.45


However, when I use cURL to visit the same address, I get this:


Connection: keep-alive
Content-Length: 1918
Content-Type: text/html; charset=UTF-8
Date: Fri, 19 Aug 2016 20:46:23 GMT
Keep-Alive: timeout=60
Server: nginx
X-Powered-By: PHP/5.4.45


How can I form a request with cURL that gives me the same response as Firefox?

Answer
  1. In Firefox, open up the Net tab in the developer options(F12) and open the URL of the page you need.
  2. Take note of all the Request Headers in the request sent to the server:

Example:

Accept  
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding 
gzip, deflate
Accept-Language 
nl,en-US;q=0.7,en;q=0.3
Connection  
keep-alive
Cookie  
_ga=GA1.2.598213448.1471644637; _gat=1
Host    
mariannesdelights.be
User-Agent  
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
  1. Put all the headers in an array in this way

$headers = array('HeaderName:HeaderValue','HeaderName2:HeaderValue2');

  1. Use the php function curl_setoption() to set the headers in the request:

curl_setopt($ch,CURLOPT_HTTPHEADER,$headers);

That should produce the exact same HTTP-Response headers.