ruby life questions ruby life questions - 1 year ago 69
Ruby Question

Ruby http, net/http, httpclient: can't parse

I am using

gem, it works fine on Windows, just moved to AWS EC2, tried it on and it gets this response:

= Response

HTTP/1.1 920 Unknown
Content-Type: text/html
Date: Wed, 21 Oct 2015 21:42:51 GMT
Connection: Keep-Alive
Content-Length: 23

<h1>File not found</h1>#<HTTP::Message:0x000000023f5168
@body="<h1>File not found</h1>",
[["Content-Type", "text/html"],
["Date", "Wed, 21 Oct 2015 21:42:51 GMT"],
["Connection", "Keep-Alive"],
["Content-Length", "23"]],
#<URI::HTTPS:0x000000023f58c0 URL:>,
#<OpenSSL::X509::Certificate: subject=#<OpenSSL::X509::Name:0x000000024ebe00>, issuer=#<OpenSSL::X509::Name:0x000000024ebec8>, serial=#<OpenSSL::BN:0x000000024de110>, not_before=2015-05-27 00:00:00 UTC, not_after=2017-05-26 23:59:59 UTC>,

It does not work only with this website,
httpclient get
for example works fine. But on Windows I get normal response from
httpclient get
. Butt when using standard NET/HTTP library I get the same 920 response on Windows.

Answer Source

This isn't ec2 related. It's most likely related to the User Agent header sent by the various http library implementations.

For example, they clearly don't like 'wget':

curl -A "Wget/1.13.4 (linux-gnu)"  -v
* Rebuilt URL to:
*   Trying
* Connected to ( port 443 (#0)
* TLS 1.2 # truncated
> GET / HTTP/1.1
> Host:
> User-Agent: Wget/1.13.4 (linux-gnu)
> Accept: */*
< HTTP/1.1 910 Unknown
< Content-Type: text/html
< Date: Thu, 22 Oct 2015 01:16:31 GMT
< Connection: Keep-Alive
< Content-Length: 23
* Connection #0 to host left intact
<h1>File not found</h1>%
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download