Here's my request:
var request = (HttpWebRequest) WebRequest.Create("https://mtgox.com/");
request.CookieContainer = new CookieContainer();
request.AllowAutoRedirect = false;
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
request.Headers[HttpRequestHeader.AcceptLanguage] = "en-gb,en;q=0.5";
request.Headers[HttpRequestHeader.AcceptCharset] = "ISO-8859-1,utf-8;q=0.7,*;q=0.7";
request.Timeout = 5000;
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
request.Method = "GET";
Using Microsoft Network Monitor, I found that
HttpWebRequest would get stuck at a stage where it's supposed to send back a client key exchange. It simply didn't. The server duly waited for it, but it never came.
What fixed it was forcing HttpWebRequest to use SSL3 instead of TLS (even though TLS is supposed to automatically turn into SSL3 if necessary):
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
Why this is so I guess I'll never know - just one of those mysterious things that would take more time to figure out than anyone I know is willing to spend...
One thing that was different about the captures: the TLS variant had an "Alert" entry in the Server Hello response, which is absent from the SSL3 exchange and also from all the TLS exchanges that actually worked. Curiously, though, the same alert is present in a capture of Firefox performing the request successfully.
Finally, it appears that there was a temporary OCSP glitch just when I was first posting this question, which has since been resolved. This added to the mess, but isn't the core problem.