simple get request returns truncated body for some url's #224
Description
If you try to fetch the following URL using typhoeus it returns a body that's only about half the size it should be.
url = "http://www.walgreens.com/store/c/always-dri-liners-long-pantiliners-unscented/ID=prod1026420-product"
r = Typhoeus::Request.get(url)
puts "-->code=#{r.code}, size=#{r.body.length} B, success? #{r.success?}"
-->code=200, size=263619 B, success? true
Whereas using net/http returns a body length for this URL that is 554524 chars, as does open-uri. If you enter that url into a browser and save the source you get something on the order of 554k.
It seems to truncate the html in the response.body right after this comment:
It never makes it to the end of the ... section.
I also tried using curl and it worked fine - saved the entire page. Somewhere between curl's and typhoeus' outputs there is a disconnect.
Has anyone seen anything like this happening?