Skip to content

simple get request returns truncated body for some url's #224

Closed
@jkcunningham

Description

If you try to fetch the following URL using typhoeus it returns a body that's only about half the size it should be.

url = "http://www.walgreens.com/store/c/always-dri-liners-long-pantiliners-unscented/ID=prod1026420-product"
r = Typhoeus::Request.get(url)
puts "-->code=#{r.code}, size=#{r.body.length} B, success? #{r.success?}"

-->code=200, size=263619 B, success? true

Whereas using net/http returns a body length for this URL that is 554524 chars, as does open-uri. If you enter that url into a browser and save the source you get something on the order of 554k.

It seems to truncate the html in the response.body right after this comment:

It never makes it to the end of the ... section.

I also tried using curl and it worked fine - saved the entire page. Somewhere between curl's and typhoeus' outputs there is a disconnect.

Has anyone seen anything like this happening?

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions