Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Utils.build_nested_query to URL-encode all query string fields #1989

Merged
merged 2 commits into from
Dec 3, 2022

Conversation

jdufresne
Copy link
Contributor

In the formatted query string, the field should be fully URL-encoded. The previous implementation left the square brackets as un-escaped.

It was noticed that if params were provided as a nested hash it produced a different result than a hash of string fields. For example, the params

{ 'q[s]' => 'name' }

would result in the query string:

q%5Bs%5D=name

but

{ q: { s: 'name' } }

would result in:

q[s]=name

Both forms now produce the same correct result with fully URL-encoded params.

In the formatted query string, the field should be fully URL-encoded.
The previous implementation left the square brackets as un-escaped.

It was noticed that if params were provided as a nested hash it produced
a different result than a hash of string fields. For example, the params

    { 'q[s]' => 'name' }

would result in the query string:

    q%5Bs%5D=name

but

    { q: { s: 'name' } }

would result in:

    q[s]=name

Both forms now produce the same correct result with fully URL-encoded
params.
@ioquatix
Copy link
Member

ioquatix commented Dec 3, 2022

I'm okay with this but it seems like a potential breaking change.

Copy link
Contributor

@jeremyevans jeremyevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this could potentially be breaking, but RFC 3986 is clear that [ and ] are reserved characters and should be percent encoded. So I think we should merge this.

@ioquatix ioquatix merged commit 10864b6 into rack:main Dec 3, 2022
@jdufresne jdufresne deleted the urlencode-brackets branch December 3, 2022 13:44
ioquatix pushed a commit that referenced this pull request Dec 5, 2022
…1989)

In the formatted query string, the field should be fully URL-encoded.
The previous implementation left the square brackets as un-escaped.

It was noticed that if params were provided as a nested hash it produced
a different result than a hash of string fields. For example, the params

    { 'q[s]' => 'name' }

would result in the query string:

    q%5Bs%5D=name

but

    { q: { s: 'name' } }

would result in:

    q[s]=name

Both forms now produce the same correct result with fully URL-encoded
params.
@AlexWayfer
Copy link
Contributor

Yeah, sadly it has broken tests of my Rack-based framework unclear and non-expected, just by some patch-version release.

AlexWayfer added a commit to AlexWayfer/flame that referenced this pull request Dec 22, 2022
@ioquatix
Copy link
Member

IIUC, one way to avoid this problem in your test is to avoid asserting the encoded representation and actually parse the URL and extract the query parameters. This should work regardless of the encoding and is probably a more robust test in general.

tenderlove added a commit that referenced this pull request Jan 17, 2023
* 3-0-sec: (24 commits)
  bump version
  Update changelog
  Fix ReDoS vulnerability in multipart parser
  Fix ReDoS in Rack::Utils.get_byte_ranges
  Forbid control characters in attributes
  Bump patch version.
  `Rack::Request#POST` should consistently raise errors. (#2010)
  Fix Rack::Lint error message for HTTP_CONTENT_TYPE and HTTP_CONTENT_LENGTH (#2007)
  Rack::MethodOverride handle QueryParser::ParamsTooDeepError (#2006)
  Bump patch version.
  Fix Regexp deprecated third argument with Regexp::NOENCODING (#1998)
  Update tests to work on latest Rubies. (#1999)
  Bump patch version.
  Allow passing through streaming bodies. (#1993)
  Remove unnecessary executable bit from test files (#1992)
  Fix Utils.build_nested_query to URL-encode all query string fields (#1989)
  Trim trailing white space throughout the project (#1990)
  Fix some typos (#1991)
  Remove leading dot to fix compatibility with latest cgi gem. (#1988)
  Fix outdated Rack::Builder rdocs and remove Lobster references (#1986)
  ...
dentarg added a commit to dentarg/sinatra that referenced this pull request Feb 9, 2023
@dentarg dentarg mentioned this pull request Feb 9, 2023
7 tasks
dentarg added a commit to sinatra/sinatra that referenced this pull request Feb 13, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Feb 24, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Mar 4, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Mar 5, 2023
@AlexWayfer
Copy link
Contributor

AlexWayfer commented Apr 10, 2023

IIUC, one way to avoid this problem in your test is to avoid asserting the encoded representation and actually parse the URL and extract the query parameters. This should work regardless of the encoding and is probably a more robust test in general.

I can agree. I've checked the tests again. It's simple tests for path method, returning String, sometimes with query. So, parsing it with standard tools may be a pain.

Maybe I should add URI.decode_uri_component, but, again, not sure how safe it is. And even this will not be simple, because I don't want to decode the testing result, but encoding the expected one will also break all chars like /, ?, &.

@ioquatix
Copy link
Member

Maybe I should add URI.decode_uri_component

I think this is probably the correct approach.

dentarg added a commit to dentarg/sinatra that referenced this pull request Apr 11, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request May 15, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request May 16, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Aug 7, 2023
geoffharcourt pushed a commit to commonlit/sinatra that referenced this pull request Nov 6, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Dec 23, 2023
dentarg added a commit to dentarg/sinatra that referenced this pull request Jan 2, 2024
dentarg added a commit to dentarg/sinatra that referenced this pull request Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants