Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix global index detection #216

Merged
merged 1 commit into from
Jan 6, 2018
Merged

Fix global index detection #216

merged 1 commit into from
Jan 6, 2018

Conversation

andrykonchin
Copy link
Member

@andrykonchin andrykonchin commented Dec 31, 2017

Fix issue with incorrect using of global secondary index.

class Message
  include Dynamoid::Document

  table key: :guid
  field :message_thread_guid

  global_secondary_index hash_key: :message_thread_guid, projected_attributes: :all
end

Querying with condition other than "equal" for hash-key of some index

Message.where('message_thread_guid.in' => ['1', '2']).to_a

leads to "Aws::DynamoDB::Errors::ValidationException: One or more parameter values were invalid: Condition parameter type does not match schema type" error

@coveralls
Copy link

coveralls commented Dec 31, 2017

Coverage Status

Coverage increased (+0.004%) to 97.325% when pulling 162e20d on andrykonchin:fix-index-detection-in-where into 125951a on Dynamoid:master.

@coveralls
Copy link

coveralls commented Dec 31, 2017

Coverage Status

Coverage increased (+0.004%) to 97.325% when pulling 5b45d6e on andrykonchin:fix-index-detection-in-where into 125951a on Dynamoid:master.

@@ -294,7 +294,7 @@ def key_present?
# But only do so if projects ALL attributes otherwise we won't
# get back full data
source.global_secondary_indexes.each do |_, gsi|
next unless query_keys.include?(gsi.hash_key.to_s) && gsi.projected_attributes == :all
next unless query.keys.map(&:to_s).include?(gsi.hash_key.to_s) && gsi.projected_attributes == :all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 276 above we have this:

if query_keys.include?(source.range_key.to_s)

Seems like we may need a more complete solution than trying to remember to do this to_s conversion everywhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pboling I considered wrapping query hash in OOP stile to incapsulate some logic and hide type conversion but still don't have simple and flexible solution

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bug is too narrow still. The problem is that we are comparing hash keys with strings, and the hash keys may not themselves be strings. Fixing this one instance but leaving the bug in other places where we do comparisons seems like a bad idea to me, as we'll just lose track of the actual problem. I think we should go with a more encompassing solution. Doesn't need to be OOP necessarily, but, why not have new method like query_keys that gives them as string, e.g. query_keys_as_string, and then call that from everywhere we do key comparisons.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pboling Agree

The simplest solution (and it's already used here in this project) is to convert all user strings to the same type, e.g. symbols. And to have proper tests :).

I will check how we handle and store all user strings like field names, key names, conditions etc to ensue we convert them to symbols.

Actually this bug isn't related to symbol/string comparison. We need to check hash_key: 'value' condition but query_keys contains filtered field names without operators so 'hash_key.contains': 'value' triggers involving index too.

@andrykonchin
Copy link
Member Author

andrykonchin commented Jan 6, 2018

@pboling @richardhsu If nobody doesn't mind I will merge this fix

@pboling
Copy link
Member

pboling commented Jan 6, 2018

I'm ok with it.

@pboling pboling merged commit fb9db64 into Dynamoid:master Jan 6, 2018
@andrykonchin andrykonchin deleted the fix-index-detection-in-where branch January 7, 2018 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants