Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #1182 #1188

Closed
wants to merge 3 commits into from
Closed

Fix #1182 #1188

wants to merge 3 commits into from

Conversation

lesleyjanenorton
Copy link
Collaborator

  • Lower cases new emails before adding to DB.
  • Updates bundleVerifiedEmails to convert existing user emails to lowercase letters before making a new sha1 hash (of the newly lowercased email) to use for scanning breaches.

Screen Shot 2019-09-04 at 5 58 25 PM

Screen Shot 2019-09-04 at 5 58 31 PM

@@ -136,7 +136,8 @@ async function add(req, res) {
}

async function bundleVerifiedEmails(email, emailSha1, ifPrimary, id, verificationStatus, allBreaches) {
const foundBreaches = await HIBP.getBreachesForEmail(emailSha1, allBreaches, true);
const lowerCaseEmailSha = sha1(email.toLowerCase());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is emailSha1 now unused?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed it is!

@pdehaan
Copy link
Contributor

pdehaan commented Sep 4, 2019

  • Lower cases new emails before adding to DB.

Do we need to update the existing records in the DB and lowercase some emails/SHAs?

@lesleyjanenorton
Copy link
Collaborator Author

@pdehaan re: your last comment, I would think that updating existing records would be ideal right? @groovecoder what would that look like/is it feasible?

Also curious: I'm no longer able to add variations of lesleyjanenorton@gmail.com without triggering the duplicate email error.

Copy link
Member

@groovecoder groovecoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dang, this actually needs a pretty extensive fix ...

  1. Lower-case email addresses before they go into subscribers.primary_email too (call email.toLowerCase() in the oauth confirmed controller before we call addSubscriber).
  2. Update all subscribers.primary_email and subscribers.primary_sha1 values to their corresponding lower-case values.
  3. Update all email_addresses.email and email_addresses.sha1 values to their corresponding lower-case values.
  4. Remove all existing non-lowercase email address sha1 value subscriptions from HIBP
  5. Re-subscribe lower-case versions of all the non-lowercase email address sha1 value subscriptions that we remove in step 3.

Step 1 can be done in this PR.

Steps 2-4 can either be done in a big batch job (probably best) or can be done ad-hoc as users sign in.

for (const email of monitoredEmails) {
if (email.verified) {
const formattedEmail = await bundleVerifiedEmails(email.email, email.sha1, false, email.id, email.verified, allBreaches);
const formattedEmail = await bundleVerifiedEmails(email.email, false, email.id, email.verified, allBreaches);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking observation ...

This function call signature reminds me I've been wanting to go thru our code-base and use JS options objects in most of our functions.

@groovecoder groovecoder deleted the 1182-hot-fix branch September 13, 2019 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants