Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

erts: Refactor internal term hashing #7249

Merged
merged 1 commit into from
May 27, 2023

Conversation

jhogberg
Copy link
Contributor

Improve performance and hash quality by using MurmurHash3 instead of our old Jenkins96 derivative, and reduce the chances of needing collision nodes in maps by making the hash width follow the word size, allowing an extra 8 levels of collision before that happens on 64-bit platforms.

@jhogberg jhogberg added team:VM Assigned to OTP team VM enhancement labels May 16, 2023
@jhogberg jhogberg added this to the OTP-27.0 milestone May 16, 2023
@jhogberg jhogberg requested a review from sverker May 16, 2023 16:55
@jhogberg jhogberg self-assigned this May 16, 2023
@github-actions
Copy link
Contributor

github-actions bot commented May 16, 2023

CT Test Results

       3 files     133 suites   57m 54s ⏱️
1 552 tests 1 500 ✔️ 51 💤 1
1 970 runs  1 899 ✔️ 70 💤 1

For more details on these failures, see this check.

Results for commit ef0d257.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

// Erlang/OTP Github Action Bot

@jhogberg jhogberg force-pushed the john/erts/tweak-internal-hash branch 2 times, most recently from 040cd7c to 91ad415 Compare May 16, 2023 17:34
@max-au
Copy link
Contributor

max-au commented May 16, 2023

Cool, now that MurmurHash3 is a part of ERTS, would that be possible to actually expose it to work over binaries? E.g. add that support to crypto:hash(murmur3, Data). It would've saved me from having a NIF.

@jhogberg
Copy link
Contributor Author

Sort of, I'm planning a erlang:term_hash(Term, Options) function that lets you choose algorithm, portability (with version), and so on.

@jhogberg jhogberg force-pushed the john/erts/tweak-internal-hash branch 2 times, most recently from 6357508 to 293d849 Compare May 16, 2023 17:52
@max-au
Copy link
Contributor

max-au commented May 16, 2023

Just need to ensure there is no confusion with erlang:phash2 and crypto:hash...

@jhogberg jhogberg force-pushed the john/erts/tweak-internal-hash branch from 293d849 to 5776496 Compare May 17, 2023 08:15
@jhogberg jhogberg added the testing currently being tested, tag is used by OTP internal CI label May 17, 2023
@jhogberg jhogberg force-pushed the john/erts/tweak-internal-hash branch 2 times, most recently from aec1ab7 to 9ea7ea6 Compare May 22, 2023 08:43
Improve performance and hash quality by using MurmurHash3 instead
of our hand-rolled Jenkins96 derivative, and reduce the chances of
needing collision nodes in maps by making the hash width follow the
word size, allowing an extra 8 levels of collision before that
happens on 64-bit platforms.
@jhogberg jhogberg force-pushed the john/erts/tweak-internal-hash branch from 9ea7ea6 to ef0d257 Compare May 23, 2023 07:42
@jhogberg jhogberg merged commit 796c1f0 into erlang:master May 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement team:VM Assigned to OTP team VM testing currently being tested, tag is used by OTP internal CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants