Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark improvements #127

Open
dumblob opened this issue Oct 1, 2023 · 1 comment
Open

Benchmark improvements #127

dumblob opened this issue Oct 1, 2023 · 1 comment

Comments

@dumblob
Copy link

dumblob commented Oct 1, 2023

Hi, just came across lotusdb and I like it (need to find out how to deep-store maps and other complex structures without resorting to plain serialization).

I noticed util.RandomValue(1024) is being called in the hot paths of the pico benchmarks.

Note this might cause a significant delay and flaw the whole benchmark. Perhaps use a huge precomputed table (1Gigabyte or more) of random numbers (precomputed after the start of the benchmark but before the time starts to be measured).
@kauruus
Copy link

kauruus commented Oct 15, 2023

also find this issue, here's my attempt to compare various random value generations methods.

implementations:

  • RandomValue: base line in main branch
  • RandomLockLoop: lock the entire loop once in RandomValue
  • RandomGlobal: use the global rand.Intn
  • RandomCrypto: use crypto/rand.Read to read random bytes directly, the generated value is not human readable.

no concurrency (GOMAXPROCS=1):

BenchmarkRandomValue/input_size_10         	10276536	       116.1 ns/op
BenchmarkRandomValue/input_size_100        	 1607594	       750.1 ns/op
BenchmarkRandomValue/input_size_1000       	  174063	      6772 ns/op
BenchmarkRandomCrypto/input_size_10        	 4167972	       286.5 ns/op
BenchmarkRandomCrypto/input_size_100       	 2154044	       561.8 ns/op
BenchmarkRandomCrypto/input_size_1000      	  478246	      2358 ns/op
BenchmarkRandomLockLoop/input_size_10      	11586986	       103.8 ns/op
BenchmarkRandomLockLoop/input_size_100     	 1930639	       621.1 ns/op
BenchmarkRandomLockLoop/input_size_1000    	  209892	      5527 ns/op
BenchmarkRandomGlobal/input_size_10        	 9024844	       131.4 ns/op
BenchmarkRandomGlobal/input_size_100       	 1332739	       897.2 ns/op
BenchmarkRandomGlobal/input_size_1000      	  138804	      8401 ns/op
BenchmarkRandomValueNoLock/input_size_10   	11404400	       103.8 ns/op
BenchmarkRandomValueNoLock/input_size_100  	 1932565	       620.5 ns/op
BenchmarkRandomValueNoLock/input_size_1000 	  208808	      5537 ns/op

using RandomValue:

BenchmarkPut-16    	  118449	     10219 ns/op	   11438 B/op	      42 allocs/op

using crypto/rand.Read:

BenchmarkPut-16    	  225318	      5844 ns/op	   14501 B/op	      40 allocs/op

using a fixed string:

BenchmarkPut-16    	  467154	      3424 ns/op	   12175 B/op	      35 allocs/op

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dumblob @kauruus