Skip to content

No durability guarantee (benchmark is misleading) #10

Closed
@damnever

Description

@damnever

I was wondering why nutsdb is so fast, so I did a quick review on your code, and there is no flush/sync call in the code, you should call it to ensure durability and update the benchmark. 💪

Also, the README should mention the byte endianness issue as well, and the isolation level is not clear.

Activity

xujiajun

xujiajun commented on Mar 7, 2019

@xujiajun
Member

@damnever hi ,thanks your advises,

as man2 said about mmap : The file may not actually be updated until msync(2) or munmap(2) is called. Actually i called munmap in this code, so data will be updated.

if you can give me your pr to improve it, it will be better;)

BTW, about the README i will continue to improve.

damnever

damnever commented on Mar 7, 2019

@damnever
Author

YES, but nutsdb declares to support ACID, so you should call it in each Commit, without it, the most important part about nutsdb is wrong.

damnever

damnever commented on Mar 7, 2019

@damnever
Author

Also, the simple lock cannot fully guarantee the ACID.

xujiajun

xujiajun commented on Mar 8, 2019

@xujiajun
Member

@damnever ,thanks :)
i have updated the README about transaction in the Caveats & Limitations,and i will rethink your advises , and i will continue to update the README, especially about the transaction.

damnever

damnever commented on Mar 8, 2019

@damnever
Author

Still, it is a misleading README, you should remove the benchmark part and all other contents those declares nutsdb is faster than others, if you remove their flush/fsync/fdatasync part, they are fast too! And anyone can write a fast kvstore.

As I mentioned before, you maybe understand the ACID properties, but you implement it wrong, and what is a non-standard transaction?

As a self-learning project is ok, but DO NOT mislead others!

xujiajun

xujiajun commented on Mar 8, 2019

@xujiajun
Member

@damnever thanks to review my README,i have updated the README about the benchmark part and about declares nutsdb is faster than others.

About a non-standard transaction or a standard transaction, i mean a transactional database like mysql that provides the ACID properties , i can call it a a standard transaction. If a database not completely including ACID properties,like redis ,i call it a non-standard transaction.

if have any question, Welcome to give me advises.

BTW, i never want to mislead others. if have any mistake about README or code etc. Welcome to submit issue and give me PR.

thanks again!

damnever

damnever commented on Mar 8, 2019

@damnever
Author

I glad to see you make some changes.

From the context:

Finally i did it and named NutsDB

xujiajun

xujiajun commented on Mar 11, 2019

@xujiajun
Member

@damnever hi, now the master branch of nutsdb has supports persistence , i have use sync function in my code to ensure durability and update the benchmark,

selected kvstore which is embedded, persistence and support transactions.

benchmark result:

badger 2019/03/11 18:06:05 INFO: All 0 tables opened in 0s
goos: darwin
goarch: amd64
pkg: github.com/xujiajun/kvstore-bench
BenchmarkBadgerDBPutValue64B-8    	   10000	    112382 ns/op	    2374 B/op	      74 allocs/op
BenchmarkBadgerDBPutValue128B-8   	   20000	     94110 ns/op	    2503 B/op	      74 allocs/op
BenchmarkBadgerDBPutValue256B-8   	   20000	     93480 ns/op	    2759 B/op	      74 allocs/op
BenchmarkBadgerDBPutValue512B-8   	   10000	    101407 ns/op	    3271 B/op	      74 allocs/op
BenchmarkBadgerDBGet-8            	 1000000	      1552 ns/op	     416 B/op	       9 allocs/op
BenchmarkBoltDBPutValue64B-8      	   10000	    203128 ns/op	   21231 B/op	      62 allocs/op
BenchmarkBoltDBPutValue128B-8     	    5000	    229568 ns/op	   13716 B/op	      64 allocs/op
BenchmarkBoltDBPutValue256B-8     	   10000	    196513 ns/op	   17974 B/op	      64 allocs/op
BenchmarkBoltDBPutValue512B-8     	   10000	    199805 ns/op	   17064 B/op	      64 allocs/op
BenchmarkBoltDBGet-8              	 1000000	      1122 ns/op	     592 B/op	      10 allocs/op
BenchmarkNutsDBPutValue64B-8      	   30000	     53614 ns/op	     626 B/op	      14 allocs/op
BenchmarkNutsDBPutValue128B-8     	   30000	     51998 ns/op	     664 B/op	      13 allocs/op
BenchmarkNutsDBPutValue256B-8     	   30000	     53958 ns/op	     920 B/op	      13 allocs/op
BenchmarkNutsDBPutValue512B-8     	   30000	     55787 ns/op	    1432 B/op	      13 allocs/op
BenchmarkNutsDBGet-8              	 2000000	       661 ns/op	      88 B/op	       3 allocs/op
BenchmarkNutsDBGetByHintKey-8     	   50000	     27255 ns/op	     840 B/op	      16 allocs/op
PASS
ok  	github.com/xujiajun/kvstore-bench	83.856s

visit https://github.com/xujiajun/nutsdb#benchmarks for detail.

damnever

damnever commented on Mar 11, 2019

@damnever
Author

I didn't go into details, as your benchmark shows, nutsdb is the simplest and fastest, it suites for the small projects(with small values?).

Here is a bit of advice, do some setup and clean up work for each benchmark case.

xujiajun

xujiajun commented on Mar 11, 2019

@xujiajun
Member

@damnever thanks your advice,but i don't know what you mean about doing some setup and cleaning up work for each benchmark case. I have use b.ResetTimer() to reset timer for each benchmark case and remove data before the case begining.

olekukonko

olekukonko commented on Mar 11, 2019

@olekukonko

@damnever I think you should be less ambiguous and clearly state the issues.

xujiajun

xujiajun commented on Mar 12, 2019

@xujiajun
Member

@damnever hi, if have no more advises about this issue, i will close it.

1 remaining item

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      No durability guarantee (benchmark is misleading) · Issue #10 · nutsdb/nutsdb