Python thread error when using a custom comparator #35
Closed
Description
I've just encounter this problem when inserting a stream of data using a custom comparator. Code is something like below:
def str_int_comparator(a, b):
a = int(a)
b = int(b)
if a < b:
return -1
if a > b:
return 1
return 0
db = plyvel.DB(db_path, create_if_missing=True,
comparator=str_int_comparator,
comparator_name='str_int_comparator')
stream = get_the_stream(...)
before = time.time()
wb = db.write_batch()
# generate a kv stream
for i, line in enumerate(stream):
if i > 0 and i % 50000 == 0:
wb.write()
wb = db.write_batch()
print "Inserted upto {} ...".format(i)
wb.put(str(i), json.dumps(line))
after = time.time()
used = after - beforeI got this after 100k elements are inserted:
Inserted upto 50000 ...
Inserted upto 100000 ...
Fatal Python error: This thread state must be current when releasing
Aborted
If I use the default comparator, everything goes well.
I have no idea the implementation but it seems to me that when LevelDB begins to compact files, the error occurs.
I'm using plyvel 0.8, LevelDB 1.15.0 and python 2.7.8.
Metadata
Assignees
Labels
No labels