Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byte storage integration into segment #4049

Merged
merged 16 commits into from
Apr 17, 2024
+1,039 −217
Merged

Byte storage integration into segment #4049

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
40b4309
byte storage with quantization
IvanPleshkov Apr 16, 2024
008aa6e
are you happy clippy
IvanPleshkov Apr 17, 2024
c283a55
don't use distance in quantized scorers
IvanPleshkov Apr 17, 2024
dd24222
fix build
IvanPleshkov Apr 17, 2024
eb4f296
add fn quantization_preprocess
IvanPleshkov Apr 17, 2024
1f39aa9
apply preprocessing for only cosine float metric
IvanPleshkov Apr 17, 2024
abb084d
fix sparse vectors tests
IvanPleshkov Apr 17, 2024
3263e8b
update openapi
IvanPleshkov Apr 17, 2024
28cdc53
more complicated integration test
IvanPleshkov Apr 17, 2024
ab480ad
update openapi comment
IvanPleshkov Apr 17, 2024
2179c7b
mmap byte storages support
IvanPleshkov Apr 17, 2024
686cf90
fix async test
IvanPleshkov Apr 17, 2024
8db8f2c
move .unwrap closer to the actual check of the vector presence
generall Apr 17, 2024
c22d7f8
fmt
generall Apr 17, 2024
74177af
remove distance similarity function
IvanPleshkov Apr 17, 2024
0edd630
avoid copying data while working with cow
IvanPleshkov Apr 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix async test
  • Loading branch information
@IvanPleshkov
IvanPleshkov committed Apr 17, 2024
commit 686cf90e4d349c04ccdcee69da81dfebc0dd0a1a
4 changes: 2 additions & 2 deletions lib/segment/src/vector_storage/dense/memmap_dense_vector_storage.rs
View file
Original file line number Diff line number Diff line change
@@ -74,14 +74,14 @@ pub fn open_memmap_vector_storage_with_async_io(
distance: Distance,
with_async_io: bool,
) -> OperationResult<Arc<AtomicRefCell<VectorStorageEnum>>> {
let storage = open_memmap_vector_storage_with_async_io_impl::<VectorElementTypeByte>(
let storage = open_memmap_vector_storage_with_async_io_impl::<VectorElementType>(
path,
dim,
distance,
with_async_io,
)?;
Ok(Arc::new(AtomicRefCell::new(
VectorStorageEnum::DenseMemmapByte(storage),
VectorStorageEnum::DenseMemmap(storage),
)))
}