Skip to content

Distributed Qdrant cluster with Replication and Sharding #291

Open
@pschwager

Description

Hi,

I'm experimenting with the replication feature of qdrant. If I understood the documentation correct, I have to set this in the qdrant helm chart:

  • replicaCount: 3
    to have 3 replicas in the sts deployed

additionally set replication factor to 3 to have the data duplicated to all of this 3 replicas:

config:
  storage:
    collection:
      replication_factor: 3

which then creates a configmap with the following contents:

  production.yaml: |                                                                                                                                                                                               
    cluster:                                                                                                                                                                                                       
      consensus:                                                                                                                                                                                                   
        tick_period_ms: 100                                                                                                                                                                                        
      enabled: true                                                                                                                                                                                                
      p2p:                                                                                                                                                                                                         
        port: 6335                                                                                                                                                                                                 
    storage:                                                                                                                                                                                                       
      collection:                                                                                                                                                                                                  
        replication_factor: 3                                                                                                                                                                                      

So also cluster.enabled is set to true

What else do I miss here? If I kill one of the 3 replicas, e.g. by setting a to low memory limit (oom crash loop), then the cluster still answers to
GET collections calls, but GET collections/<collection> doesn't work anymore

If I call GET collections/<collection>/cluster while the cluster is up, I see that the collection has replication factor 3 set:

{
  "result": {
    "status": "grey",
    "optimizer_status": "ok",
    "segments_count": 8,
    "config": {
        "shard_number": 3,
        "replication_factor": 3,
        "write_consistency_factor": 1,
        "on_disk_payload": true
      },

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions