Supported redis version: redis >= 6.0
- redis >= 5.0 (use module
dict/zset
api; if use 4.*.*, cpdict/zset
op from redis do the same op; useRedisModule_SubscribeToKeyspaceEvents
api sub keyspace events,seeRedisModuleEventCallback
api help doc detail) - redis >= 6.0 (use module
RedisModule_NotifyKeyspaceEvent
and Server events definitionsRedisModuleEvent_**
api, sub CronLoop(serverCron) event to do resize/rehash dict),REDISMODULE_NOTIFY_LOADED
need load rdb init db slots keys. - redis >= 7.0 (use module
RedisModuleEvent_EVENTLOOP
api and use AE api with hiredis(1.2) adapter to connect)
Tips: use ruby gendoc.rb | less
or cat Module_Call | less
see api help doc or access modules-api-ref, hello** example and test/modules case.
- load module init hash slot size, default size 2^10, max size 2^16. (once make sure the slot size, don't change it)
- load module init activerehashing,databases from config, activerehashing used to sub server event to rehash slot keys dict
- load module init num_threads, if thread_num>0,init thread pool size to do migrate job, default donot use thread pool.
- sub ServerEvent
CronLoop(ServerLoop),FlushDB,Shutdown
- sub CronLoop server event hook to resize/rehash dict (db slot keys tables)
- sub FlushDB server event hook to delete one/all dict (db slot keys tables)
- sub Shutdown server event hook to release dicts (db slot keys tables) and free memory.
- sub KeyspaceEvents
STRING,LIST,HASH,SET,ZSET, LOADED; GENERIC, EXPIRED
- sub keyspaces
STRING,LIST,HASH,SET,ZSET, LOADED
notify event hook to add dict/skiplist keys - sub keyspaces
GENERIC, EXPIRED
notify event hook to delete dict/skiplist keys
- sub keyspaces
- support slot tag key migrate, for (smart client/proxy)'s configSrv admin contoller layer use it.
use
SLOTSMGRTTAGSLOT
cmd to migrate slot's key with same tag, default use slotsrestore batch send key, ttlms, dump rdb val ... (restore with replace) SLOTSRESTORE
if num_threads>0, init thread pool size to sendslotsrestore
batch keys job. loadmodule like this./redis/src/redis-server --port 6379 --loadmodule ./redisxslot.so 1024 4 --dbfilename dump.6379.rdb
- about migrate cmd, create a thread async block todo per client, splite batch migrate, don't or less block other cmd run. loadmodule like this
./redis/src/redis-server --port 6379 --loadmodule ./redisxslot.so 1024 4 async --dbfilename dump.6379.rdb
- support setcpuaffinity for migrate async thread like redis bio job thread config setcpuaffinity on linux/bsd(syntax of cpu list looks like taskset). loadmodule like this
./redis/src/redis-server --port 6379 --loadmodule ./redisxslot.so 1024 0 async 1,3 --dbfilename dump.6379.rdb
- about migrate cmd, support pipeline buffer migrate, use migrate cmd like this
SLOTSMGRTTAGSLOT 127.0.0.1 6379 30000 835 withpipeline
. usewithpipeline
current don't support thread pool and async block migrate.
git clone https://github.com/redis/redis.git
cd redis && make && cd ..
git clone https://github.com/weedge/redisxslot.git
# make help, default with hiredis static lib
cd redisxslot && make RM_INCLUDE_DIR=$(pwd)/../redis/src
../redis/src/redis-server --port 6379 --loadmodule ./redisxslot.so --dbfilename dump.6379.rdb
Tips:
- if want spec redis version(>=6.0.0) run
make RM_INCLUDE_DIR={redis_path}/src
,default use latest version(redismodule.h) - if use vscode debug, u can reference docs/launch.json
use conanfile py script todo ci with makefile release.
- use
redis/src/redis-benchmark
add test data, test 1m data mgrt, see doc/test.md. - need use redis tcl script
test_helper.tcl
to run test case;runbash -x tests/run_test.sh
, so easy~ - ci loadmodule use test case to Redis test suite and run all test case. if new feat, add some test case in tests/redisxslot.tcl
- run in local docker
# build redisxslot with latest redis stable version
make docker_img
# run redisxslot with latest redis stable version
make docker_run
# then run redis-cli in container with bridge docker network for mgrt between container
- if docker img is ok, u can push build's img to inner/cloud img hub or docker hub for ci/cd, like this:
# login
docker login
# tag a docker hub name
#docker image tag {taghash} weedge/redisxslot:latest_0.1.0
docker image tag redisxslot:latest_0.1.0 weedge/redisxslot:latest_0.1.0
# push your docker hub
docker push weedge/redisxslot:latest_0.1.0
# then pull this remote img to run, u can use it~ :)
docker run -itd --name weedge-redisxslot -p 16379:17000 weedge/redisxslot:latest_0.1.0
127.0.0.1:6660> setex 122{tag} 86400 v3
OK
127.0.0.1:6660> setex 1{tag} 86400 v3
OK
127.0.0.1:6660> sadd 12{tag} m1 m2 m3
(integer) 0
127.0.0.1:6660> hmset 123{tag} f1 v1 f2 v2
OK
127.0.0.1:6660> lpush 1234{tag} l1 l2
(integer) 2
127.0.0.1:6660> zadd 1235{tag} 100 z1 10 z2
(integer) 2
127.0.0.1:6660> slotshashkey 123{tag}
1) (integer) 899
127.0.0.1:6660> slotsinfo 899 1
1) 1) (integer) 899
1) (integer) 6
127.0.0.1:6660> SLOTSMGRTTAGSLOT 127.0.0.1 6666 3000 899
1) (integer) 6
2) (integer) 0
127.0.0.1:6660> SLOTSMGRTTAGSLOT 127.0.0.1 6666 3000 899
1) (integer) 0
2) (integer) 0
127.0.0.1:6666> slotsinfo 0 1024
1) 1) (integer) 899
1) (integer) 6
127.0.0.1:6666> get 122{tag}
"v3"
127.0.0.1:6666> ttl 122{tag}
(integer) 86133
127.0.0.1:6666> get 1{tag}
"v3"
127.0.0.1:6666> ttl 1{tag}
(integer) 86120
127.0.0.1:6666> hgetall 123{tag}
1) "f1"
2) "v1"
3) "f2"
4) "v2"
127.0.0.1:6666> lrange 1234{tag} 0 -1
1) "l2"
2) "l1"
127.0.0.1:6666> zrange 1235{tag} 0 10 withscores
1) "z2"
2) "10"
3) "z1"
4) "100"
Tips:
- if use codis-proxy, codis-dashboard config set
migration_method = "sync"
- if use redisxslot module to migrate with slot, please use cmd
slotsrestore
cmd instead ofrestore
to migrate