Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add key to the responses #24

Open
chippyash opened this issue Jan 13, 2025 · 11 comments
Open

Add key to the responses #24

chippyash opened this issue Jan 13, 2025 · 11 comments

Comments

@chippyash
Copy link

I'm having difficulty in using the DoMulti methods. This is not because the command doesn't work, but because Go does not honor order in maps etc.

I have a workaround which is to use ordered maps, but it's a lot of weight to add.

It appears that cacheResponses := cl.DoMulti(context.TODO(), cmds...) will return the responses in the order they were sent, but matching them to the original request is difficult because of Go vagaries about map and slice order, hence the need for ordered maps

If I may suggest, add the sent key to the responses, so that we can match the two.

Kind regards
Ashley

@chippyash
Copy link
Author

FYI. I'm trying to implement a reasonably agnostic cache manager at https://github.com/chippyash/go-cache-manager

@rueian
Copy link
Collaborator

rueian commented Jan 13, 2025

Hi @chippyash,

Are you looking for these?

func MGet(client Client, ctx context.Context, keys []string) (ret map[string]ValkeyMessage, err error) {

func MGetCache(client Client, ctx context.Context, ttl time.Duration, keys []string) (ret map[string]ValkeyMessage, err error) {

@rueian
Copy link
Collaborator

rueian commented Jan 13, 2025

I guess you would probably interested in https://github.com/valkey-io/valkey-go/tree/main/valkeyaside.

@chippyash
Copy link
Author

@rueian I'll take a look and get back to you

I just read your docs and saw that I can do a DoMulti, so considered this a suitable construct. The problem is with Go handling of slices, so I hand the function a slice of key names, but the responses are, as far as I can tell, not necessarily coming back in the order I expect them. So if the response can contain the 'sent' key name, that would allow a quick match.

@chippyash
Copy link
Author

Simply put, Change
type ValkeyResult struct {
err error
val ValkeyMessage
}

to

type ValkeyResult struct {
key string
err error
val ValkeyMessage
}

With the function to retrieve the key.

@rueian
Copy link
Collaborator

rueian commented Jan 14, 2025

The MGet helper I mentioned previously can help you retrieve the corresponding keys.

@chippyash
Copy link
Author

But it isn't clear in your readme docs. I'll take a look, but it should not be hidden away

Please consider the change, hopefully it's really simple.

Kind regards
Ashley

@rueian
Copy link
Collaborator

rueian commented Jan 14, 2025

But it isn't clear in your readme docs. I'll take a look, but it should not be hidden away

The MGet and MGetCache helpers indeed lack some examples on the README. PR for that is welcome.

Please consider the change, hopefully it's really simple.

Do you mean adding a key string field to the ValkeyResult struct? Actually, that is not the right move. Note that many commands don't have a key associated and many commands have multiple keys associated.

Adding a field in the response struct to echo the command is also unnecessary because, besides the MGet helper, you can use the index of the response slice to get the corresponding command. Just remember to use .Pin() to keep commands from being recycled. For example:

	cmds := make(valkey.Commands, 0, 10)
	for i := 0; i < 10; i++ {
		cmds = append(cmds, client.B().Get().Key(strconv.Itoa(i)).Build().Pin())
	}
	for i, resp := range client.DoMulti(context.Background(), cmds...) {
		fmt.Println(resp.ToString()) // this is the result
		fmt.Println(cmds[i].Commands()[1]) // this is the corresponding key
	}

@chippyash
Copy link
Author

chippyash commented Jan 14, 2025

@rueian Thanks for that. That is what I am after. How do I recycle the command manually after it has been pinned?

Also for other readers, if you are doing client side caching, the incantation is:

cmds := make([]valkey.CacheableTTL, 0, len(keys))
ttl := time.Second * 60
for _, key := range keys {
	cmds = append(cmds, valkey.CT(cl.B().Get().Key(key).Cache().Pin(), ttl))
}
for i, resp := range cl.DoMultiCache(context.TODO(), cmds...) {
       cmdKey := cmds[i].Cmd.Commands()[1]
       val := resp.ToString()
       //or val := resp.ToAny()
}

@rueian
Copy link
Collaborator

rueian commented Jan 14, 2025

How do I recycle the command manually after it has been pinned?

Recycling a command is tricky. You shouldn’t recycle it if Do() or DoMulti() returns early due to a context deadline or other errors. Otherwise, the command could still be accessed later by the client, leading to data races and incorrect behavior.

In summary, you can only recycle a command after receiving its response from Valkey. However, handling this condition is too complex for users, so we don’t have an API for it.

@chippyash
Copy link
Author

@rueian Thank you for your very timely help, which has been invaluable. I have now released a pre-production version of the library at https://github.com/chippyash/go-cache-manager. It will go through some battle testing in 3 other applications. I think you can consider this topic closed. Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants