Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

log_redis_info error #254

Closed
smlbiobot opened this issue Jul 5, 2021 · 2 comments · Fixed by #255 or sthagen/samuelcolvin-arq#7
Closed

log_redis_info error #254

smlbiobot opened this issue Jul 5, 2021 · 2 comments · Fixed by #255 or sthagen/samuelcolvin-arq#7

Comments

@smlbiobot
Copy link

I am using a Redis Labs instance where for unknown reasons, the server infos are not being returned, and so the following throws a ValueError — I have yet to figure out what the issue is, and had to change this part of connections to stop the issue:

connections.py

async def log_redis_info(redis: Redis, log_func: Callable[[str], Any]) -> None:
    with await redis as r:
        info, key_count = await asyncio.gather(r.info(), r.dbsize())
    log_func(
        f'redis_version={info["server"]["redis_version"]} '
        f'mem_usage={info["memory"]["used_memory_human"]} '
        f'clients_connected={info["clients"]["connected_clients"]} '
        f'db_keys={key_count}'
    )
async def log_redis_info(redis: Redis, log_func: Callable[[str], Any]) -> None:
    try:
        with await redis as r:
            info, key_count = await asyncio.gather(r.info(), r.dbsize())
    except ValueError as e:
        log_func("REDIS INFO VALUE ERROR - Skipping…")
    else:

        log_func(
            f'redis_version={info["server"]["redis_version"]} '
            f'mem_usage={info["memory"]["used_memory_human"]} '
            f'clients_connected={info["clients"]["connected_clients"]} '
            f'db_keys={key_count}'
        )

It’s clearly an issue, possibly with Redis (or the Redis Enterprise Cloud - whom I am contacting also). However, since this is a non-crucial part of the library, it would be great if this unexpected behavior can be contained in a try-loop as above.
@smlbiobot
Copy link
Author

smlbiobot commented Jul 5, 2021 •
edited
Loading

I have found the issue. When using aioredis in combination with the Redis Enterprise Cloud implementations at Redis Labs with redis_version:6.0.9, running this will throw a ValueError

info = await redis.info(section="Server")

In essence, the code in arq assumes that info() will return every section but now it returns nothing, and so it gives this error:

  File "/path/to/venv/versions/cr-api-web-38/lib/python3.8/site-packages/aioredis/commands/server.py", line 273, in parse_info
    section, *block = block.strip().splitlines()
ValueError: not enough values to unpack (expected at least 1, got 0)

I can’t say for sure if this is an issue with aioredis or arq or the specific Redis version, but as stated in my first message, if we can simply wrap this in a try-block it would be great.

Alternatively, this will fetch the values and make sure things won’t fail if the the returned values aren’t as expected.

info_server, info_memory, info_clients, key_count = await asyncio.gather(
    r.info(section="Server"),
    r.info(section="Memory"),
    r.info(section="Clients"),
    r.dbsize(),
)

redis_version = info_server.get('server', {}).get('redis_version')
mem_usage = info_memory.get('memory', {}).get('used_memory_human')
clients_connected = info_clients.get('clients', {}).get('connected_clients')

log_func(
    f'redis_version={redis_version} '
    f'mem_usage={mem_usage} '
    f'clients_connected={clients_connected} '
    f'db_keys={key_count}'
)

@samuelcolvin
Copy link
Member

Hi @smlbiobot thanks for reporting this.

I think the second solution would be great, I'll try and create a PR today.

samuelcolvin added a commit that referenced this issue Jul 6, 2021
@samuelcolvin
log_redis_info, fix #254
63b13e1
@samuelcolvin samuelcolvin mentioned this issue Jul 6, 2021
Merged
samuelcolvin added a commit that referenced this issue Jul 6, 2021
@samuelcolvin
log_redis_info (#255)
118308a 
* log_redis_info, fix #254

* uprev

* linting and fix CI
@sthagen sthagen mentioned this issue Jul 9, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

log_redis_info python-arq/arq
log_redis_info (#255) python-arq/arq
2 participants
@samuelcolvin @smlbiobot