Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: [Nightly] Insert/ delete failed reporting time tick long delay #38589

Closed
1 task done
NicoYuan1986 opened this issue Dec 19, 2024 · 2 comments
Closed
1 task done

[Bug]: [Nightly] Insert/ delete failed reporting time tick long delay #38589

NicoYuan1986 opened this issue Dec 19, 2024 · 2 comments
Assignees
@yanliang567 @NicoYuan1986
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone
2.4.18

Comments

@NicoYuan1986
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 98052cf
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Insert/ delete failed reporting time tick long delay.

[2024-12-18 17:58:09 - DEBUG - ci_test]: (api_request)  : [Collection.delete] args: ['NewStr != "0"', None, 180], kwargs: {} (api_request.py:56)
[2024-12-18 17:58:09 - ERROR - pymilvus.decorators]: RPC error: [delete], <MilvusException: (code=9, message=quota exceeded[reason=time tick long delay])>, <Time:{'RPC start': '2024-12-18 17:58:09.205091', 'RPC error': '2024-12-18 17:58:09.207851'}> (decorators.py:145)
[2024-12-18 17:58:09 - ERROR - ci_test]: Traceback (most recent call last):
  File "/src/milvus-cloud-test/python_client/utils/api_request.py", line 26, in inner_wrapper
    res = func(*args, **_kwargs)
  File "/src/milvus-cloud-test/python_client/utils/api_request.py", line 57, in api_request
    return func(*arg, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/orm/collection.py", line 580, in delete
    res = conn.delete(self._name, expr, partition_name, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 146, in handler
    raise e from e
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 142, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 181, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 121, in handler
    raise e from e
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 650, in delete
    raise err from err
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 644, in delete
    check_status(response.status)
  File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/utils.py", line 63, in check_status
    raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=9, message=quota exceeded[reason=time tick long delay])>
 (api_request.py:39)
[2024-12-18 17:58:09 - ERROR - ci_test]: (api_response) : <MilvusException: (code=9, message=quota exceeded[reason=time tick long delay])> (api_request.py:40)

Expected Behavior

pass

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response
@NicoYuan1986 NicoYuan1986 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 19, 2024
@NicoYuan1986 NicoYuan1986 added this to the 2.4.18 milestone Dec 19, 2024
@NicoYuan1986 NicoYuan1986 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 19, 2024
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Dec 19, 2024
@XuanYang-cn
fix: State trans error in concurrent Release and Watching
8db49a5 
See also: milvus-io#38589

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Dec 19, 2024
@XuanYang-cn
fix: ChannelManager concurret Release and Watch bug
327d2c6 
See also: milvus-io#38589

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
This was referenced Dec 19, 2024
Merged
Merged
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Dec 19, 2024
@XuanYang-cn
fix: ChannelManager concurret Release and Watch bug
fe7bfa8 
See also: milvus-io#38589

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue Dec 19, 2024
@XuanYang-cn
fix: State trans error in concurrent Release and Watching (#38591)
e82af48 
See also: #38589
pr: #38590

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue Dec 19, 2024
@XuanYang-cn
fix: ChannelManager concurret Release and Watch bug (#38590)
c0b855d 
See also: #38589

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn
Copy link
Contributor

/assign @NicoYuan1986
/unassign
Please help verify

@yanliang567
Copy link
Contributor

not reproduce on milvus 2.4.18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yanliang567 yanliang567

@NicoYuan1986 NicoYuan1986

Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
2.4.18
Development

No branches or pull requests
3 participants
@XuanYang-cn @yanliang567 @NicoYuan1986