Skip to content

Large numbers of parties cause the dreaded RESOURCE_EXHAUSTED error #10283

Open
@da-tanabe

Description

A large number of parties onboarded onto any ledger will eventually cause clients that list the universe of parties to receive this lovely error (with the max size of course highly dependent on the number of parties):

io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 4870993
	at io.grpc.Status.asRuntimeException(Status.java:535)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:553)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:68)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:739)

The right solution is to probably change the ListKnownParties call to return a stream of ListKnownPartiesResponse instead of a single object, and to chunk up the responses:

// List the parties known by the backing participant.
// The list returned contains parties whose ledger access is facilitated by
// backing participant and the ones maintained elsewhere.
// Errors:
// - ``UNAUTHENTICATED``: if the request does not include a valid access token
// - ``PERMISSION_DENIED``: if the claims in the token are insufficient to perform a given operation
rpc ListKnownParties (ListKnownPartiesRequest) returns (ListKnownPartiesResponse);

With the current API design, this endpoint will only ever return more data over time, and raising this limit globally, forever doesn't seem like a good idea.

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions