Is there any chance we would be able to get the active_at_offset field added to the response message?

As discussed here, we currently have to wait until the last GRPC message to know the offset (if we requested LEDGEREND). This can be really resource intensive, as we need to cache the entire ACS before performing any action that depends on knowing the ledger position.

@lucianojoublanc-da My instinct would be to modify the behavior of the already existing offset such that it is populated in every response message. @cocreature pointed out that it would be breaking change though.

@cocreature Why would it be a breaking change? Current description [1] leaves it unspecified what is the content of the offset field in every but the last message. If anything, it hints that this field is not empty with the "Required" note.

If wonder if we could lose any useful semantics if we start populating offset in every message.
I thought about detecting the stream end but it seems useless because the stream should end by itself just after the last message.

[1] Current description of the offset field:

  // Included in the last message.
  // The client should start consuming the transactions endpoint with this offset.
  // The format of this field is described in ``ledger_offset.proto``.
  // Required
  string offset = 1;

Is there any chance we would be able to get the active_at_offset field added to the response message?

As discussed here, we currently have to wait until the last GRPC message to know the offset (if we requested LEDGEREND). This can be really resource intensive, as we need to cache the entire ACS before performing any action that depends on knowing the ledger position.

@lucianojoublanc-da : why do you need to see the offset in the response at all with this change? I am under the assumption that with this change the caller chooses the offset up front (e.g., using a call to GetLedgerEnd) and submits it to the ledger-api server, so the offset is already known to the caller before receiving any message. Isn't that so @pbatko-da ?

I am under the assumption that with this change the caller chooses the offset up front (e.g., using a call to GetLedgerEnd)

If you submit a call to getActiveContracts with LEDGEREND, that's not the case. This would save you one extra round-trip.

There's also the fact that the spec is somewhat ambiguous in this regard : I don't think it actually says that you'll get back the same offset that you set in the request ..

@lucianojoublanc-da My instinct would be to modify the behavior of the already existing offset such that it is populated in every response message. @cocreature pointed out that it would be breaking change though.

@cocreature Why would it be a breaking change? Current description [1] leaves it unspecified what is the content of the offset field in every but the last message. If anything, it hints that this field is not empty with the "Required" note.

I agree with you, that would be my preference, but I think that in practice there will be people who have coded up their client based on what they observe when e.g. calling the sandbox, and this would break that behaviour.

Code from this PR combined with the getLedgerEnd call is fixing the UX that Luciano has pointed out.
Therefore, we should not make a breaking change and start sending the offset on every message batch. We can leave that chestnut for later. It is a client-app performance optimization that I am not convinced really brings much to the table.

@lucianojoublanc-da : why do you need to see the offset in the response at all with this change? I am under the assumption that with this change the caller chooses the offset up front (e.g., using a call to GetLedgerEnd) and submits it to the ledger-api server, so the offset is already known to the caller before receiving any message. Isn't that so @pbatko-da ?

@meiersi-da - correct