Not sure about this one, but I think it may be useful to be a bit more explicit on this one:

On another thought per replica may be shorter to per shard per peer

Would it be worth it to create a function that builds and returns the new RateLimiters. We'd be able to use it in the constructor above, and here, sharing code.

Shall we make it a bit more strict and make failed a counter? And assert we've failed at least 5 (50%) times?

rate limit per second won't allow us to configure bursts. Could we switch to requests per minute?

Internally the rate limiter computes the number of requests allowed per second.

let tokens_per_sec = rate.requests_num() as f64 / rate.period().as_secs_f64();

Having an input of 10 req/s is equivalent to 600 req/m.

Those tokens are replenished in real time based on the elapsed time since the last check.
We do start at full capacity to allow burst on start.

Here is the impl:

pub fn check(&mut self) -> bool {
        let now = Instant::now();
        let elapsed = now.duration_since(self.last_check);
        self.last_check = now;

        // Refill tokens based on elapsed time.
        self.tokens += self.tokens_per_sec * elapsed.as_secs_f64();
        if self.tokens > self.capacity as f64 {
            self.tokens = self.capacity as f64;
        }

        if self.tokens >= 1.0 {
            self.tokens -= 1.0; // Consume one token.
            true // Request allowed.
        } else {
            false // Request denied.
        }
    }

Which kind of burst scenario are you concerned about?

Discussed with Andrey regarding his concerns, he wants to be able to have an actual burst capacity of one minute worth of tokens and not one second.

I will merge this PR as it is because it is tidy and has been reviewed.
Then propose the adjusted implementation in a new PR for clarity.