What would you like to be added?

Etcd has multiple processes that executed by leader, however neither etcd nor raft guarantees that there is only 1 cluster member that identifies itself as a leader at one time.

Raft only guarantees that commited entries will not be lost by ensuring that they are persisted on quorum of members and electing a leader requires a quorum. However, there are periods of time where two members can consider themselves a leader, even though in reality they no longer have quorum and cannot commit any entries. An example is when leader is disconnected from other members and they elect new leader, there will be a period of time when old and new leader will be present.

TODO:

• Identify all the etcd processes that depend on being executed by only the leader aka isLeader(). Example cluster version picking.
• Define list of invariants that those process should fulfil. For example: idempotent, transactional etc.
• Validate those invariants on all of those processes
• Codify those invariants so no future changes impact etcd correctness.

Why is this needed?

To surface the issue with incorrect pattern used in etcd and improve the codebase.