In Lucas's upgrades PR a discussion span off about how we can ensure clusters are never left in a corrupted state after failed/borked upgrades. I think @pipejakob encapsulated it best:

It would be great if our upgrade plan had ... properties, so that kubeadm or any random component could crash at any time during the upgrade, and we would never leave the cluster in an unusable state where human intervention would be required to fix it

More context here: kubernetes/kubernetes#48899 (comment). We should look at if there's any easy way to do this.

/cc @luxas @timothysc