While @saad-ali working on #10335, we noticed that influxdb container is killed, and restarted because exceeding its memory limit (the limit is 2Gi, but influxdb is using 3.2Gi at peak on 4 nodes cluster in @saad-ali's test). After influxdb is restarted, we found all data is lost. It turns out there is no volume back up its data at node:

https://github.com/GoogleCloudPlatform/kubernetes/blob/master/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml