Add troubleshooting docs

OpenBMP · Mar 7, 2022 · f207011 · f207011
1 parent 30f1a4e
commit f207011
Show file tree

Hide file tree

Showing 3 changed files with 57 additions and 7 deletions.
diff --git a/getting_started.md b/getting_started.md
@@ -149,13 +149,13 @@ You should have a similar output as below from ```docker ps``:
 
 ```
 CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                          NAMES
-1be395ea4401        confluentinc/cp-kafka:7.0.1       "/etc/confluent/dock…"   30 minutes ago      Up 30 minutes       0.0.0.0:9092->9092/tcp         obmp-kafka
-f8a7331889f4        openbmp/whois:2.1.0               "/bin/sh -c '/usr/lo…"   30 minutes ago      Up 30 minutes       0.0.0.0:4300->43/tcp           obmp-whois
-584af4f63798        openbmp/postgres:2.1.0            "docker-entrypoint.s…"   30 minutes ago      Up 30 minutes       0.0.0.0:5432->5432/tcp         obmp-psql
-b9b8d09d13ee        grafana/grafana:8.3.4             "/run.sh"                30 minutes ago      Up 30 minutes       0.0.0.0:3000->3000/tcp         obmp-grafana
-03154609cd99        confluentinc/cp-zookeeper:7.0.1   "/etc/confluent/dock…"   30 minutes ago      Up 30 minutes       2181/tcp, 2888/tcp, 3888/tcp   obmp-zookeeper
-29114c109b23        openbmp/psql-app:2.1.0            "/usr/sbin/run"          30 minutes ago      Up 30 minutes       0.0.0.0:9005->9005/tcp         obmp-psql-app
-7e97347b4ae8        openbmp/collector:2.1.0           "/usr/sbin/run"          30 minutes ago      Up 30 minutes       0.0.0.0:5000->5000/tcp         obmp-collector
+f262aee1b240        confluentinc/cp-kafka:7.0.1       "/etc/confluent/dock…"   39 minutes ago      Up 39 minutes       0.0.0.0:9092->9092/tcp         obmp-kafka
+aed91c7ef358        confluentinc/cp-zookeeper:7.0.1   "/etc/confluent/dock…"   40 minutes ago      Up 39 minutes       2181/tcp, 2888/tcp, 3888/tcp   obmp-zookeeper
+5a1add7bf676        openbmp/postgres:2.1.0            "docker-entrypoint.s…"   40 minutes ago      Up 40 minutes       0.0.0.0:5432->5432/tcp         obmp-psql
+95f011c89965        openbmp/psql-app:2.1.0            "/usr/sbin/run"          40 minutes ago      Up 39 minutes       0.0.0.0:9005->9005/tcp         obmp-psql-app
+757f03efdb10        grafana/grafana:8.3.4             "/run.sh"                40 minutes ago      Up 40 minutes       0.0.0.0:3000->3000/tcp         obmp-grafana
+61e2af0a4044        openbmp/collector:2.1.0           "/usr/sbin/run"          40 minutes ago      Up 40 minutes       0.0.0.0:5000->5000/tcp         obmp-collector
+159678da7f34        openbmp/whois:2.1.0               "/bin/sh -c '/usr/lo…"   40 minutes ago      Up 40 minutes       0.0.0.0:4300->43/tcp           obmp-whois
 ```
 
 You should be able to login to grafana at http://<vm ip/name>:3000/

diff --git a/troubleshooting/README.md b/troubleshooting/README.md
@@ -0,0 +1,7 @@
+---
+sort: 5
+---
+
+# Troubleshooting
+
+{% include list.liquid all=true %}
diff --git a/troubleshooting/postgres.md b/troubleshooting/postgres.md
@@ -0,0 +1,43 @@
+---
+title: Postgres
+---
+
+# Postgres Troubleshooting
+
+## Missing Tables
+
+### Symptom
+You might see errors in ```obmp-psql.log``` indicating missing tables or in Grafana you'll have
+blank/empty graphs. 
+
+### Cause
+There are several time series tables (www.timescale.com) that are configured to use
+[postgres tablespace](https://www.postgresql.org/docs/14/manage-ag-tablespaces.html) **timeseries**.  This
+tablespace is created when you first install the ```openbmp/postgres``` container. 
+
+If you are missing the timeseries tablespace, then several tables will be missing. 
+
+### Fix
+If you are not using the ```openbmp/postgres``` container, then you will need to manually
+configure the **timeseries** tablespace.  This can use the same disk/volume as main, but normally
+you would dedicate storage for the time series data to allow for larger retention times.
+
+See [obmp-docker/postgres/Dockerfile](https://github.com/OpenBMP/obmp-docker/blob/main/postgres/Dockerfile) for
+details on how it creates the **timeseries** tablespace.
+
+After creating the timeseries tablespace, you'll need to re-initialize the DB. You can do that by
+running the following commands. 
+
+```warning
+The below will erase your current DB. If you do not want to do that, then you'll need to manually
+add the timeseries tables. This is in [obmp-psql/database](https://github.com/OpenBMP/obmp-psql/blob/main/database/1_base.sql).
+You will also need to add the views that are missing becuase the tables were not created the first time.
+There are only a few views that reference the timeseries tables. 
+```
+
+```
+rm -f ${OBMP_DATA_ROOT}/config/do_not_init_db
+OBMP_DATA_ROOT=${OBMP_DATA_ROOT} docker-compose -p obmp stop psql-app
+OBMP_DATA_ROOT=${OBMP_DATA_ROOT} docker-compose -p obmp up -d
+```
+