Skip to content

Commit

Permalink
Updates for 2.0.1
Browse files Browse the repository at this point in the history
  • Loading branch information
TimEvens committed Jan 28, 2022
1 parent efcbc0d commit e05642d
Show file tree
Hide file tree
Showing 2 changed files with 101 additions and 17 deletions.
57 changes: 40 additions & 17 deletions getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,32 +137,27 @@ Containers will restart if they crash or if the system is rebooted.
* Login to grafana via http://<ip/hostname>:3000/
* Click the link at the bottom left (above the help icon) to **Sign In**.
* Sign in as **admin** using the password in the compose file
* Click on the dashbard icon (middle left) and select **Manage**
* Click on **Home** to open that dashoard
* In the upper left, next to the dashboad name of **Home** there is a **star** icon. Click that.
* Click on the dashboard icon (middle left) and select **Browse**
* Under the **General** folder, click on **OBMP-Home**
* In the upper left, next to the dashboard name of **General/OBMP-Home** there is a **star** icon. Click that.
* Click on the user icon (same to sign in) and select **Preferences**
* Under **Preferences** select **Home** from the **Home Dashboard** list.
* Under **Preferences** select **OBMP-Home** from the **Home Dashboard** list.
* Click Save.

Now when you login, the home dashboard will be set.

```warning
**TODO:** Will try to fix this via grafana provisioning or similar.
```

The OBMP-Home dashboard should not be set.

### (8) Verify

You should have a similar output as below from ```docker ps``:

```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f3e1e6861593 openbmp/psql-app:build-50 "/usr/sbin/run" 32 minutes ago Up 20 minutes 0.0.0.0:8080->8080/tcp, 0.0.0.0:9005->9005/tcp obmp-psql-app
7a0e1393fffa confluentinc/cp-kafka:6.0.2 "/etc/confluent/dock…" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp obmp-kafka
f3de5896671d grafana/grafana:latest "/run.sh" About an hour ago Up About an hour 0.0.0.0:3000->3000/tcp obmp-grafana
b4250fd91080 openbmp/collector:build-50 "/usr/sbin/run" About an hour ago Up About an hour 0.0.0.0:5000->5000/tcp obmp-collector
7c8aa1766f9d openbmp/postgres:build-50 "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp obmp-psql
379afacb3b1d confluentinc/cp-zookeeper:6.0.2 "/etc/confluent/dock…" About an hour ago Up About an hour 2181/tcp, 2888/tcp, 3888/tcp obmp-zookeeper
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2d83f75e71fa confluentinc/cp-kafka:7.0.1 "/etc/confluent/dock…" 7 minutes ago Up 7 minutes 0.0.0.0:9092->9092/tcp obmp-kafka
1ca92be96ffe openbmp/psql-app:2.0.1 "/usr/sbin/run" 7 minutes ago Up 7 minutes 0.0.0.0:9005->9005/tcp obmp-psql-app
000f4f87fb1d grafana/grafana:8.3.4 "/run.sh" 7 minutes ago Up 7 minutes 0.0.0.0:3000->3000/tcp obmp-grafana
227a427fb754 openbmp/postgres:2.0.1 "docker-entrypoint.s…" 7 minutes ago Up 7 minutes 0.0.0.0:5432->5432/tcp obmp-psql
bdf6e6a376ea openbmp/collector:2.0.1 "/usr/sbin/run" 7 minutes ago Up 7 minutes 0.0.0.0:5000->5000/tcp obmp-collector
be1c6358d087 confluentinc/cp-zookeeper:7.0.1 "/etc/confluent/dock…" 7 minutes ago Up 7 minutes 2181/tcp, 2888/tcp, 3888/tcp obmp-zookeeper
```

You should be able to login to grafana at http://<vm ip/name>:3000/
Expand All @@ -188,6 +183,34 @@ You should see positive numbers in the offset columns and a lag normally less th
When you have a lot of peers that are all RIB dumping at the same time, the lag might be in the
millions. This should not last too long and it should normalize where you see a lag less than 1000.



###Check if Consumer is connected to Kafka

```
# Get the consumer group name - Below shows only one, which is the default obmp-psql-consumer
ubuntu@server:~$ docker exec -it obmp-kafka kafka-consumer-groups --bootstrap-server localhost:29092 --list
obmp-psql-consumer
# Query the consumer group. Loook for a connected host in the HOST column, next to consumer-id
ubuntu@server:~$ docker exec -it obmp-kafka kafka-consumer-groups \
--bootstrap-server localhost:29092 \
--describe --group obmp-psql-consumer
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
obmp-psql-consumer openbmp.parsed.collector 4 263 263 0 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
obmp-psql-consumer openbmp.parsed.router 6 10 10 0 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
obmp-psql-consumer openbmp.parsed.peer 5 6 6 0 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
obmp-psql-consumer openbmp.parsed.ls_link 2 4733 4733 0 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
obmp-psql-consumer openbmp.parsed.unicast_prefix 5 2550156 2550190 34 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
obmp-psql-consumer openbmp.parsed.base_attribute 2 1211641 1211656 15 obmp-psql-consumer-0b4cdb21-7847-4bcb-a303-6fdaec0fbd46 /172.24.0.6 obmp-psql-consumer
<truncated output>
```



### Clear and start a container

```
Expand Down
61 changes: 61 additions & 0 deletions release_notes/2.0.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: v2.0.1
---

Changes:

* Log4j security fixes by upgrading to 2.17.1
* Upgrades all components and libraries to current versions as of Jan-2022
* Various bug fixes
* Schema changes to improve performance
* AS_PATH and communities are now Postgres arrays with indexing. This improves
performance over using regexp. The improvement deprecates the as_path_analysis and
gen_asn_stats tables
* TimescaleDB compression has been added to many tables to improve disk usage
* Retention control has been moved from cron to timescaleDB job process. Retention should now
be changed via the retention policies. Status of the jobs can be monitored via
timescaleDB job status. See https://docs.timescale.com/api/latest/data-retention/ for more details.
* global_ip_rib table has been improved to support updates every 5 minutes
* RPKI validator deprecated and replaced with URL based downloads
* PeeringDB integration


# Upgrade to version 2.0.1

Manually merging the PSQL schema changes is possible, but it is not currently automated.
Unfortunately there are several changes to the schema that make
it difficult to update. For example, converting from string AS_PATH to array AS_PATH. For this reason,
it is recommended that you perform a wipe and fresh install. Future versions should support
auto merge/upgrades operations.

### Fresh Install to Upgrade

> You can separate the compose file to be deployed using many VMs. In order to do that, you'll need to
adjust the FQDNs and ports that are used to access resources, such as Kafka and Postgres.

1. Login to your VM and make a backup of the current docker-compose.yml
```cp docker-compose.yml docker-comopose.yml.bk```
2. Run ```OBMP_DATA_ROOT=/var/openbmp docker-compose -p obmp down``` to shutdown and remove the
current deployment.
3. ```wget https://raw.githubusercontent.com/OpenBMP/obmp-docker/main/docker-compose.yml```
4. Update the docker-compose.yml file variables and volumes based on your previous compose file.
You can use ```diff``` to see the differences that need to be merged/updated.
5. ```rm -f ${OBMP_DATA_ROOT}/config/do_not_init_db``` to allow the DB to be reinitialized
6. ```rm -rf ${OBMP_DATA_ROOT}/postgres/data/*``` and ```rm -rf ${OBMP_DATA_ROOT}/postgres/ts/*``` to
remove the current postgres data. **Make sure the files are deleted.** If using ```sudo``` the wildcard
doesn't work. Use ```sudo bash -c ...``` instead.
7. ```rm -rf ${OBMP_DATA_ROOT}/kafka-data/*``` to remove the current Kafka data.
8. ```rm -rf ${OBMP_DATA_ROOT}/zk-data/*; rm -rf ${OBMP_DATA_ROOT}/zk-log/*``` to remove the
current zookeeper data.
9. Update Grafana.
1. ```rm -rf ${OBMP_DATA_ROOT}/grafana/plugins /var/openbmp/grafana/alerting /var/openbmp/grafana/grafana.db```
2. ```rm -rf ${OBMP_DATA_ROOT}/grafana/dashboards/```
3. ```rm -rf ${OBMP_DATA_ROOT}/grafana/provisioning/```
4. ```git clone https://github.com/OpenBMP/obmp-grafana.git``` or do a ```git pull```
5. ```cp -r obmp-grafana/dashboards obmp-grafana/provisioning ${OBMP_DATA_ROOT}/grafana/```
10. ```OBMP_DATA_ROOT=/var/openbmp docker-compose -p obmp up -d``` to start the new/upgraded version
of OBMP. This will reinitialize the DB. It does take a little time on initial start.




0 comments on commit e05642d

Please sign in to comment.