chore: Set statement timeout for Postgres (#6641)

By default, Rails does not set a timeout on database statements. For example, this will run for a full day, even if your ruby process goes away. But it's configurable in the database.yml with the statement_timeout variable. Hence we are enforcing a 14s timeout by default. Migration commands inside chatwoot will run with a 10 minutes timeout. For specific cases like migrations, we can override this timeout using the environment variable POSTGRES_STATEMENT_TIMEOUT while starting a new rails console. Test the timeouts from the rails console using. ``` ActiveRecord::Base.connection.execute("SELECT pg_sleep(15);") ``` ref: https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts#postgresql ref: https://til.hashrocket.com/posts/b44baf657d-railspg-statement-timeout-
chatwoot · Mar 13, 2023 · 7cbf185 · 7cbf185
1 parent 8f4d479
commit 7cbf185
Show file tree

Hide file tree

Showing 6 changed files with 12 additions and 7 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -25,7 +25,7 @@
   // 1025,8025 mailhog
   "forwardPorts": [8025, 3000, 3035],
 
-  "postCreateCommand": ".devcontainer/scripts/setup.sh && bundle exec rake db:chatwoot_prepare && yarn",
+  "postCreateCommand": ".devcontainer/scripts/setup.sh && POSTGRES_STATEMENT_TIMEOUT=600s bundle exec rake db:chatwoot_prepare && yarn",
   "portsAttributes": {
     "3000": {
       "label": "Rails Server"

diff --git a/.env.example b/.env.example
@@ -52,6 +52,8 @@ POSTGRES_HOST=postgres
 POSTGRES_USERNAME=postgres
 POSTGRES_PASSWORD=
 RAILS_ENV=development
+# Changes the Postgres query timeout limit. The default is 14 seconds. Modify only when required.
+# POSTGRES_STATEMENT_TIMEOUT=14s
 RAILS_MAX_THREADS=5
 
 # The email from which all outgoing emails are sent

diff --git a/Procfile b/Procfile
@@ -1,3 +1,3 @@
-release: bundle exec rails db:chatwoot_prepare
+release: POSTGRES_STATEMENT_TIMEOUT=600s bundle exec rails db:chatwoot_prepare
 web: bin/rails server -p $PORT -e $RAILS_ENV
 worker: bundle exec sidekiq -C config/sidekiq.yml
diff --git a/bin/update b/bin/update
@@ -21,7 +21,7 @@ chdir APP_ROOT do
   # system('bin/yarn')
 
   puts "\n== Updating database =="
-  system! 'bin/rails db:migrate'
+  system! 'POSTGRES_STATEMENT_TIMEOUT=600s bin/rails db:migrate'
 
   puts "\n== Removing old logs and tempfiles =="
   system! 'bin/rails log:clear tmp:clear'

diff --git a/config/database.yml b/config/database.yml
@@ -5,7 +5,10 @@ default: &default
   port: <%= ENV.fetch('POSTGRES_PORT', '5432') %>
    # ref: https://github.com/mperham/sidekiq/issues/2985#issuecomment-531097962
   pool: <%= Sidekiq.server? ? Sidekiq.options[:concurrency] : ENV.fetch('RAILS_MAX_THREADS', 5) %>
-
+  variables:
+    # we are setting this value to be close to the racktimeout value. we will iterate and reduce this value going forward
+    statement_timeout: <%= ENV["POSTGRES_STATEMENT_TIMEOUT"] || "14s" %>
+
 development:
   <<: *default
   database: <%= ENV.fetch('POSTGRES_DATABASE', 'chatwoot_dev') %>

diff --git a/deployment/setup_20.04.sh b/deployment/setup_20.04.sh
@@ -362,7 +362,7 @@ EOF
 function run_db_migrations(){
   sudo -i -u chatwoot << EOF
   cd chatwoot
-  RAILS_ENV=production bundle exec rails db:chatwoot_prepare
+  RAILS_ENV=production POSTGRES_STATEMENT_TIMEOUT=600s bundle exec rails db:chatwoot_prepare
 EOF
 }
 
@@ -574,7 +574,7 @@ The database migrations had not run as Postgres and Redis were not installed
 as part of the installation process. After modifying the environment
 variables (in the .env file) with your external database credentials, run
 the database migrations using the below command.
-'RAILS_ENV=production bundle exec rails db:chatwoot_prepare'.
+'RAILS_ENV=production POSTGRES_STATEMENT_TIMEOUT=600s bundle exec rails db:chatwoot_prepare'.
 ***************************************************************************
 
 EOF
@@ -744,7 +744,7 @@ function upgrade() {
   rake assets:precompile RAILS_ENV=production
 
   # Migrate the database schema
-  RAILS_ENV=production bundle exec rake db:migrate
+  RAILS_ENV=production POSTGRES_STATEMENT_TIMEOUT=600s bundle exec rake db:migrate
 
 EOF