Consider an option to set UNICORN_SIDEKIQ_MAX_RSS environment variable

[kot0dama]

Enhancement Proposal

Hi,

Our Discourse instances are frequently failing health checks. After digging up the root cause, it would seem Sidekiq memory usage would be (at least one of) the culprit:

Sidekiq is consuming too much memory (using: 597.82M) for 'discourse.ubuntu.com', restarting
Booting Sidekiq 6.5.12 with Sidekiq::RedisConnection::RedisAdapter options {:host=>"REDACTED", :port=>6379, :namespace=>"sidekiq"}
[...]
Detected rogue Sidekiq pid 418761 mem 588632064, killing
Booting Sidekiq 6.5.12 with Sidekiq::RedisConnection::RedisAdapter options {:host=>"REDACTED", :port=>6379, :namespace=>"sidekiq"}

This is a well-known issue in the community:
https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory/280933
https://meta.discourse.org/t/sidekiq-is-consuming-too-much-memory-restarting/48395/36

The workaround is to up the maximum memory through setting the UNICORN_SIDEKIQ_MAX_RSS environment variable to 1000, thus allowing 1GB instead of 500MB default per:

config/unicorn.conf.rb:
        [ENV["UNICORN_SIDEKIQ_MAX_RSS"].to_i, 500].max.megabytes

A workaround this issue is to manually set the environment variable in the deployment, by updating the statefulset with:

--- discourse-k8s.ss.yaml.orig  2024-12-05 02:45:56.429989147 +0000
+++ discourse-k8s.ss.yaml       2024-12-05 02:38:31.271628120 +0000
@@ -115,6 +115,8 @@
           value: discourse
         - name: PEBBLE_SOCKET
           value: /charm/container/pebble.socket
+        - name: UNICORN_SIDEKIQ_MAX_RSS
+          value: "1000"
         image: registry.jujucharms.com/charm/1do4bht9nkx2kuk2thi94lw9kzlgiytu42u4r/discourse-image@sha256:454c0dcf2ab0707869ef1f59bf3692668af0b45b9e10fbfe35cfd89554ba6c88
         imagePullPolicy: IfNotPresent
         livenessProbe:

Would it be possible to add a charm option to achieve this behavior?
Thank you!

[hloeung]

@alithethird, can we land a quick fix to bump this to 1000 per @kot0dama's post for now? Then think about how to make it tunable at a later stage.

[alithethird]

Created the PR. It is configurable and set to 1000 in default.

[hloeung]

Nice one!

[alithethird]

Merged the feature.

[kot0dama]

Wonderful, thank you!