CELERY_WORKER_SHUTDOWN_INTERVAL=65m

agnesyangwei · August 20, 2024, 6:06pm

Being new to Grafana OnCall, I’d like to understand the environment variable for Grafana OnCall
CELERY_WORKER_SHUTDOWN_INTERVAL=65m

Does it mean the celery pod restarting every 65m?
What is the reason behind?

THANK YOU!

agnesyangwei · August 26, 2024, 6:13pm

In values.yaml for grafana/oncall helm chart, found comments for worker_shutdown_interval

github.com

grafana/oncall/blob/0ac7c40671f866b23151821ab9f0d8ab08666261/helm/oncall/values.yaml#L183


      
            #     name: my-redis-tls
            #     defaultMode: 0640
          
          # Celery workers pods configuration
          celery:
            replicaCount: 1
            worker_queue: "default,critical,long,slack,telegram,webhook,celery,grafana,retry"
            worker_concurrency: "1"
            worker_max_tasks_per_child: "100"
            worker_beat_enabled: "True"
            ## Restart of the celery workers once in a given interval as an additional precaution to the probes
            ## If this setting is enabled TERM signal will be sent to celery workers
            ## It will lead to warm shutdown (waiting for the tasks to complete) and restart the container
            ## If this setting is set numbers of pod restarts will increase
            ## Comment this line out if you want to remove restarts
            worker_shutdown_interval: "65m"
            livenessProbe:
              enabled: true
              initialDelaySeconds: 30
              periodSeconds: 300
              timeoutSeconds: 10

  ## Restart of the celery workers once in a given interval as an additional precaution to the probes
  ## If this setting is enabled TERM signal will be sent to celery workers
  ## It will lead to warm shutdown (waiting for the tasks to complete) and restart the container
  ## If this setting is set numbers of pod restarts will increase
  ## Comment this line out if you want to remove restarts
  worker_shutdown_interval: "65m"