Grafana Docker and data persistence

Have you made the data directories first? Are the permissions on them correct? Personally i hate using $PWD and always set a full path. I like using /docker_data as it makes it obvious to anyone after me what it is.

Try this…

version: '3'

services:
  grafana:
    image: grafana/grafana
    container_name: grafana
    depends_on:
      - influxdb
    ports:
      - 3000:3000
    volumes:
      - /docker_data/grafana_data:/var/lib/grafana
      - /docker_data/certs:/certs
    env_file:
      - /docker_data/grafana_conf/config.monitoring 
    environment:
      - HTTP_USER="{{ grafana_user }}"
      - HTTP_PASS="{{ grafana_passwd }}"
      - INFLUXDB_HOST=influxdb
      - INFLUXDB_PORT=8086
      - INFLUXDB_NAME="{{ db_name }}"
      - INFLUXDB_USER="{{ influxdb_user }}"
      - INFLUXDB_PASS="{{ influxdb_passwd }}"
    restart: always
   
  prometheus:
    image: prom/prometheus
    container_name: prometheus
    volumes:
      - /docker_data/prometheus_conf:/etc/prometheus/
      - /docker_data/prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/usr/share/prometheus/console_libraries'
      - '--web.console.templates=/usr/share/prometheus/consoles'
    ports:
      - 9090:9090
    links:
      - grafana
      - influxdb
    restart: always 
  
  influxdb:
    image: docker.io/influxdb
    container_name: influxdb
    ports:
      - "8086:8086"
    volumes:
      - /docker_data/influxdb_data:/var/lib/influxdb
    env_file:
      - '/docker_data/influxdb_conf/env.influxdb' 
    restart: always  

That should give you a stack.

I use Ansible for all the lifting -

docker-install.yml

--- 
    - 
      connection: localhost
      hosts: localhost
      tasks: 
        - 
          args: 
            chdir: /home/
          command: "curl -fsSL get.docker.com -o get-docker.sh"
          name: "Download docker script"
        - 
          name: "run docker script"
          script: /home/get-docker.sh
        - 
          name: "start Docker"
          service: 
            name: docker
            state: started
        - 
          apt: 
            name: python-pip
            state: present
          name: "install pip"
          register: results
        - 
          debug: 
            var: results
        - 
          command: "pip install docker-compose"
          name: "install docker-compose"
        - 
          name: "include vars file"
          include_vars: docker_install.var.yml
        - 
          name: "create docker data folders"
          file:
            path: "{{ item }}"
            state: directory
            mode: 0777
          with_items:
              - /docker_data/portainer_data
              - /docker_data/influxdb_data
              - /docker_data/grafana_data
              - /docker_data/chronograf_data
              - /docker_data/prometheus_data
        -      
          name: "create docker config folders"
          file:
            path: "{{ item }}"
            state: directory
            mode: 0775
          with_items:
              - /docker_data/influxdb_conf
              - /docker_data/grafana_conf
              - /docker_data/prometheus_conf
        - 
          name: "create telegraf.conf from template values"
          template: 
            dest: /docker_data/telegraf_conf/telegraf.conf
            src: telegraf.conf.jk2
        - 
          name: "create telegraf.conf outputs from template values"
          template: 
            dest: /docker_data/telegraf_conf/telegraf.d/telegraf.outputs.conf
            src: telegraf.outputs.conf.j2
        - 
          name: "create telegraf.conf inputs from template values"
          template: 
            dest: /docker_data/telegraf_conf/telegraf.d/telegraf.inputs.conf
            src: telegraf.inputs.jk2        
        - 
          file: 
            mode: 420
            path: /docker_data/telegraf_conf/telegraf.conf
          name: "change file permission"
        - 
          name: "create grafanacfg.output file with values"
          template: 
            dest: /docker_data/grafana_conf/grafanacfg.output
            src: grafanacfg.input.jk2
        - 
          name: "create influx env file"
          template: 
            dest: /docker_data/influxdb_conf/env.influxdb
            src: env.influxdb.j2
        - 
          name: "create grafana config file"
          template: 
            dest: /docker_data/grafana_conf/config.monitoring
            src: config.monitoring.j2
        - 
          name: "create docker-compose.yml from template"
          template: 
            dest: /docker_data/docker-compose.yml
            src: docker-compose.yml.j2
        - 
          name: "create docker-compose .env from template"
          template: 
            dest: /docker_data/.env
            src: docker-compose.env.j2    
        - 
          args: 
            chdir: /docker_data/
          command: "docker-compose build"
          name: "docker build"
        - 
          args: 
            chdir: /docker_data/
          command: "docker-compose up -d"
          name: "docker up"
        - 
          name: "Wait 300 seconds for port 3000 to become open on the host, don't start checking for 10 seconds"
          wait_for: 
            delay: 10
            port: 3000
        - 
          name: "Set Grafana admin password"
          command: docker exec -t grafana bash -c  'grafana-cli --homepath /usr/share/grafana admin reset-admin-password "{{ grafana_passwd }}"'
        - 
          name: "check if grafana can be accessed"
          command: "curl -s http://grafana.staged-by-discourse.com/api/org -u {{ grafana_user }}:{{ grafana_passwd }}"
        - 
          name: "create datasoure in grafana"
          command: "curl -L --header 'Content-Type: application/json' --header 'Accept: application/json' -d@//docker_data/grafana_conf/grafanacfg.output http://grafana.staged-by-discourse.com/api/datasources -u {{ grafana_user }}:{{ grafana_passwd }}"
        - 
          name: "create TelegrafHost dashboard in grafana using jsonfile"
          command: "curl -L --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/TelegrafHostMetrics.json  http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
        - 
          name: "create DockerMetrics dashboard in grafana using jsonfile"
          command: "curl --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/DockerMetricsperContainer.json  http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
        - 
          name: "create JVM dashboard in grafana using jsonfile"
          command: "curl --header 'Content-Type: application/json' --header 'Accept: application/json' -d@/home/Configfiles/dashboards/jvm-metrics-jolokia-2_rev1.json  http://grafana.staged-by-discourse.com/api/dashboards/db -u {{ grafana_user }}:{{ grafana_passwd }}"
    

docker-install.var.yml (example this wont work!)

grafana_user: !vault |
        $ANSIBLE_VAULT;1.1;AES256
    
grafana_passwd: !vault |
        $ANSIBLE_VAULT;1.1;AES256

influxdb_admin_user: !vault |
        $ANSIBLE_VAULT;1.1;AES256

influxdb_admin_passwd: !vault |
        $ANSIBLE_VAULT;1.1;AES256

influxdb_user: !vault |
        $ANSIBLE_VAULT;1.1;AES256

influxdb_passwd: !vault |
        $ANSIBLE_VAULT;1.1;AES256

I suggest you use ansible-vault to not save passwords in plain text.

This should give you a good start, there are some files you will need to create, they are mainly your current config files saved as .jk2 or j2 (where I forgot the k). The dashborads I auto deploy can be removed or you can go to the Grafana page for them and download the yml.