Logs from new server not available on Grafana Cloud

Grafana Loki
1

Hello,

I successfully configured the Grafana Agent to send some logs from one server to a Grafana Cloud instance.

Now I want to configure the same type of setup but on a different internal server to send different logs to the same Grafana cloud instance. I tried to do a similar configuration than the first server for which everything works, but when I generate test error logs to test my setup on my second server, nothing is displayed in the Grafana Cloud instance.
Here are the checks that I made:

  • Last version of Grafana Agent installed: v0.36.0
  • Grafana Agent service running
  • positions file updated with a new line corresponding to the test log file soon after generating it.
  • servers are on Windows Server 2019

Here is the agent-config.yaml from the server from which the logs are not sent. I just replaced the username value by “username” and the password value by “APIKey”.
I tested the “Teliway” job, which is not available to select in the Explore section of my Grafana Cloud instance during the timeframe when I generated the test log file, as if no log was sent to Grafana Cloud.

integrations:
  prometheus_remote_write:
  - basic_auth:
      password: APIKey
      username: username
    url: https://prometheus-prod-22-prod-eu-west-3.grafana.net/api/prom/push
  agent:
    enabled: true
    relabel_configs:
    - action: replace
      source_labels:
      - agent_hostname
      target_label: instance
    - action: replace
      target_label: job
      replacement: "integrations/agent-check"
    metric_relabel_configs:
    - action: keep
      regex: (prometheus_target_.*|prometheus_sd_discovered_targets|agent_build.*|agent_wal_samples_appended_total|process_start_time_seconds)
      source_labels:
      - __name__
  # Add here any snippet that belongs to the `integrations` section.
  # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
logs:
  configs:
  - clients:
    - basic_auth:
        password: APIKey
        username: username
      url: https://logs-prod-013.grafana.net/loki/api/v1/push
    name: integrations
    positions:
      # filename: /tmp/positions.yaml
      filename: "D:\\Grafana\\Positions\\positions.yaml"
    scrape_configs:
      # Add here any snippet that belongs to the `logs.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
    - job_name: integrations/agent
      journal:
        max_age: 24h
        labels:
          instance: <hostname>
          job: integrations/agent
      pipeline_stages:
        - match:
            selector: '{unit!="grafana-agent.service"}'
            action: drop
            drop_counter_reason: only_keep_grafana_agent_logs
        - regex:
            expression: "(level=(?P<log_level>[\\s]*debug|warn|info|error))"
        - labels:
            level: log_level
      relabel_configs:
        - source_labels: ['__journal__systemd_unit']
          target_label: 'unit'
    - job_name: YoozArchivesErreurs
      encoding: ISO-8859-1
      static_configs:
      - targets:
          - localhost
        labels:
          job: Yooz
          level: error
          __path__: "C:\\PGI01\\Yooz_Exchange\\Dev\\Archives erreurs\\*.ERR"
    - job_name: YoozWORK
      encoding: ISO-8859-1
      static_configs:
      - targets:
          - localhost
        labels:
          job: Yooz
          level: error
          __path__: "C:\\PGI01\\Yooz_Exchange\\Dev\\WORK\\**\\*.ERR"
    - job_name: TeliwayCegid
      encoding: ISO-8859-1
      static_configs:
      - targets:
          - localhost
        labels:
          job: Teliway
          level: error
          __path__: "C:\\PGI01\\Import_Teliway\\**\\*.ERR"
metrics:
  configs:
  - name: integrations
    remote_write:
    - basic_auth:
        password: APIKey
        username: username
      url: https://prometheus-prod-22-prod-eu-west-3.grafana.net/api/prom/push
    scrape_configs:
      # Add here any snippet that belongs to the `metrics.configs.scrape_configs` section.
      # For a correct indentation, paste snippets copied from Grafana Cloud at the beginning of the line.
  global:
    scrape_interval: 60s
  # wal_directory: /tmp/grafana-agent-wal
  wal_directory: "C:\\ProgramData\\grafana-agent-wal"

Could you please help me on how to debug this?

Thanks

2

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.