Log structure/ json parsing (noob)

Grafana Explore
,
1

Hi Forum,

I’ve totally new to loki (also new-ish to grafana!), and I’m particularly interested in outputting information from python scripts. I’m struggling to understand the proper relationship between labels, log messages and json parsing - fundamentally struggling to understand the log message structure.

I’ve also noticed some differences in behaviour between the docker container and the cloud offering

Ideally I’d send some quite broad log messages which may contain strings and numbers (floats and ints) that I may want to graph as well - here’s a snippet:

class GrafanaClient:

    def __init__(self):
        self.logsource = os.getenv('HOSTNAME')
        self.username = os.getenv('GRAFANA_USERNAME')
        self.password = os.getenv('GRAFANA_API_KEY')
        self.loki_server = os.getenv('GRAFANA_LOKI')

    def loki_log(self, loki_log):
        # push msg log into grafana-loki
        push_route = '/loki/api/v1/push'
        url = f'https://{self.username}:{self.password}@{self.loki_server}{push_route}'
        logger.info(f'target endpoint: {url}')
        headers = {
            'Content-type': 'application/json'
        }
        logger.info(loki_log)
        log = requests.post(url, json=loki_log, headers=headers)
        logger.info(f'responsed with {log.status_code}')

def main():
    load_dotenv()
    client = GrafanaClient()
    cloud_labels = {
        "source": "test-metrics",
        "job": "varlogs",
        "host": client.logsource
    }
    cloud_payload = {
        "streams": [
            {
                "stream": cloud_labels,
                "values": [
                    [
                        cloudtime(),
                        "fizzbuzz",
                    ]
                ]
            }
        ]
    }
    client.loki_log(cloud_payload)


if __name__ == "__main__":
    main()

So in my “cloud_payload” I was hoping to put something more interesting in there, ideally so I can compose some useful info from tests or scripts.

I started refactoring an example from github, into a shape that might not be the best but I prefer to work with so fizzbuzz is a pasteable! Like I say I’m messsing around with a cloud grafana trial at the moment.

Loki accepts the message but oh my, I guess I want a timeseries/ log hybrid type thing - maybe I am using the wrong tool! :confused:

Thanks for reading this far and any help really appreciated.