Nodejs logging using winston

How to log from my node application using winston ?
I can find one article about it. But that howto requires patching to make it work,

Sending a log from an application must be pretty basic. But I can’t find any examples.
I hope someone has a simple example of getting a log from an application running in nodejs.
A simple “Hello world” :slight_smile:

Welcome

In your nodejs app just pop this in for starters

const { createLogger, transports, format } = require("winston");
const LokiTransport = require("winston-loki");


const l_log = createLogger({
    transports: [new LokiTransport({
        host: "http://127.0.0.1:3100",
        labels: { app: 'honeyshop'},
        json: true,
        format: format.json(),
        replaceTimestamp: true,
        onConnectionError: (err) => console.error(err)
      }),
      new transports.Console({
        format: format.combine(format.simple(), format.colorize())
      })]
  })

  l_log.info({ message: `Hello World`, labels: { 'origin': 'api' } })

Hi @yosiasz and thank you for your reply.

In your example, the host is on the same machine as the node app is running.
I would prefer not to set up grafana system myself. Rather use the Grafana Cloud Free account ( Grafana Cloud | Observability platform overview ).

As Dan Mihai Dinu writes in his post - winston-loki does not work when sending logs to the hosted grafana service.

Bummer, it doesn’t work. After a good hour of debugging, I noticed that winston-loki works fine with local instances of Grafana Loki, but it doesn’t authenticate properly to send requests to Grafana Cloud Logs

From the screenshot in his article, I see that the version he is patching is winston-loki 6.0.5.
The latest version is now 6.0.6 and I don’t know if the problem is fixed or not. And I’m not sure if his patch for version 6.0.5 will work on 6.0.6.

To me, this seems like a bug. But I’m not sure whether it is in Grafana cloud or winston-loki.

Hey,

You are correct @terchris . Bummer indeed. I think you are going to be the one to lead us where no person has gone before :wink:

So the main question is why would it not work when it is a remote host?
Also can one bypass winston-loki and hit loki using it’s own api?

And check out how other languages do it and grok it in javascript

Thinking out loud

Hi
Based on the article by Dan Mihai Dinu I have tested sending data to grafana cloud using postman.


It works just fine. There is a undocumented part in his article. That is the first parameter in values. In my example, when I tested yesterday, I set it to “1669731652002000001”,
It is the UNIX timestamp ( https://www.unixtimestamp.com/)

Also note that the URL contains the userid and password. Mine looks like this:
https://333666:eyJrIjoiMDc5.......@logs-prod-eu-west-0.grafana.net/loki/api/v1/push

This is the body of my message:

{
  "streams": [
    {
      "stream": {
        "level": "info",
        "app": "urbalurba"
      },
      "values": [
        [
             "1669731652002000001",
          "{\"timestamp\":\"2022-11-15T12:08:57.322Z\",\"logMessage\":\"Updated merge\",\"jobname\":\"webpage_info2merge\",\"idName\":\"bamble.kommune.no\",\"counter\":null,\"totalCount\":null,\"level\":\"info\",\"message\":\"Updated merge\"}"
        ]
      ]
    }
  ]
}

Cool. Not sure what you are trying to point out in your last post. Anyways I would Implement what worked in postman in the code for nodejs

Meaning a POST also adding user name and token in the url

Or I believe createLogger has one option for that: basicAuth

Hi
In my previous post I was just pointing out that it is possible to send log directly to grafana cloud using the API.

I could implement my own function to just send my log directly to grafana cloud using that API. Then there would be no need for winston-loki at all.
But I’m sure that the internal workings of winston-loki is more complex than just a wrapper to that API. So it is better to solve the problem in the winston-loki.

I do not have the skills to fix the winston-loki :frowning:

1 Like

Are you 100% sure? :wink:

Check this out

logger.add(new LokiTransport({
  host: 'http://127.0.0.1:3100',
  json: true,
  basicAuth: 'username:password',
  labels: { job: 'winston-loki-example' }
}))

So I spun up a cloud loki

image

then using the basicAuth setting I used the user and password as follows

transports: [new LokiTransport({
        host: "https://your_cloud_url_from_above",
        labels: { app: 'kumbamboom'},
        json: true,
        basicAuth: 'user_from_above:password_from_above',
        format: format.json(),
        replaceTimestamp: true,
1 Like

Thanks @yosiasz
Based on your input and Winston-loki not working with Grafana Cloud Loki · Issue #108 · JaniAnttonen/winston-loki · GitHub I created this simple code. I think that my problem was that I had my parameters wrong (URL and probably password). There are probably return codes that can be inspected to help in case it does not work. I was looking for a simple code to test- I could not find one so I created it.


/* The siplest test ever to see if grafana cloud and winston is working 
A small node app that sends a log message to Grafana Cloud Loki using the winston-loki transport.
To you by terchris and friends.
*/
import winston from "winston";
import LokiTransport from "winston-loki"

// https://grafana.com/orgs/urbalurba (change urbalurba to your own org)
// in the Loki card. Click on the Details button. On the next page you will se the Grafana Data Source settings.
// Create a password  and replace the const variables below with your own values. 
const GRAFANA_HOST = "https://logs-prod-eu-west-0.grafana.net";
const MY_APP_NAME = "urbalurba";

const GRAFANA_USERID = "333666";
const GRAFANA_PASSWORD = "eyJr1234567890FjYzZlOTg2NTE5ZDIyY12345678900ODdlZjAx1234567890IsIm4iOiJ1234567890iwi1234567890cyM30=";

// start leave this as is
const GRAFANA_BASICAUTH = GRAFANA_USERID + ":" + GRAFANA_PASSWORD;
const logger = winston.createLogger({

    level: 'debug',
      
    transports: [
      new LokiTransport({
        host: GRAFANA_HOST,
          labels: { app: MY_APP_NAME },
          json: true,
          basicAuth: GRAFANA_BASICAUTH,
          format: winston.format.json(),
          replaceTimestamp: true,
          onConnectionError: (err) => console.error(err)
        }),
      new winston.transports.Console({}),      
    ],
  });
// end leave this as is  

// send some log messages
logger.info("Starting test");
logger.debug("sending debug message");
logger.warn("sending warn message");
logger.error("sending error message");
logger.info("done testing");

// To see the logging in grafana cloud, go to Explore https://urbalurba.grafana.net/explore (change urbalurba to your own org)
// At the top there is a dropdown. Select the one with the Loki logo (in my case grafanacloud-urbalurba-logs) 
// In the "Select label" dropdown select "app" and in the "Select value" dropdown select "urbalurba" (change urbalurba to your own app name)
// Click on the "Run Query" button in the upper right corner. You should see the log messages you sent above.
1 Like

@terchris Hey there.
I tried hitting the url using postman. I’m getting a 204 response and the labels are also getting created in Loki but the logs shows “No logs found” What am I missing? TIA

Update: The logs for the post request that I made yesterday is showing up now. But its not showing for the recent requests. I guess there is a lag. Any help will be appreciated.

@architectinprogress I was facing the same issue, look at the timestamp in nanoseconds that you send and see that the endpoint matches the one needed for Loki!
Note: by default, Grafana only looks 6h ago.
POST https://<username>:<api_key>@<subdomain>.grafana.net/loki/api/v1/push

{
  "streams": [
    {
      "stream": {
        "level": "info",
        "app": "bees"
      },
      "values": [
        [
          "1692536838000000000",
          "{\"level\":\"info\",\"message\":\"Hello bananas!\"}"
        ]
      ]
    }
  ]
}

If anyone needs an on the fly patch for winston-loki, this worked for me:

import winston from 'winston';
import LokiTransport from 'winston-loki';
import dotenv from 'dotenv';
import fetch from 'node-fetch';

dotenv.config(); // Load environment variables

// Extend LokiTransport
class ExtendedLokiTransport extends LokiTransport {
    constructor(options) {
        // Pass options to the base constructor
        super(options);

        // Ensure options are stored
        this.options = options;
    }

    log(info, callback) {
        const { message, ...metadata } = info;

        // Prepare the log entry
        const logEntry = {
            message,
            ...metadata
        };

        // Remove the `level` field from metadata
        delete metadata.level;

        // Prepare the payload
        const payload = {
            streams: [
                {
                    stream: this.options.labels || {}, // Ensure labels are provided
                    values: [[`${Date.now()}000000`, JSON.stringify(logEntry)]],
                },
            ],
        };

        // Prepare headers
        const headers = {
            'Content-Type': 'application/json',
            Authorization: `Basic ${Buffer.from(this.options.basicAuth).toString('base64')}`,
        };

        // Send the payload to Loki
        fetch(this.options.host, {
            method: 'POST',
            headers,
            body: JSON.stringify(payload),
        })
            .then((response) => {
                if (!response.ok) {
                    console.error(`Loki transport error (${response.status}): ${response.statusText}`);
                } else {
                    console.log('Log successfully sent to Loki');
                }
                callback();
            })
            .catch((err) => {
                console.error('Loki transport fetch error:', err);
                callback(err);
            });
    }
}

// Create the logger using the extended transport
export const logger = winston.createLogger({
    level: 'info',
    format: winston.format.json(),
    transports: [
        new ExtendedLokiTransport({
            host: process.env.LOKI_URL,
            basicAuth: `${process.env.LOKI_USERNAME}:${process.env.LOKI_PASSWORD}`,
            labels: { app: 'exactplay-unified-api-server' },
        }),
    ],
});
1 Like