I have an embedded hardware device which is not capable of running promtail. It exposes its log on a plain-text HTTP endpoint. Is there some way to ingest this data to Loki? I’d like to poll it every minute and ingest the new lines, skipping all the lines which were already ingested. Each line does have a valid timestamp, but I don’t know how to avoid ingesting duplicates.
I don’t think promtail supports that. I’d write a simple script and forwards the logs to a listening promtail agent (or even to Loki directly).
Ok thanks for the response. If my script forwards the entire log output, would promtail be smart enough to ignore the lines it already has, or would I need to include that parsing logic myself?
You’ll need to keep a checkpoint on your script and discard logs that have already been forwarded.