I met some stranger question. When I Set the promtail configuation as pipline_stage to regex incoming log’s timepstamp and level,like this
server:
http_listen_port: 9080
grpc_listen_port: 0
http_server_read_timeout: 20s
http_server_write_timeout: 10s
grpc_server_max_recv_msg_size: 1048576
grpc_server_max_send_msg_size: 1048576
grpc_server_max_concurrent_streams: 512
log_level: info
positions:
filename: /app/docker/promtail/application-positions.yaml
sync_period: 10s
target_config:
sync_period: 10s
clients:
- url: http://loki-ali.hfinside.com/loki/api/v1/push
tenant_id: prod
scrape_configs: - job_name: consul
static_configs:- targets:
- localhost
labels:
app: ‘fpc’
ip: 172.19.38.72
path: /app/docker/fpc/logs/*.log
pipeline_stages:
- localhost
- regex:
expression: ‘(?P\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}) (?P[A-Z]+) .*’ - labels:
timestamp:
level:
- targets:
after this configuation,the entry log will be dupilcated and displayed on grafana
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
Log labels
app fpc
ip 172.19.38.72
filename /app/docker/fpc/logs/rocketmq_client-2022-08-04.0.log
level INFO
timestamp 2022-08-04 00:00:00.031
Detected fields
ts 2022-08-05T13:44:51.811Z
tsNs 1659707091811610186
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
2022-08-05 21:44:51
2022-08-04 00:00:00.031
2022-08-04 00:00:00.031 INFO [MQClientFactoryScheduledThread] RocketmqClient - [PULL_TPS] [PROD_C_RMQ_PA_ACCT_IN_RESP_1025@PROD_Q_PA_FPC_INACCT_RESP_NOTIFY] Stats In One Minute, SUM: 26 TPS: 0.43 AVGPT: 1.00
there are 9 row same log send to loki server?
but when I delete the pipeline_stage in configuration,everything become to normal,just get 1 row on the grafana