Detailed debug output sample from the collector of a part which causes the error when querying in Grafana:
2025-01-17 13:07:33.485 {"kind": "exporter", "data_type": "logs", "name": "debug"}
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(The request path {Path} does not match the path filter)
2025-01-17 13:07:33.485 -> Path: Str()
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(The request path does not match the path filter)
2025-01-17 13:07:33.485 SeverityNumber: Debug(5)
2025-01-17 13:07:33.485 SeverityText: Debug
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.7392168 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.7392168 +0000 UTC
2025-01-17 13:07:33.485 LogRecord #0
2025-01-17 13:07:33.485 InstrumentationScope Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware
2025-01-17 13:07:33.485 ScopeLogs SchemaURL:
2025-01-17 13:07:33.485 ScopeLogs #4
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(Request did not match any endpoints)
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(Request did not match any endpoints)
2025-01-17 13:07:33.485 SeverityNumber: Debug(5)
2025-01-17 13:07:33.485 SeverityText: Debug
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.7390758 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.7390758 +0000 UTC
2025-01-17 13:07:33.485 LogRecord #0
2025-01-17 13:07:33.485 InstrumentationScope Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware
2025-01-17 13:07:33.485 ScopeLogs SchemaURL:
2025-01-17 13:07:33.485 ScopeLogs #3
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(No candidates found for the request path '{Path}')
2025-01-17 13:07:33.485 -> Path: Str(/)
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(No candidates found for the request path '/')
2025-01-17 13:07:33.485 SeverityNumber: Debug(5)
2025-01-17 13:07:33.485 SeverityText: Debug
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.7390544 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.7390544 +0000 UTC
2025-01-17 13:07:33.485 LogRecord #0
2025-01-17 13:07:33.485 InstrumentationScope Microsoft.AspNetCore.Routing.Matching.DfaMatcher
2025-01-17 13:07:33.485 ScopeLogs SchemaURL:
2025-01-17 13:07:33.485 ScopeLogs #2
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(Request reached the end of the middleware pipeline without being handled by application code. Request path: {Method} {Scheme}://{Host}{PathBase}{Path}, Response status code: {StatusCode})
2025-01-17 13:07:33.485 -> StatusCode: Int(404)
2025-01-17 13:07:33.485 -> Path: Str(/)
2025-01-17 13:07:33.485 -> PathBase: Str()
2025-01-17 13:07:33.485 -> Host: Str(testteam123.d.aks.dummysite.com)
2025-01-17 13:07:33.485 -> Scheme: Str(http)
2025-01-17 13:07:33.485 -> Method: Str(GET)
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(Request reached the end of the middleware pipeline without being handled by application code. Request path: GET http://testteam123.d.aks.dummysite.com/, Response status code: 404)
2025-01-17 13:07:33.485 SeverityNumber: Info(9)
2025-01-17 13:07:33.485 SeverityText: Information
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.7395529 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.7395529 +0000 UTC
2025-01-17 13:07:33.485 LogRecord #2
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(Request finished {Protocol} {Method} {Scheme}://{Host}{PathBase}{Path}{QueryString} - {StatusCode} {ContentLength} {ContentType} {ElapsedMilliseconds}ms)
2025-01-17 13:07:33.485 -> QueryString: Str()
2025-01-17 13:07:33.485 -> Path: Str(/)
2025-01-17 13:07:33.485 -> PathBase: Str()
2025-01-17 13:07:33.485 -> Host: Str(testteam123.d.aks.dummysite.com)
2025-01-17 13:07:33.485 -> Scheme: Str(http)
2025-01-17 13:07:33.485 -> Method: Str(GET)
2025-01-17 13:07:33.485 -> Protocol: Str(HTTP/1.1)
2025-01-17 13:07:33.485 -> ContentLength: Int(0)
2025-01-17 13:07:33.485 -> : Empty()
2025-01-17 13:07:33.485 -> StatusCode: Int(404)
2025-01-17 13:07:33.485 -> ElapsedMilliseconds: Double(0.6366)
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(Request finished HTTP/1.1 GET http://testteam123.d.aks.dummysite.com/ - 404 0 - 0.6366ms)
2025-01-17 13:07:33.485 SeverityNumber: Info(9)
2025-01-17 13:07:33.485 SeverityText: Information
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.739474 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.739474 +0000 UTC
2025-01-17 13:07:33.485 LogRecord #1
2025-01-17 13:07:33.485 Flags: 1
2025-01-17 13:07:33.485 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.485 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.485 -> RequestPath: Str(/)
2025-01-17 13:07:33.485 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.485 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.485 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.485 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.485 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.485 -> {OriginalFormat}: Str(Request starting {Protocol} {Method} {Scheme}://{Host}{PathBase}{Path}{QueryString} - {ContentType} {ContentLength})
2025-01-17 13:07:33.485 -> QueryString: Str()
2025-01-17 13:07:33.485 -> Path: Str(/)
2025-01-17 13:07:33.485 -> PathBase: Str()
2025-01-17 13:07:33.485 -> Host: Str(testteam123.d.aks.dummysite.com)
2025-01-17 13:07:33.485 -> Scheme: Str(http)
2025-01-17 13:07:33.485 -> : Empty()
2025-01-17 13:07:33.485 -> : Empty()
2025-01-17 13:07:33.485 -> Method: Str(GET)
2025-01-17 13:07:33.485 -> Protocol: Str(HTTP/1.1)
2025-01-17 13:07:33.485 Attributes:
2025-01-17 13:07:33.485 Body: Str(Request starting HTTP/1.1 GET http://testteam123.d.aks.dummysite.com/ - - -)
2025-01-17 13:07:33.485 SeverityNumber: Info(9)
2025-01-17 13:07:33.485 SeverityText: Information
2025-01-17 13:07:33.485 Timestamp: 2025-01-17 12:07:28.7390022 +0000 UTC
2025-01-17 13:07:33.485 ObservedTimestamp: 2025-01-17 12:07:28.7390022 +0000 UTC
2025-01-17 13:07:33.484 LogRecord #0
2025-01-17 13:07:33.484 InstrumentationScope Microsoft.AspNetCore.Hosting.Diagnostics
2025-01-17 13:07:33.484 ScopeLogs SchemaURL:
2025-01-17 13:07:33.484 ScopeLogs #1
2025-01-17 13:07:33.484 Flags: 1
2025-01-17 13:07:33.484 Span ID: 945f812b6ade3461
2025-01-17 13:07:33.484 Trace ID: e1ea4a4f662dca8ec126f7d5d5172b94
2025-01-17 13:07:33.484 -> RequestPath: Str(/)
2025-01-17 13:07:33.484 -> RequestId: Str(0HN9MTGPVHB54:00000001)
2025-01-17 13:07:33.484 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.484 -> ParentId: Str(0000000000000000)
2025-01-17 13:07:33.484 -> TraceId: Str(e1ea4a4f662dca8ec126f7d5d5172b94)
2025-01-17 13:07:33.484 -> SpanId: Str(945f812b6ade3461)
2025-01-17 13:07:33.484 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" completed keep alive response.)
2025-01-17 13:07:33.484 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.484 Attributes:
2025-01-17 13:07:33.484 Body: Str(Connection id "0HN9MTGPVHB54" completed keep alive response.)
2025-01-17 13:07:33.484 SeverityNumber: Debug(5)
2025-01-17 13:07:33.484 SeverityText: Debug
2025-01-17 13:07:33.484 Timestamp: 2025-01-17 12:07:28.7394148 +0000 UTC
2025-01-17 13:07:33.484 ObservedTimestamp: 2025-01-17 12:07:28.7394148 +0000 UTC
2025-01-17 13:07:33.484 LogRecord #2
2025-01-17 13:07:33.484 Flags: 0
2025-01-17 13:07:33.484 Span ID:
2025-01-17 13:07:33.484 Trace ID:
2025-01-17 13:07:33.484 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" started.)
2025-01-17 13:07:33.484 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.484 Attributes:
2025-01-17 13:07:33.484 Body: Str(Connection id "0HN9MTGPVHB54" started.)
2025-01-17 13:07:33.484 SeverityNumber: Debug(5)
2025-01-17 13:07:33.484 SeverityText: Debug
2025-01-17 13:07:33.484 Timestamp: 2025-01-17 12:07:28.7386444 +0000 UTC
2025-01-17 13:07:33.484 ObservedTimestamp: 2025-01-17 12:07:28.7386444 +0000 UTC
2025-01-17 13:07:33.484 LogRecord #1
2025-01-17 13:07:33.484 Flags: 0
2025-01-17 13:07:33.484 Span ID:
2025-01-17 13:07:33.484 Trace ID:
2025-01-17 13:07:33.484 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" accepted.)
2025-01-17 13:07:33.484 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:07:33.484 Attributes:
2025-01-17 13:07:33.484 Body: Str(Connection id "0HN9MTGPVHB54" accepted.)
2025-01-17 13:07:33.484 SeverityNumber: Debug(5)
2025-01-17 13:07:33.484 SeverityText: Debug
2025-01-17 13:07:33.484 Timestamp: 2025-01-17 12:07:28.7385581 +0000 UTC
2025-01-17 13:07:33.484 ObservedTimestamp: 2025-01-17 12:07:28.7385581 +0000 UTC
2025-01-17 13:07:33.484 LogRecord #0
2025-01-17 13:07:33.484 InstrumentationScope Microsoft.AspNetCore.Server.Kestrel.Connections
2025-01-17 13:07:33.484 ScopeLogs SchemaURL:
2025-01-17 13:07:33.484 ScopeLogs #0
2025-01-17 13:07:33.484 -> k8s.cluster.name: Str(d-aks)
2025-01-17 13:07:33.484 -> telemetry.sdk.version: Str(1.11.0)
2025-01-17 13:07:33.484 -> telemetry.sdk.language: Str(dotnet)
2025-01-17 13:07:33.484 -> telemetry.sdk.name: Str(opentelemetry)
2025-01-17 13:07:33.484 -> service.version: Str(17-01-2025.982)
2025-01-17 13:07:33.484 -> service.instance.id: Str(testteam1.testteam1-web-589756bdf5-d8f7m.testteam1-web)
2025-01-17 13:07:33.484 -> k8s.replicaset.name: Str(testteam1-web-589756bdf5)
2025-01-17 13:07:33.484 -> k8s.pod.name: Str(testteam1-web-589756bdf5-d8f7m)
2025-01-17 13:07:33.484 -> k8s.node.name: Str(aks-workerpool2-39622677-vmss000000)
2025-01-17 13:07:33.484 -> k8s.namespace.name: Str(testteam1)
2025-01-17 13:07:33.484 -> k8s.deployment.name: Str(testteam1-web)
2025-01-17 13:07:33.484 -> k8s.container.name: Str(testteam1-web)
2025-01-17 13:07:33.484 -> service.name: Str(testteam1-web)
2025-01-17 13:07:33.484 Resource attributes:
2025-01-17 13:07:33.484 Resource SchemaURL:
2025-01-17 13:07:33.484 2025-01-17T12:07:33.484Z info ResourceLog #0
2025-01-17 13:07:33.482 2025-01-17T12:07:33.482Z info Logs {"kind": "exporter", "data_type": "logs", "name": "debug", "resource logs": 1, "log records": 9}
Sample of part which works fine when querying in Grafana:
2025-01-17 13:08:33.573 {"kind": "exporter", "data_type": "logs", "name": "debug"}
2025-01-17 13:08:33.573 Flags: 0
2025-01-17 13:08:33.573 Span ID:
2025-01-17 13:08:33.573 Trace ID:
2025-01-17 13:08:33.573 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" stopped.)
2025-01-17 13:08:33.573 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:08:33.573 Attributes:
2025-01-17 13:08:33.573 Body: Str(Connection id "0HN9MTGPVHB54" stopped.)
2025-01-17 13:08:33.573 SeverityNumber: Debug(5)
2025-01-17 13:08:33.573 SeverityText: Debug
2025-01-17 13:08:33.573 Timestamp: 2025-01-17 12:08:28.7383097 +0000 UTC
2025-01-17 13:08:33.573 ObservedTimestamp: 2025-01-17 12:08:28.7383097 +0000 UTC
2025-01-17 13:08:33.573 LogRecord #1
2025-01-17 13:08:33.573 Flags: 0
2025-01-17 13:08:33.573 Span ID:
2025-01-17 13:08:33.573 Trace ID:
2025-01-17 13:08:33.573 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:08:33.573 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" disconnecting.)
2025-01-17 13:08:33.573 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:08:33.573 Attributes:
2025-01-17 13:08:33.573 Body: Str(Connection id "0HN9MTGPVHB54" disconnecting.)
2025-01-17 13:08:33.573 SeverityNumber: Debug(5)
2025-01-17 13:08:33.573 SeverityText: Debug
2025-01-17 13:08:33.573 Timestamp: 2025-01-17 12:08:28.738083 +0000 UTC
2025-01-17 13:08:33.573 ObservedTimestamp: 2025-01-17 12:08:28.738083 +0000 UTC
2025-01-17 13:08:33.573 LogRecord #0
2025-01-17 13:08:33.573 InstrumentationScope Microsoft.AspNetCore.Server.Kestrel.Connections
2025-01-17 13:08:33.573 ScopeLogs SchemaURL:
2025-01-17 13:08:33.573 ScopeLogs #1
2025-01-17 13:08:33.573 Flags: 0
2025-01-17 13:08:33.573 Span ID:
2025-01-17 13:08:33.573 Trace ID:
2025-01-17 13:08:33.573 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" sending FIN because: "{Reason}")
2025-01-17 13:08:33.573 -> Reason: Str(The Socket transport's send loop completed gracefully.)
2025-01-17 13:08:33.573 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:08:33.573 Attributes:
2025-01-17 13:08:33.573 Body: Str(Connection id "0HN9MTGPVHB54" sending FIN because: "The Socket transport's send loop completed gracefully.")
2025-01-17 13:08:33.573 SeverityNumber: Debug(5)
2025-01-17 13:08:33.573 SeverityText: Debug
2025-01-17 13:08:33.573 Timestamp: 2025-01-17 12:08:28.7387579 +0000 UTC
2025-01-17 13:08:33.573 ObservedTimestamp: 2025-01-17 12:08:28.7387579 +0000 UTC
2025-01-17 13:08:33.573 LogRecord #1
2025-01-17 13:08:33.573 Flags: 0
2025-01-17 13:08:33.573 Span ID:
2025-01-17 13:08:33.573 Trace ID:
2025-01-17 13:08:33.573 -> {OriginalFormat}: Str(Connection id "{ConnectionId}" received FIN.)
2025-01-17 13:08:33.573 -> ConnectionId: Str(0HN9MTGPVHB54)
2025-01-17 13:08:33.573 Attributes:
2025-01-17 13:08:33.573 Body: Str(Connection id "0HN9MTGPVHB54" received FIN.)
2025-01-17 13:08:33.573 SeverityNumber: Debug(5)
2025-01-17 13:08:33.573 SeverityText: Debug
2025-01-17 13:08:33.573 Timestamp: 2025-01-17 12:08:28.7376904 +0000 UTC
2025-01-17 13:08:33.573 ObservedTimestamp: 2025-01-17 12:08:28.7376904 +0000 UTC
2025-01-17 13:08:33.573 LogRecord #0
2025-01-17 13:08:33.573 InstrumentationScope Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets
2025-01-17 13:08:33.573 ScopeLogs SchemaURL:
2025-01-17 13:08:33.573 ScopeLogs #0
2025-01-17 13:08:33.573 -> k8s.cluster.name: Str(d-aks)
2025-01-17 13:08:33.573 -> telemetry.sdk.version: Str(1.11.0)
2025-01-17 13:08:33.573 -> telemetry.sdk.language: Str(dotnet)
2025-01-17 13:08:33.573 -> telemetry.sdk.name: Str(opentelemetry)
2025-01-17 13:08:33.573 -> service.version: Str(17-01-2025.982)
2025-01-17 13:08:33.573 -> service.instance.id: Str(testteam1.testteam1-web-589756bdf5-d8f7m.testteam1-web)
2025-01-17 13:08:33.573 -> k8s.replicaset.name: Str(testteam1-web-589756bdf5)
2025-01-17 13:08:33.573 -> k8s.pod.name: Str(testteam1-web-589756bdf5-d8f7m)
2025-01-17 13:08:33.573 -> k8s.node.name: Str(aks-workerpool2-39622677-vmss000000)
2025-01-17 13:08:33.573 -> k8s.namespace.name: Str(testteam1)
2025-01-17 13:08:33.573 -> k8s.deployment.name: Str(testteam1-web)
2025-01-17 13:08:33.573 -> k8s.container.name: Str(testteam1-web)
2025-01-17 13:08:33.573 -> service.name: Str(testteam1-web)
2025-01-17 13:08:33.573 Resource attributes:
2025-01-17 13:08:33.573 Resource SchemaURL:
2025-01-17 13:08:33.573 2025-01-17T12:08:33.573Z info ResourceLog #0
2025-01-17 13:08:33.573 2025-01-17T12:08:33.572Z info Logs {"kind": "exporter", "data_type": "logs", "name": "debug", "resource logs": 1, "log records": 4}