Is it possible to extract fields that cannot be labelled (due to high cardinality) into alert notifications?

Hi,

I understand that we’re not supposed to create labels for fields with high cardinality (e.g. source IP, or timestamps etc.);

Is it possible to somehow extract the information at alerting time and include the information in the alert notifications?

E.g. say the log looks like this
Jan 01 00:00:00 2024 level=“error”, source_ip=“1.2.3.4”, event=“auth failure”

in the alert we have
expr: count_over_time({job=xxx} |= auth failure [5m]) > 0
labels:
event: “Authentication Failure”
annotations:
message: {{$labels.event}} from IP {{$???}}

is there a way to extract the “1.2.3.4” into the message ({{$???}} portion), and send it to the contact point as part of the annotation or something, so that I can see the IP in the alert?

Thank you very much!