I made it work via using the following template line in the Severity field of a Grafana -> Contact Point -> PagerDuty settings instead of using Notification Templates.
{{- range .Alerts -}}{{or .Labels.severity "critical" -}}{{- end -}}
DON’T FORGET to change the escalation policy for the Grafana service on the PageDuty side.
I hope it helps someone.