Slack mentions not working

I’m trying to get Slack mentions working. We have Slack set up as a notification channel fine, it posts alerts into various channels with various Webhooks etc, and that all works well.

However, mentions just don’t seem to work. I have tried putting username @username, trying to find the internal Slack user ID etc, no luck. Whatever text I put in the field is included in with the message posted to Slack, but maybe its escaped or something, and Slack does not treat it as a proper Mention and does not highlight it or notify the mentioned user.

Am I missing something?