It should generate dashboard titles, descriptions, and change summaries, it work on 12.3 but failed on 12.4, LLM plugin version 1.0.6, Also even retrieving FlameGraph data from Pyroscope is failing.
ref. LLM: Dashboard/Panel Title/Description generation is not working after update to 12.4 · Issue #119781 · grafana/grafana · GitHub