Just had the same problem.
In my case, I found that Grafana could not write into the /var/lib/grafana/plugins directory, as the ownership (after adding in other plugins before the upgrade) was not grafana:grafana.
Changed permissions of the folder accordingly, performed an install of the Elasticsearch plugin (via the UI), and restarted the grafana-server service (existing Elastic datasources were not found until this occurred).
I fixed the issue by simply mounting an empty directory to /usr/share/grafana/data/plugins-bundled/elasticsearch, which means that the started container no longer has a bundled elasticsearch plugin. Then I added elasticsearch to the plugins to be installed, and I have the newest version installed without Grafana every trying to uninstall an old version.