Table vs Timeseries visualization

We are developing a datasource plugin and we are planning to support Timeseries and Table format for visualizing data. If I understand it correctly, the table format JSON can only be understood by Table panel and single stat. My question is , in the Add query screen, if user has already selected Table Format, during visualization component selection how do I restrict it to show only applicable components(panels).?For example Graph, Gauge , Progress Bar etc wont work with Table JSON format so what is the point in showing non-applicable panels for the user to choose?