I am not familiar with AWS Athena syntax but you want a list of numbers table you can use to left join into your real table. The following is in SQL Server. So your question is really more AWS athena than it is grafana unless there is a way of filling gaps in grafana.
;WITH cteN AS
(
select 1 as Number UNION
select 2 UNION
select 3 UNION
select 4 UNION
select 5 UNION
select 6 UNION
select 7 UNION
select 8 UNION
select 9 UNION
select 10 UNION
select 11 UNION
select 12
), ctesum as (
SELECT month(ct.period_start_date) as Mesi,
sum(ct.line_item_cost) AS cost
from CUR_Table ct
where user_cliente = 'MCS'
--and $__timeFilter(bill_billing_period_start_date)
group by month(period_start_date)
)
select n.Number as month, isnull(c.cost,0.00)
from cteN n
left join ctesum c on n.Number = c.Mesi
It generates the following barchart
histogram
