sessions get’s renewed whenever a request is made, so if you have Grafana on TV wall or open in an active tab with a dashboard with refresh it will never expire.
This is not how session cookies work, session cookies (cookies marked session are treated in a specific way in browsers), they usually only expire when you close the browser. The backend session linked to the session id will expire if no requests are made during the lifetime. So if you leave grafana open with no activity (no dashboard panel refresh) for x time period (same as session lifetime) and your login_rember_days is zero then you should get logged out.
After your session config changes make sure you clear cookies. Could be good to make sure the test is valid. If you still see that you do not get logged out after not using Grafana for x time period then it could be something wrong indeed.