It never really ends all at once for me. I light up more sporadically as we get near the end of the season, but the occasional fires continue into early May. It's 22 here this morning and overcast, so I'll keep one going all day. I'm so fat on wood right now that there is no inclination to conserve it, so if it's even a little chilly, I'm lighting up.
At the milder end of the season temps, a sunny vs cloudy day is often the difference between lighting up or not. If the sun is out all day, then there is enough solar gain in the house that a fire is not necessary, while an overcast day at the same outside temp will warrant a fire.