About 90%+ of the time for me when a longer bar isn't cutting straight, it's the teeth that need to be sharpened correctly. There are 2 angles that need to be angled correctly, but there is something else many don't quite get and that's the depth of the cut made in the tooth. I try to get all 3 equations correctly and then those long bar cutting quirks go away. On a few occasions it was a bent bar, then even so the saw still cut fair in comparing using a chain that had teeth that needed to be sharpened correctly.
Since you say the problem exists only after a few hours of cutting smaller brush, maybe another fix is to tighten the chain slightly as a lose chain is more vulnerable to not cutting straight when there is side torque applied as the operator is cutting. As others have listed, letting the saw cut instead of directing helps. You can steer a cut too, but then not as well, if at all, when the chain is loose.
Bent bar is another more obvious thing to check. You may also need to true up the bar's levelness where the chain rides on. Don't know the price of the tool, but recall it's fairly cheap; $23 or so?
Good luck on solving that problem. Would be nice to hear later when you get the riddle solved.
ePhoenix