I know of a guy that had a problem with that. He was close to a hose so it wasn't a big deal. I know I might sound like a weenie, but better to be safe than sorry here. The fine dust that you make when milling (powder when I'm cutting hard locust lately) is a lot more combustible than regular chips. Any time you increase the surface area of the combustible material, you increase the possibility of flash-fire. Think coal mine explosions here. I'm not trying to be overdramatic, but it COULD happen, even if only once in a very great while.
I had mine out then replaced it when I modded the muffler, and heard about a local guy that lit his sawdust on fire. Saw still has a fair amount more than stock. I'm afraid if I remove it, it wont run right b/c of lack of backpressure--I have an older non-EPA 394XP that has a pretty big port to begin with.