If it was me, and you said you checked with the supervisor. But, many county ROW's are measured so many feet from the center of the road out towards the shoulder, both directions. Maybe the measurement, if it exists, is taken starting from the markers. There must be a document on record at the county court house that states it in writing if that's the case. Might be worth some time checking into it.
All that should be specified in the property deed.
They can't just build a public road on private property.....Have the guy put it in writing on an official letterhead that you are the sole owner of the property and responsible for the property. Then, when you close your road to remove the tree and the county officials show up to find out why their road is closed, you can hand them the bill...
If they are claiming access rights for maintenance purposes, they are responsible for maintaining the road. That maintenance includes hazardous trees. Have a hazardous tree evaluation done and have a copy sent by certified mail to whoever maintains the road. That way they either will do something to avoid future liability exposure, or, if they do nothing, you can prove in court that they knew there was a problem if it falls. I wouldn't touch that tree without documentation from the county stating that you are the owner of the property, and have full authority to act on your own behalf as the owner to do whatever is necessary to remove the tree safely, including closing the road.
Take a close look at whatever markers are in the road. If they are permanent geodetic type brass pins, they should have some sort of identification stamped into them. Municipalities don't generally have those placed for private property. They are there to identify the center line of the municipal right-of-way, and there should be records that specify the distance from the centerline of the roadway to the edge of the right-of-way. Generally there is no set distance, just so many feet from point A. It could be 100 feet, it could be 1 foot, depending on how the property was parceled out and when the road was built. The town, county or state can't place guardrail installations on private property for liability reasons, they are required to be in the public right-of-way. So, being that the tree is right in front of the guard rail installation, its in their right-of-way, whether they like it or not.
By the way, that is an illegal guardrail installation. The guard rail should pass to the outside of the tree, so any car that might strike it would be directed around the tree, avoiding a potentially serious impact. Also, Federal standards mandate a proper attenuator unit be attached to any elevated termination of a guardrail installation to prevent the end of the rail from impaling any vehicle that might strike it head on.