The Hanging Man is not really in the forest, but it's in a strip of neutral land and does not owe allegiance to anything. It's a pub and an inn for any wayfarer, and travelers use it as a meeting point. Or as a place to get drunk and bathe after coming out of the Forever Forest.