Due to less than required chimney height and distance from surrounding structures that you describe, for all practical purposes, your pipe is like in the middle of a canyon.
Particularly with any wind at all, I can understand why you have to "line up the sun and moon and stars" to get an updraft started.
I would concentrate on correcting chimney pipe deficiencies, especially in light of a "vocal" neighbor, and when corrected, your starting fire procedures will be much closer to normal
