It should fit, but some say you maybe need to grind a little, othervise the brakeband will drag to the Clutch! BTW On the UK Ebay there has been a complete brake, with plastic handle, that was stated to fit the 181!
HUSQVARNA 181 se 181se CHAIN BRAKE SPROCKET - CLUTCH | eBay