Dirt bike exhaust cleaning?

whenever i get my exhaust wet and muddy it seems to get a rusty kind of layer on the pipe coming just out of the engine. i have taken it apart and tried to use a wash rag and some water but that doesnt work there. its the matte black paint that honda has on their stock 230s. i dont want to scratch it off with a wire brush and mess up the paintish ****