I haven't had much luck substituting the metric allens for the tiny SAE ones - the ones that fit always slip on me, and usually trash the fastener.
On the other hand, once the screw is loosened, the last time I took an M&P rear sight off I used a table instead of a vice. Left some slide marks, and the sight wasn't a super-tight one to begin with, but it worked fine. I believe it's left to right removing, right to left installing. If you can't mark up a table, a hockey puck makes a nice work surface.