The M4 is normally mounted forward of the rear sight such that two sights aren't close to having physical contact. And, with the mount that comes on the M4, you are 1/3 co-witness. Works very well for me. Don't know why you'd want to remove the rear sight unless you think it looks better in which case I'd just remove both front and rear sights.