I removed the flags on my two carry guns and stowed them away. The lock hole is intact and unless you know what you are looking for, you would not know the difference.
I would stay away from grinding the flag and reinstalling it, there is no reason to do such a thing.
Or you could remove it & get the "plug". That's my plan. I wouldn't want an open hole there.