Turns out the hammer is bent/not level. If you lay it flat on a table you can press on where the pin goes in, and see the top half rise up. Gotta hunt for another hammer I guess.