A skinny rubber band wouldn't interfere...but I may see about either replacing the spring or putting something behind it to give it more tension.

If that fails..maybe file the notch a bit deeper..it's not notched all that deeply anyhow..and the latch piece will protrude from the lever enough to take advantage of a deeper notch.