You didn't list the bore diameter or the diameter of the cast bullets or their velocity your shooting but, my guess is the hardness of the cast bullets your shooting is either too hard or too soft?
Steve