The first cartridge I started reloading for in 1972 was 9X19mm, and the first bullet mold I acquired was for that caliber in 125-grain round-nose. I have loaded and fired many thousands of cast bullet 9mm loads since then, and using a dozen different pistols to do so. I have cast bullets using recovered range bullets (composition totally unknown), wheel weights (BHN ~12), up to pure linotype metal (BHN ~22), all bare (unplated) and lubed with NRA Alox formula.
Lots of variations in bore & groove diameters, as well as chamber dimensions. The finished cartridge must fit the magazine, must feed properly, and must chamber easily. Beyond those basics it is all about establishing a load that will reliably cycle the action of the pistol in use.
My Lyman manual shows a starting load of Bullseye at 3.7 grains and a max at 4.4 grains, with other published sources recommending similar range. I suspect that the 3.5 grain load is on the low side, might be contributing to the problem reported, and could cause other problems in some pistols.
One factor that seems to be overlooked by many folks is the need for complete removal of all traces of jacketed bullet residue in the bore and rifling. Nothing will cause problems with cast bullet performance more than residual copper and gilding metal. Pistols that have been heavily used with jacketed ammo may require some effort in cleaning the bore before cast bullets can be expected to perform well.
My recommendations would be:
1. Start with a thorough cleaning using solvent capable of removing copper jacket residue. This may require two or three good cleanings, depending on what you have to deal with. Residual copper residue will show itself as a greenish stain on the patches after soaking the barrel and scrubbing with a bore brush. We want our final cleaning patches to come out perfectly clean.
2. Begin with the starting load to evaluate performance, moving upward as necessary to assure reliable functioning. Ten rounds or so at each load level should be enough to accomplish this.
3. Once reliable function has been reached start testing for accuracy. Best performance will usually be found at or about 5% below book maximum load.
4. If Bullseye doesn't cut it for your pistol you might want to try something more in the middle of the burning range like AA #5 or Unique. My personal favorite for years was Blue Dot, but Unique has provided very good performance in 9mm cast bullet loads also.