Tecumseh, you don't state which rifle you own, but that cartridge should do much better than that... I have a friend in Wisconsin with a Tikka 300 WSM. Several years ago I loaded him seven boxes using WW brass, Fed 215 primers, and a powder charge of DP 85 (from Accurate; supposed to be the same burn rate as their Western Powder Hunter). I used 165 GS projectiles and one grain lower than their Hunter max charge, and his first attempt with that load was three shots into half MOA at 100. I was pretty tickled considering it was all guess work and the rifle was 1350 miles away.
I would keep working at it. I have no idea what your powder charge is, but you might try a magnum LR primer if you are over 60 grains, and I have to believe you are.
I'm shooting a Weatherby Vanguard with a Timney trigger. That load was using 67 gr of RL-19 with a Federal large rifle prime and Norma brass. It printed a 2 inch group with 66.5 gr of RL-19, it's just a matter of dialing in the powder charge and I'll have it shooting sub moa with a couple more range sessions.