I load for 4 different 204's, none of them mine. My fast gun is a 22-250 and I shoot 50, 52, and 55 weight bullets in it. I load 70's and up in my 243 for the windy days. Shot 70NBT's at 3,385 fps for the longest time, and I love the Sierra 80 Blitz for ground-hogs but I've shot a lot of 85 HPBT's and now I'm working up something for the 87 V-MAX's for this season.
I've seen 204's shoot great right out of the box, and then I've seen some so finicky they couldn't shoot ANY factory ammo worth a darn. And if you get unlucky and can only shoot 32's, well... you might find you would have been better off with 50-55's on a windy day. But the 204 can be made to be extremely accurate and I do think it's a great round. I just stick to the classics myself. Good CZ 527 in 22 hornet as my main fur gun, a Rem 700 BDL in 222, a Savage Predator Hunter Max 1 in 22-250, and the Tikka T3 Lite then in 243win. Oh yeah and a CZ 453 Varmint with a 5oz set trigger in 17hmr for the really light stuff and for plinking. Those are pretty much my varmint brothers. I do also have an old 1917 Savage Sporter in 25-20 which was the predecessor of the 22 hornet, where the hornet was replaced by the 222 - which is still arguably the most accurate and reloader-friendly of the bunch!
They are all great. I actually bought my 243 first, thinking it could cover as a 22-250 with smaller 55, 58 grain bullets and then still be an effective deer rifle for crop damage work. But then I still bought a 22-250. Sort of like on the low end, had I bought the 22 hornet first, I would have probably never bought the 17hmr - but then again... I'm glad I did, and may have anyway. They all have their place just depending on what you want to shoot.