While I have a couple of .204s and several .223s,,, If I were restricted to one caliber for all those chores, the nod would have to go to a .223 as it will shoot the light weight bullets at a pretty good clip for better than average trajectories, and handle the heavier bullets for distance and larger targets...with good results...
I love my .204s, but they do have their limitations and some try to push past that and then complain about performance..
With my .223s, I can adjust my loads for the purpose intended and still keep a reasonable amount of accuracy, or fine tune them for maximum accuracy....204s are much more finicky about accurate loads...
And if you can't hit your target, why are you pulling the trigger..???