I have hunted deer exclusively with a handgun for 12 years now. I started with .44 Blackhawk, then Super Redhawk. Then I went with a .44 Desert Eagle and eventually upgraded to the .50AE. Ended up selling that rig for a .460 S&W. Poor choice (more on this later). Last year, after talking to a lot of other handgun hunters, I purchased a Thompson Center Pro Hunter in 7mm-08. This gun works awesome for me. Compared to the handguns that I previously mentioned, this rig is a Cadillac. I have it mounted with a Leupold variable 2.5 -8X handgun scope. It has a 14 inch barrel with muzzle brake. From a sandbag, I can shoot one inch groups at 100yds pretty consistently. I killed 5 deer (4 does and one decent buck) last season with it. The closest was 30 yards and the farthest was 175yds. That being said, if you are really serious about hunting deer with a handgun, consider the T/C in a rifle cartridge. The Rem XP and Savage Striker are good guns too, but they are no longer produced and therefore hard to find. One of these setups firing a rifle cartridge will give better accuracy and flatter trajectory than your revolver cartridges. If you reload....all the better.
Now to the .460 that someone in a previous post mentioned. I had one. I did not like the gun for these reasons: 1) Barrel life on these guns is 1200 to 1400 rounds with factory loads. If you don't believe me, just call S&W and they will tell you. You can, of course, reload a much "lighter" load and increase the barrel life. If that's what you opt to do, then just go with a .454. I owned mine only 11 months and had to have the barrel replaced. 2) The gun "coppers up" very badly. I would literally put 10 rounds through the gun and it would be heavily coppered (both reloads and factory loads). Then I would spend HOURS cleaning it. 3) Compared to my 7mm-08 Thompson that I have now, that .460 was a bear to shoot. Muzzle blast was much more extreme. I once leaned it against a tree to take a shot at a deer. The result was that I got my face and eyes full of bark and splinters from the cylinder-gap blast. Forgot my safety glasses on that day. IMHO, the .460 is a "white elephant."
I know this is a long post, but I hope this helps you.