I have killed a BUNCH of deer with the 223 (mostly longer barreled bolt guns). If you make & take good shots (broadside, behind the shoulder bow style shots) the 55gr vmax will work just fine; been there done that. If you need more penetration the 65gr gameking, 60 partition or any of the Barnes x bullets will be more helpful (even in the shoulders). The 55gr sierra BTHP & the 60gr Varminter are wonderful deer bullets & overall much better than the vmax. Keep shots inside ranges where you KNOW you can hit in a softball size area and you'll have no problems.
Not sure what you mean your scope is "calibrated" to the 55 vmax??? If you mean "sighted in for it" thats a minor change & you shouldn't have a problem with a short trip to the range. If you mean the reticle, BDC, turret or some type of drop correction is calabrated to it... Throw that out the window, no sense in shooting deer past point blank range (inside 300 yards) with a 223Rem & 55gr bullet.
If you can't wait for good shots, can't pass opportunities, & plan to take raking angle shots... Save up for a bigger gun or upper.