Acrylic is a good material...real cast acrylic that is, not the stuff you see on ebay that people are calling acrylic (although it will work also).
Not sure what you meant about delrin, its good toneboard material, that you can work by hand like everything else?
Nothing wrong with wood, even exotic. Its good to be concerned but Ive never really heard of anyone getting sick. I might stay away from the rosewoods..(cocboolo etc) just because it seems to be the worst for people. There is a well known crow call maker that uses coco as his standard material for toneboards...they boil them in water and it supposedly renders them inert, and they wont warp...says him, I dont really know. Really.... something like maple, osage etc are good. Purple heart is a great exotic.
I have used many materials, lots of plastics, and I like them all. Some are easier to work than others. PVC is super easy. Nylon 6/6 is a favorite but kinda tough to work. Delrin is pretty easy. You can also cast something like Alumilite resin and work with that. Cast acrylic is crisp and loud, but may be a little tough to work with depending on your tools.