Jim, are you making collision detection calculations in devicespace, viewspace, or worldspace?
Worldspace calculations would work with both your current planar background as well as your future 3D background.
There are also a number of D3D api functions to help with volumetric calculations (such as mentioned above) as well as transformations which could be used to rotate the lionfish's dorsal fin triangles (treated as a whole) around an "articulated joint" vertex, providing you with a mechanism to allow the fish to proceed with its turn "between" the coral while "folding back" its dorsal fins against its body.
Just a thought from a monkey.
