Well don’t hold me to this but here’s the way I understand it.
I would think that the phantom power addresses the XLR inputs and not the TRS. The risk of damaging anything with phantom power would still be pretty low, as I know even working dynamic mics can easily withstand the 48V of phantom power without any damage.