The most likely is that the default BCM2835 module got loaded back in, I guess.
In theory, although I haven't tried it, it should work with the custom kernel, since it uses external overlays and that's the whole point of the device tree system as far as I understand.
... I'll give that a go tomorrow but if you feel like experimenting just run the phatdac installer AFTER setting up the USB gadget thinghy the way you want it.