So now its hooked up with Jumper wires, and not plugged in directly?
If yes, did you connect the two grounds on pin 25 and 39?
If you use this,
with a stacking header
https://shop.pimoroni.com/?q=11mm
You can plug it into the Pi Zero, and your pirate audio in on top of that.
Then wire your relay etc to the proto zero. I used one with my Pirate Radio to add my own bigger buttons.