ESP IoT pHAT problems

another thing that comes to mind though is: how are you powering your Pi Zero? if you are starving your Pi+ESP then it's quite possible that some pins state can't quite be set 'high'. This would certainly lead to a bunch of issues.