Unfortunately it is quite a common issue, that often occur with USB3 ports, see here: https://learn.adafruit.com/introducing-trinket/faq
What computer are you using? If you have an external USB2 hub available, preferably one that is powered externally, try that as a possible solution.