Hi,
A usb port should be able provide 500ma, though if you are using a laptop it might not be able to do that much.
Better to prove things using a 1A 5v wall power pack.
Two points for your test, when you turn on the relay are the contacts connected to any load; that is, as mentioned eariler is it the switching of a mains load across the contacts that is causing the problem ?
You do not give any details of the relay being used, its coil resistance and working voltage / amperage.
Also consider that, usb voltage can be as low as 4.75v, that is then used to power the relay coil , switched by the transistor you use, which causes a voltage drop of 0.7v so you could have as little a 4.0v across the relay which may be too low to energize it properly.