starts second time

I've got a pc that won't reboot, when I'm installing something and I need to reboot, the pc doesn't start up again, I've got to switch the power off and on even then it doesn't start, it only ever starts after switching it of and on twice, It's the same in the morning when I switch it on it never starts first time I've always got to switch it back of with the button at the front then switch it back on before it starts. It always starts second time everytime but never on the first power up. Does anybody know what could be causing this? I don't think it software related cause the only thing in the pc is XP and a few antispy programs, it also done this when I got it with ME installed and now after it's been formated and a new os installed it still does it, which is why I thing it's more hardware related, anybody got any ideas?
Thanks,
Fangy.

 

change the power supply

 

Thanks I'll give it a try.