Odd network connection problems

I am having an odd connection problem. I have 3 computers wired to a router & a laptop connecting (sometimes) wireless.

When my XP Home SP2 comp sits idle or if I use other programs that connect to the internet, it won't let IE connect. I use the connection diagnostic;sometimes it fixes it & can connect. Other times it will say there is something wrong with the router or couldn't detect a connection but I can connect with Firefox or some online games. Heres the weird part: one of the games uses IE as part of its interface & I can use IE through the game even after IE has told me it couldn't connect.

The WinMe comp use to have connection problems from sitting idle but since I connected it through the Vonage internet phone router, that problem went away.

My Frankenstein comp (named this because I put it together from free spare parts) runs XP pr0 SP1. I have not noticed any problems from sitting idle but if I use any other program that connects to the internet, IE will no longer. I don't have Firefox on it so I can't try that.

Rebooting the computers usually restores the connection. The times it doesn't, powering the router off/on will fix it.

The laptop's wireless connection is spotty at best. I am not completely positive when it loses connection. My neighbour has an unsecure wireless router & whenever the laptop would lose connection with my router it would automatically connect to his. I have since marked his connection as "on demand" so I can catch when mine is lost. Which so far seems to be everytime I power up & try to connect. Only thing that seems to fix this is power off/on the router. Rebooting doesn't seem to work.

I am thinking that IE doesn't like my router for some reason but have not totally ruled out that the router doesn't like IE. I was wondering if I assigned each comp with a fixed IP rather than letting it obtain one automatically, if that might cure the problem.

Any suggestions?