Home networking issue. Please help. (LONG)

First a little background about my setup. Sorry, I am not real tech savvy. I have 2 PCs. One downstairs and one upstairs. Both are running Win2k Pro. Computer 1 downstairs has a wired linksys ethernet card installed with the correct 2000 drivers. It's working properly according to device manager. Computer 2 upstairs has a linksys wireless-G 2.4GHz ethernet card installed. It is also being reported as working properly. I just signed up for DSL service and received a free 2-in-1 Westell Versalink modem and router in one with wireless capability. The modem/router is downstairs. I have the downstairs computer wired directly into an ethernet port on the back of the modem/router with a standard cable that was provided. The upstairs computer is obviously connected through the wireless adapter card.

Now the issue. Both computers connect just fine to the internet. No problems. The problem is that I want to share resources between the computers, but I can't get them to talk. The upstairs computer seems to be ok. I can ping sites and the IP on the machine itself and receive replies. I can even get onto the network group I created on microsoft networks and put both computers in the group. I gave both computers distinct names and made sure they were both in the same workgroup. I even set up username and passwords identical on each so that I could connect to each other. I go into the workgroup and can see the computer I'm on upstairs, and I can even see the computer downstairs. When I click on the icon for the upstairs computer, I can even access the folders I decided to share. But I can not ping the downstairs computer. Also, when I go to click on it's icon in the network group I created, I can't access it. It says that the path is not found. So I decided to go and test the computer downstairs to see if I could find any issues. I could access the internet downstairs. But I could not communicate with anything. I tried ping. I could not ping myself downstairs. I could not even ping the default gateway, the computer upstairs by IP or name and I couldn't ping any valid websites. I could ping one thing only that I read about online. I pinged some basic IP that I read about that is 127.0.0.1 I believe. I actually got a reply from that. But that is the only thing I get a reply from. I can see the workgroup I created under microsoft networks in networking, but I can not even access it to see the 2 computers in it. It says something about servers not specified or something. I checked firewalls. I have zone alarm, but it isn't running. Doesn't it have to be running to affect the computer? I don't know how to check to see if Win2K itself has a firewall for internet or incoming traffic. I looked at the router settings and the firewall says that it is allowing all traffic. And I don't know how to check if pinging has been disabled as someone mentioned to me. I checked all properties on the adapter cards on both computers and could find no option about enabling or disabling internet firewall protection. So I don't think that is the issue either. Sorry for all the info. I am just desperate to find someone who has an answer on how to get my computers to connect on my home network. Thanks again in advance. Any help is greatly appreciated. (Both computers have the same subnet mask and default gateway. I tried IP release and renew on the downstairs computer, and both commands worked just fine on it. I don't know what the problem is?)