Possible network card issues

Hey guys,

The past few weeks I've been having really unreliable internet. Speed are supposed to be >50Mbps, but fluctuate between 1-4 Mbps (sometimes for a few min, sometimes for a few days) and then return to normal but not for long. I replaced my modem (twice) and had two technicians come out from comcast who assured me that everything is running perfectly fine. When I showed them the speeds on the desktop it confirmed my complaint, however any other device they used had a perfect connection. Additionally my girlfriend brought her laptop over and it worked fine wired and wireless.

Therefore I now looking at issues with my computer, and I noticed the LED lights at the Ethernet port are Solid Amber, and flashing yellow. My drivers are all up to date and it is a new install of windows. Does anyone have any suggestions?

I have an MSI FM2-A75MA-E35 MOBO, AMD A10 Trinity APU, SSD, 8GB RAM.
My computer is plugged directly into the modem, I tried several different Ethernet cables just to be sure.

Any help is appreciated, thanks!