Manually Deleting Driver files

I work with a program that uses VCP (virtual comp port) USB drivers. On occasion, I find the need to uninstall the drivers to get the USB Serial converter to work correctly. Sometimes, I can't get the driver to uninstall using "Uninstall" in the device manager (It just reinstalls itself upon reboot) I found the .sys and the .dll files that I can delete to get it to work, but I was wondering if there is a registry key for it, and generally which group would I find a drivers at?