Windows and Linux both typically allow USB audio devices to override any built in devices. That's been default behavior since at least Windows 7.
Specific software limitations aside, when you insert the USB Audio device, what does the tray icon for sound tell you? Is the audio device showing as USB? If it is, that means Windows has defaulted to the USB device, which is good. The next part will be to determine why it doesn't work.
The majority of USB audio devices don't use or need drivers, as they typically use chips that have been around for ages. (Some high end devices or mixer boards may require drivers, but those are a far cry from most simple USB headsets.)
It sounds as if there is another software issue in Windows preventing USB audio. If specific drivers are installed for the built in audio, try uninstalling them, as Windows has a fairly good ability to get basic audio drivers working, and you may find after that USB audio works.
Realtek usually doesn't have issues. Connexant typically is full of issues and you are better off using the generic or Windows Update version of the driver. Intel used to have audio chips, but if I recall, they were rebadged Connexant chips. IDT is another audio chip supplier, both them and Connexant both are weird, typically seen in prebuilt machines. I prefer Realtek.
Oh, and if this machine was an upgrade to 10, all the more reason to ditch the drivers as sometimes it's still remnants of the Windows 7/8 versions which can and do cause issues.
