Are you transferring few large files, or thousands of small files? The bookkeeping of creating thousands of files often overwhelms the time taken to actually transfer the data, and that’s an issue that no amount of USB throughput speed can really solve.
Sometimes people think, “well I transferred 2 GB last week and it was really fast, now I’m transferring 2 GB today and it’s really slow” without pausing to consider that last week it was a single large .zip file and today it’s 24,000 small text files.
(Although increasing the drive’s seek time would help. Since you’re talking about a USB drive, it’s probably 5400 RPM or at best 7200 RPM. I’m sure a SSD would have no problem running at full USB2 speeds, even with thousands of tiny files.)