I had some fun adding USB gamepad support to my Space Invaders game and thought others might enjoy it. I used built-in Linux drivers. I tested with a Nintendo Switch USB controller (D-pad to move, any button to fire.) Some of the code might need to be adjusted for other controllers; there are some comments in that section that might help.
The basic idea is to hackily extend the intc module to be able to also check the gamepad device and return a fake interrupt flag for it, though Linux is itself handling whatever real interrupts are associated with USB. When that flag is set, I read from the device to see what the input is, and pass it to the rest of the game.
Here are some snippets in diff
format from git, which probably won't fit your files exactly, but should give an idea of what I changed.
diff --git a/userspace/apps/space_invaders/CMakeLists.txt b/userspace/apps/space_invaders/CMakeLists.txt