Tecnically with just a kernel optimized for your CPU, realtime patches, NTSync and a custom MESA build (with -O2 and -march set to your CPU) would give a good boost instead of trying to recompile verything.