Even assuming this does have a measurable performance effect for the kind of processors you run Linux on (as opposed to something like a Cortex-M), you only need to have load-big-endian and store-big-endian instructions.