Wild assed guess: the cold slows down the chemical reactions in the flesh of the banana but cannot save the skin. Putting bananas in a bag makes them ripen faster, and a fridge is just a larger enclosed space.