At the end of the day perhaps the most insight we'll get into why the model is saying what it does will be to ask it! Far from ideal of course, and no better than asking a person why they said/did something (which is often an after-the-fact guess). However, at least any such explanation may be using the same internal model/reasoning as what generated the speech in the first place, so conversational probing may support some sort of triangulation into what was behind it!