I changed all the media.autoplay settings from the default except block-event.enabled, with default set to 1. With that it seems to do what I want. I'm not sure if the strange media.block-autoplay-until-in-foreground behavior described in the comment is still happening but I set that to false also.
https://utcc.utoronto.ca/~cks/space/blog/web/FirefoxMediaAut...
I changed all the media.autoplay settings from the default except block-event.enabled, with default set to 1. With that it seems to do what I want. I'm not sure if the strange media.block-autoplay-until-in-foreground behavior described in the comment is still happening but I set that to false also.