Pinyin is very well-adapted for (Mandarin) Chinese. It isn't specifically designed for English learners, but if you understand phonetics and learn the sounds first, that shouldn't matter. The alphabet is originally from Latin and is used in a variety of ways in different languages, e.g. Turkish c sounds like English j, German j sounds like English y, so the foreign accent problem exists generally.
There are two bilabial stop consonants, called p and b, in many different languages, with different sounds, e.g.
Mandarin English French
unaspirated voiced - b b
aspirated voiced - - -
unaspirated unvoiced b - p
aspirated unvoiced p p -
though there is, as you point out, an unaspirated unvoiced allophone of p in "spade". Other stops (t, d, k, g) follow a similar pattern.
English speakers using English p and b in Mandarin will still be understood even if their b sounds foreign, but French speakers might not be, because French p sounds like a Mandarin b.
The only change I'd like is replacing Pinyin -ong with -ung.
I think it would also help to expand -iu to -iou and -ui to -uei, and of course u->ü where applicable. IMEs could still accept both forms.
Pinyin is pretty well-adapted for entering Chinese, but I think that goal is at odds with being easy to pronounce. I'm not sure that having one ISO standard for both purposes is a good idea, although if China ever starts to export more culture, maybe everyone will eventually learn how to pronounce Hanyu Pinyin.
Choosing aspirated unvoiced consonants as "typical" of English is very weird. Native English speakers do not consciously distinguish aspiration but they do distinguish voicedness so it seems it would be more accurate to call the aspirated form the allophone (aspiration only occurs at the beginning of words and stressed syllables, and never after "s" as you point out, so the unaspirated form probably occurs more frequently too).
Still, I agree with your main point. The Latin alphabet is not used like the IPA by most languages, even English has not preserved all the original Latin sounds as used by the Romans. And there's even precedent for some of the sound choices made in Pinyin that would seem completely alien to English speakers. For example "c" is used in all the Slavic Latin alphabets (e.g. Polish, Czech, etc) for the unaspirated version of the Pinyin "c" sound. German and Pinyin use the letter "z" for the same sound, etc.
There are two bilabial stop consonants, called p and b, in many different languages, with different sounds, e.g.
though there is, as you point out, an unaspirated unvoiced allophone of p in "spade". Other stops (t, d, k, g) follow a similar pattern.English speakers using English p and b in Mandarin will still be understood even if their b sounds foreign, but French speakers might not be, because French p sounds like a Mandarin b.
The only change I'd like is replacing Pinyin -ong with -ung.