I started out writing machine code without an assembler and so had to hand assemble a lot of stuff. After a while you end up just knowing the common codes and can write your program directly. This was also useful because it was possible to write or modify programs directly through an interface sometimes called a "front panel" where you could change individual bytes in memory.

Back in 1985 I did some hand-coding like this because I didn't have access to an assembler: https://blog.jgc.org/2013/04/how-i-coded-in-1985.html and I typed the whole program in through the keypad.

Same here. On/For the ZX Spectrum, looking up the hex-codes in the back of the orange book. At least it was spiral-bound to make it easier.

Later still I'd be patching binaries to ensure their serial-checks passed, on Intel.

I had a similar experience of writing machine code for Z80-based computers (Amstrad CPC) in the 90's, as a teenager. I didn't have an assembler so I manually converted mnemonics to hex. I still remember a few opcodes: CD for CALL, C9 for RET, 01 for LD BC, 21 for LD HL... Needless to say, the process was tedious and error-prone. Calculating relative jumps was a pain. So was keeping track of offsets and addresses of variables and jump targets. I tended to insert nops to avoid having to recalculate everything in case I needed to modify some code... I can't say I miss these times.

I'm quite sure none of my friends knew any CPU opcode; however, people usually remembered a few phone numbers.

The instruction sets were a lot simpler at the time. The 8080 instruction set listing is only a few pages, and some of that is instructions you rarely use like RRC and DAA. The operand fields are always in the same place. My own summary of the instruction set is at https://dercuano.github.io/notes/8080-opcode-map.html#addtoc....

It wasn't unusual in the 80s to type in machine code listings to a PC; I remember doing this as an 8-year-old from magazines, but I didn't understand any of the stuff I was typing in.

Typing from mags, getting interested in how the magic works by learning to use a hex monitor and trying out things. I was a kid so time enough.

I didn't know you could do it differently for years after I started.

I implemented a PDP-11 in 2007-10 and I can still read PDP-11 Octal