As you said, everyone is different. I drink coffee (~2 shots of espresso) almost at the exact same time every morning. Then again after lunch.

There's basically zero impact if I miss the morning one.

I've done multiple stretches of not drinking coffee (weeks/months), with also basically no noticeable changes: not more or less tired, no change in sleep, etc.