Isn't the difference though, that Leela is a pure RL implementation whereas NNUE/SF still uses some tablebase stuff?

No, NNUE doesn't necessarily need tablebases and Leela uses tablebases too. The difference is that their network architecture are different and their search algos are different.

Leela uses mcts, sf does not. SF also trains on human positions, while I think Leela is self play only.

AlphaZero and co only trained via self-play because that was their research goal, and they looked unwinnable because they were the first to get this kind of neural net working, but it doesn't seem like it's the best option.

AZ notably got completely tilted once it started losing, doesn't necessarily recognize strange positions you can't normally get into, and doesn't care about its win margin at all.

SF14's net was actually trained on lc0 data. Human data is way too weak for SF to learn from.

Yes and I think it's really generous of the LC0 to "donate" that data. On the other side LC0 uses SF during the training process since a while.

It makes sense. They're both open source programs working towards better understanding of chess. There are actually a few people who develop for both. They are obviously very different types of engines, but they are much closer to friendly rivals than enemies.