Where are the pub/sub magic overlays over kafka ?

How does spark work on this ...

does not seem very fit for large scale.