Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One thing Kafka bite me is for each partition there can be only one consumer. If your consumer had performance issue (e.g using CPython) then you are out of luck.


The thing this article pointed out to me, which I didn't know before, it that is why you should just set the partition count very high to begin with. Then you just horizontally scale consumers.


Yeas that's the lesson I learned the hard way. The publisher was from another department and had only 1 partition with large volume of data.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: