Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I think the author completely misses a mention that sequential access to cold store is nowhere as slow as access to ram.

Cold store looses big in only scenarios where you have sequential and completely random seek patterns, and there are lots of way to optimise that in read and write heavy workloads. This was the art of running performant multi-terabyte DBs in a world prior to SSDs and ramdisks.

For very big index walks, you want to have data to be more flat, and seeks to be sorted, so there is higher chance that needed records would be accessed without page eviction. Modern DBs, I believe, do something like that internally.

And for write heavy loads, there is no alternative to revising "data structures 101." You can reduce the disk load by may times over with a properly picked tree, graph, or log structure.



Just wanted to add that tree and graph structures of course can be stored sequentially on disk or a flash drive (log-structured). For instance with a really fast random access storage and some additional space you can even save the history, that is lightweight snapshots of your data. That is you always append data.

This way you can also use clever versioning algorithms to predict the performance (no read or write peaks) and to lower the space consumption.


Do you a list of required reading?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: