Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I wonder if DuckDB could do better than SQLite?


Tried it. I think it is marginally better than SQLite, but it is not the best solution.

I work for IPinfo and was quite excited when they added support for the IP address data type. However, at the end of the day, the most efficient lookup mechanism has to be the MMDB database. Rather doing the enrichment of IP addresses inside of DuckDB, the better solution was doing an enrichment outside of DuckDB using the MMDB database, dump it on a csv and inserting the CSV in DuckDB as a table.

See the top comment of the thread for more context around MMDB.


Yeah I gave it a shot, it didn't have much over SQLite other than the file size was a lot smaller.

Also, I love ipinfo excellent service! We use it all the time at work.


Awesome. We’ve integrated with platforms like GCP BQ, Snowflake, Clickhouse, and Postgres etc. and still, MMDB stands out as the most performant, especially for enriching millions of IPv6 addresses.

The best approach might be to enrich IP addresses via an MMDB database, then dump the results in temp table and query the results as you would any table. If you haven't tried the free db, you should as this illustrate this point pretty well: https://ipinfo.io/products/free-ip-database

It’s great to hear that you’re using our service! Please feel free to reach out anytime if you have any queries/feedback. This thread alone contains comments from 2 IPinfo engineers and me!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: