DOM navigation for fetching some data is for tryhards. Using a regex to grab the... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		rightbyte 3 months ago \| parent \| context \| favorite \| on: AI scrapers request commented scripts DOM navigation for fetching some data is for tryhards. Using a regex to grab the correct paragraph or div or whatever is fine and is more robust versus things moving around on the page.

chaps 3 months ago | [–]

Doing both is fine! Just, once you've figured out your regex and such, hardening/generalizing demands DOM iteration. It sucks but it is what is is.

horseradish7k 3 months ago | [–]

but not when crawling. you don't know the page format in advance - you don't even know what the page contains!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact