Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Got this to work!

    shot-scraper https://news.com.au/ \
      --init-script 'delete Object.getPrototypeOf(navigator).webdriver' \
      --user-agent 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0'
Code and screenshots and prototype --init-script feature here: https://github.com/simonw/shot-scraper/issues/147#issuecomme...


If you want to go down the nowsecure.nl rabbithole (often used as a benchmark for passing bot detection) [1] is a good resource. It includes a few fixes that undetected-chromedriver doesn’t.

1. https://seleniumbase.io/help_docs/uc_mode/#uc-mode


Nice! I was going to say that using experimental headless mode with the "User-Agent Switcher for Chrome" plugin might work too.


Love your work!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: