Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

No, nothing like that. I wonder how that detection works?

I tried this and it took a shot of a "bot detected" screen:

    shot-scraper https://news.com.au/
But when I used interactive mode I could take the screenshot - run this:

    shot-scraper -i https://news.com.au/
It opens a Chrome window. Then hit "enter" in the CLI tool to take the screenshot.


Got this to work!

    shot-scraper https://news.com.au/ \
      --init-script 'delete Object.getPrototypeOf(navigator).webdriver' \
      --user-agent 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0'
Code and screenshots and prototype --init-script feature here: https://github.com/simonw/shot-scraper/issues/147#issuecomme...


If you want to go down the nowsecure.nl rabbithole (often used as a benchmark for passing bot detection) [1] is a good resource. It includes a few fixes that undetected-chromedriver doesn’t.

1. https://seleniumbase.io/help_docs/uc_mode/#uc-mode


Nice! I was going to say that using experimental headless mode with the "User-Agent Switcher for Chrome" plugin might work too.


Love your work!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: