Selenium Web Browser Automation

100 readers

4 users here now

Users forum for selenium browser testing. Do not advertise here.

founded 1 year ago

MODERATORS

Terevos

[email protected]

I'm a human, But I can't pass the human check in the broswer of selenium (stackoverflow.com)

submitted 4 months ago by jerron to c/selenium

3 comments fedilink hide all child comments

I'm using selenium to open the webpage https://www.optionistics.com/quotes/stock-option-chains/GOOG It was loaded successfully at first.


from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC

from selenium.webdriver.firefox.service import Service as FirefoxService
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.firefox.options import Options


options = Options()
options.set_preference("general.useragent.override", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36")
driver = webdriver.Firefox(service=FirefoxService(GeckoDriverManager().install()),options=options)
driver.get("https://www.optionistics.com/quotes/stock-option-chains/GOOG")

And then I came across the human verification after I changed the settings in the web page (for example, the "report date") and the form submitted to fetch new data. The problem is, I can't pass the check myself by clicking checkbox. After click, it would pend for a while before refreshd and showed the checkbox again. I clicked and it won't let me through

I'm not meant to bypass it. I just want to get myself verified. Is the Cloudflare expecting something from the header?

all 4 comments

sorted by: hot top controversial new old

[–] Kojichan 3 points 4 months ago (1 children)

CloudFlare has an issue with you if you've previously failed human checks or are using a contained browser, with or without VPN and ad blocking extensions. They check your browsing history, and what other human checks you may or may not have failed, just by you using the internet. If they can't set a validation cookie for tracking you between other CloudFlare services, they may treat you as a bot and hang your connection.

I typically fail the "Select all the motorcycles" test because they show me scooters all the time. By doing that, my computer gets marked as suspicious, slowing down some of my connections that need to go through CloudFlare, sometimes just hanging my connection for a little time.

A serious question does arise, though. Have you tried using a regular chromium type browser to see its headers sent to CloudFlare to identify what might need to be sent?

[–] jerron 1 points 4 months ago

Yes, all the sessions with regular browsers are working as expected. The problem only occurs in the browser launched by selenium. And the check didn't appear at all when I changed to undetected_chromedriver, even including some automations that indeed by bot.

[–] breadsmasher 2 points 4 months ago* (last edited 4 months ago)

My first assumption would be its due to running within the selenium session will be detected as a bot