I'm using selenium to open the webpage https://www.optionistics.com/quotes/stock-option-chains/GOOG It was loaded successfully at first.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.service import Service as FirefoxService
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.firefox.options import Options
options = Options()
options.set_preference("general.useragent.override", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36")
driver = webdriver.Firefox(service=FirefoxService(GeckoDriverManager().install()),options=options)
driver.get("https://www.optionistics.com/quotes/stock-option-chains/GOOG")
And then I came across the human verification after I changed the settings in the web page (for example, the "report date") and the form submitted to fetch new data. The problem is, I can't pass the check myself by clicking checkbox. After click, it would pend for a while before refreshd and showed the checkbox again. I clicked and it won't let me through
I'm not meant to bypass it. I just want to get myself verified. Is the Cloudflare expecting something from the header?
Yes, all the sessions with regular browsers are working as expected. The problem only occurs in the browser launched by selenium. And the check didn't appear at all when I changed to undetected_chromedriver, even including some automations that indeed by bot.