Skip to content

16th October 2022

Coding⚑

Languages⚑

Selenium⚑

  • New: Click on element.

    Once you've opened the page you want to interact with driver.get(), you need to get the Xpath of the element to click on. You can do that by using your browser inspector, to select the element, and once on the code if you right click there is a "Copy XPath"

    Once that is done you should have something like this when you paste it down.

    //*[@id=”react-root”]/section/main/article/div[2]/div[2]/p/a

    Similarly it is the same process for the input fields for username, password, and login button.

    We can go ahead and do that on the current page. We can store these xpaths as strings in our code to make it readable.

    We should have three xpaths from this page and one from the initial login.

    first_login = '//*[@id=”react-root”]/section/main/article/div[2]/div[2]/p/a'
username_input = '//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[2]/div/label/input'
password_input = '//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[3]/div/label/input'
login_submit = '//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[4]/button/div'

    Now that we have the xpaths defined we can now tell Selenium webdriver to click and send some keys over for the input fields.

    from selenium.webdriver.common.by import By

driver.find_element(By.XPATH, first_login).click()
driver.find_element(By.XPATH, username_input).send_keys("username")
driver.find_element(By.XPATH, password_input).send_keys("password")
driver.find_element(By.XPATH, login_submit).click()

  • New: Bypass Selenium detectors.

    Sometimes web servers react differently if they notice that you're using selenium. Browsers can be detected through different ways and some commonly used mechanisms are as follows:

    • Implementing captcha / recaptcha to detect the automatic bots.
    • Non-human behaviour (browsing too fast, not scrolling to the visible elements, ...)
    • Using an IP that's flagged as suspicious (VPN, VPS, Tor...)
    • Detecting the term HeadlessChrome within headless Chrome UserAgent
    • Using Bot Management service from Distil Networks, Akamai, Datadome.

    They do it through different mechanisms:

    If you've already been detected, you might get blocked for a plethora of other reasons even after using these methods. So you may have to try accessing the site that was detecting you using a VPN, different user-agent, etc.