Browser Automation: How to Automate Button Clicks and Keyboard Input with Python and Selenium

2020-10-21 23:00:04 | #programming #python #automation | Part 3 of 3

Make sure to complete the prerequisites, above, as this tutorial requires a project skeleton with VirtualEnv, Selenium, and a browser driver of your choosing. It's also important that you're familiar with navigating the HTML DOM tree, as we will be targeting specific elements with out automation code. Once completed, you should have the following folders and files:

▾ python-browser-automation/
  ▸ venv/
  ▾ drivers/
    chromedriver
    geckodriver
  main.py

To verify that the virtual environment is active, on Windows 10, make sure (venv) is in the PowerShell command prompt. For example, (venv) PS C:\Users\username\python-browser-automation>

To verify that the virtual environment is active, on Linux Ubuntu, make sure (venv) is in the terminal command prompt.

Using Browser Automation to Populate Form Fields

If you've completed our tutorial about HTML DOM Inspection, you should already be able to understand the following HTML markup, which is similar to simple subscription forms you find online. This specific example contains a subscription form that contains an email field, a checkbox, a submit button, and a hidden field. You're going to learn how to target and type values into all of them, without any manual input, using only automated browser code with Python and Selenium.

With this code in hand, and a few small tweaks, you'll be able to automate forms like it, on any website of your choosing. Just navigate to the page with the form you want to automate, inspect the element or view the source code, and read through the HTML tags to find ways to target it.

<html>
  <head>
    <title>Web Page Title</title>
  </head>
  <body>
    <section class="container">
      <form method="POST" id="subscribe-form" action="https://api.codeboxsystems.com/mock/request" onsubmit="return handleSubscribe()" target="_top" enctype="application/x-www-form-urlencoded" _lpchecked="1">
        <input type="hidden" name="Action" value="SendMessage">

        <fieldset>
          <label>
            <input type="email" name="email" id="subscribe-email" required="">
          </label>
          <br>
          <label class="checkbox">
            <input type="checkbox" name="terms" id="subscribe-terms" required="">
            I agree to the <a href="#">terms and conditions</a>
          </label>
          <br>

          <input type="submit" id="subscribe-submit" value="Submit">
        </fieldset>
        <div class="alert info hide" id="subscribe-info"></div>
        <div class="alert success hide" id="subscribe-success"></div>
        <div class="alert error hide" id="subscribe-error"></div>
      </form>
    </section>
  </body>
</html>

The following code navigates to a mock website, with the above subscription form, and fills out the email field with a fake email address. We also leverage time.sleep() to wait 5 seconds before closing the web driver, whose only purpose is to give you time to see the form fill. You can adjust the amount of seconds, or omit this line, entirely. This example uses the Chrome web driver, but feel free to replace line 8 with your web driver instantiation code of choice from the previous chapters.

import time
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


driver = webdriver.Chrome(executable_path='drivers/chromedriver')


def find_element_by_id(id):
    try:
        WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, id)))
        element = driver.find_element_by_id(id)
        return element
    except Exception as e:
        print(e)


# Navigate
driver.get('https://mock.codeboxsystems.com')

# Fill out form
subscribe_input = find_element_by_id('subscribe-email')
subscribe_input.clear()
subscribe_input.send_keys('test@email.com')

time.sleep(5)

driver.quit()

Line 1: Imports the time package, giving us access to the sleep() function in line 29. As mentioned before, this allows you to force your program to wait the specified number of seconds, before proceeding to the next line of code.

Lines 2-5: Imports all of the necesary Selenium-related modules and functions.

Line 8: Instantiates a web driver instance that we store in the driver variable, for future invocations.

Lines 11-17: We define a find_element_by_id() that we can call whenever we want to target an HTML element by its ID. In line 13, we use the WebDriverWait() function, and pass in our driver and a specified delay. In this case, we want Selenium to wait up to 2 seconds for an element to become available. We then provide our until() callback with with some UI targeting convenience functions and pass in id from our parameters.

Once the element is returned, we store it into element in line 14, and return it to the function caller. We've also wrapped the function code in a try/except block, so we can see if an error occurred in the console.

The rest of the code is fairly straightforward. We provide a path for the browser to navigate to in line 21, we fetch the element in line 24, empty out the field in line 25, and send input values in line 26. In line 30, close the browser.

Using Browser Automation to Automate Button Clicks and Form Submissions

This time, we're going to submit the fake email we supplied in the previous section. The following code will be used to click the subscription form's submit button, but it's general-purpose enough to click any button we want to target. This example uses the Chrome web driver, but feel free to replace line 8 with your web driver instantiation code of choice from the previous chapters.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By


driver = webdriver.Chrome(executable_path='drivers/chromedriver')


def find_element_by_id(id):
    try:
        WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, id)))
        element = driver.find_element_by_id(id)
        return element
    except Exception as e:
        print(e)


def check_checkbox(id):
    element = find_element_by_id(id)
    if element is not None:
        driver.execute_script('arguments[0].click();', element)
        is_selected = element.is_selected()
        if not is_selected:
            driver.execute_script('arguments[0].click();', element)


def assert_text(id, value):
    element = find_element_by_id(id)
    if element is not None:
        text = element.text
        try:
            assert text == value
        except AssertionError as e:
            print(e)


# Navigate
driver.get('https://mock.codeboxsystems.com')

# Fill out form
subscribe_input = find_element_by_id('subscribe-email')
subscribe_input.clear()
subscribe_input.send_keys('test@email.com')
check_checkbox('subscribe-terms')

# Submit
subscribe_button = find_element_by_id('subscribe-submit')
subscribe_button.click()
assert_text('subscribe-success', 'Success')

driver.quit()

Line 19-25: Here, we define a check_checkbox() method. We utilize our find_element_by_id() method, defined from the previous chapter, and only proceed with the rest of the code if an element is found. We then rely on driver.execute_script() to run JavaScript. This function is very versatile, and we're using it here because it's more reliable than other ways of clicking on a checkbox with Selenium. Sometimes, executing JavaScript is your only option.

Lines 28-35: We define an assert_text() function that accepts an element's id, and a text value to validate against. This is useful for when we perform an operation, and want to verify that some expected text was rendered.

Line 45: We check the checkbox with the "subscribe-terms" ID.

Lines 48-50: We target the subscribe button, click, and verify that an element with an ID of "subscribe-success" appeared with the value "Success".

Conclusion

As a general rule of thumb, before you write Python/Selenium code, you'll want to manually perform the steps you'd like to automate. You should then inspect what gets rendered to the browser, and make note of elements that appear and disappear, their IDs, and their text values, because they give you ways to access and run assertions to verify that you are getting the expected result.

The examples above allow you to target and pass a specified value to any input field with an ID. Future tutorials will teach you additional targeting methods, such as targeting by class name, tag name, or attribute. Subscribe to get notified when they get released.

If you're interested in programs that carry out your computer tasks for you, take our Automation the Easy Way with Python course. This course teaches CSV and Excel file generation, API requests, website scraping, email delivery, task scheduling, and browser click, mouse, and keyboard automation. Automate your daily tasks, free up time, and get ahead, today.

Want To See More Exercises?

View Exercises View Courses

Comments

You must log in to comment. Don't have an account? Sign up for free.

Subscribe to comments for this post

Want To Receive More Free Content?

Would you like to receive free resources, tailored to help you reach your IT goals? Get started now, by leaving your email address below. We promise not to spam. You can also sign up for a free account and follow us on and engage with the community. You may opt out at any time.



Tell Us About Your Project









Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA &amp; testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.

Contact