Browser Automation: How to Automate Button Clicks and Keyboard Input with Python and Selenium
2020-10-21 23:00:04 | | Part 3 of 3
- Linux Ubuntu 20.04
- Windows 10
- macOS Catalina
Make sure to complete the prerequisites, above, as this tutorial requires a project skeleton with VirtualEnv, Selenium, and a browser driver of your choosing. It's also important that you're familiar with navigating the HTML DOM tree, as we will be targeting specific elements with out automation code. Once completed, you should have the following folders and files:
▾ python-browser-automation/ ▸ venv/ ▾ drivers/ chromedriver geckodriver main.py
To verify that the virtual environment is active, on Windows 10, make sure (venv) is in the PowerShell command prompt. For example, (venv) PS C:\Users\username\python-browser-automation>
To verify that the virtual environment is active, on Linux Ubuntu, make sure (venv) is in the terminal command prompt.
Using Browser Automation to Populate Form Fields
If you've completed our tutorial about HTML DOM Inspection, you should already be able to understand the following HTML markup, which is similar to simple subscription forms you find online. This specific example contains a subscription form that contains an email field, a checkbox, a submit button, and a hidden field. You're going to learn how to target and type values into all of them, without any manual input, using only automated browser code with Python and Selenium.
With this code in hand, and a few small tweaks, you'll be able to automate forms like it, on any website of your choosing. Just navigate to the page with the form you want to automate, inspect the element or view the source code, and read through the HTML tags to find ways to target it.
<html> <head> <title>Web Page Title</title> </head> <body> <section class="container"> <form method="POST" id="subscribe-form" action="https://api.codeboxsystems.com/mock/request" onsubmit="return handleSubscribe()" target="_top" enctype="application/x-www-form-urlencoded" _lpchecked="1"> <input type="hidden" name="Action" value="SendMessage"> <fieldset> <label> <input type="email" name="email" id="subscribe-email" required=""> </label> <br> <label class="checkbox"> <input type="checkbox" name="terms" id="subscribe-terms" required=""> I agree to the <a href="#">terms and conditions</a> </label> <br> <input type="submit" id="subscribe-submit" value="Submit"> </fieldset> <div class="alert info hide" id="subscribe-info"></div> <div class="alert success hide" id="subscribe-success"></div> <div class="alert error hide" id="subscribe-error"></div> </form> </section> </body> </html>
The following code navigates to a mock website, with the above subscription form, and fills out the email field with a fake email address. We also leverage time.sleep() to wait 5 seconds before closing the web driver, whose only purpose is to give you time to see the form fill. You can adjust the amount of seconds, or omit this line, entirely. This example uses the Chrome web driver, but feel free to replace line 8 with your web driver instantiation code of choice from the previous chapters.
import time from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By driver = webdriver.Chrome(executable_path='drivers/chromedriver') def find_element_by_id(id): try: WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, id))) element = driver.find_element_by_id(id) return element except Exception as e: print(e) # Navigate driver.get('https://mock.codeboxsystems.com') # Fill out form subscribe_input = find_element_by_id('subscribe-email') subscribe_input.clear() subscribe_input.send_keys('email@example.com') time.sleep(5) driver.quit()
Line 1: Imports the time package, giving us access to the sleep() function in line 29. As mentioned before, this allows you to force your program to wait the specified number of seconds, before proceeding to the next line of code.
Lines 2-5: Imports all of the necesary Selenium-related modules and functions.
Line 8: Instantiates a web driver instance that we store in the driver variable, for future invocations.
Lines 11-17: We define a find_element_by_id() that we can call whenever we want to target an HTML element by its ID. In line 13, we use the WebDriverWait() function, and pass in our driver and a specified delay. In this case, we want Selenium to wait up to 2 seconds for an element to become available. We then provide our until() callback with with some UI targeting convenience functions and pass in id from our parameters.
Once the element is returned, we store it into element in line 14, and return it to the function caller. We've also wrapped the function code in a try/except block, so we can see if an error occurred in the console.
The rest of the code is fairly straightforward. We provide a path for the browser to navigate to in line 21, we fetch the element in line 24, empty out the field in line 25, and send input values in line 26. In line 30, close the browser.
Using Browser Automation to Automate Button Clicks and Form Submissions
This time, we're going to submit the fake email we supplied in the previous section. The following code will be used to click the subscription form's submit button, but it's general-purpose enough to click any button we want to target. This example uses the Chrome web driver, but feel free to replace line 8 with your web driver instantiation code of choice from the previous chapters.
from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By driver = webdriver.Chrome(executable_path='drivers/chromedriver') def find_element_by_id(id): try: WebDriverWait(driver, 2).until(EC.visibility_of_element_located((By.ID, id))) element = driver.find_element_by_id(id) return element except Exception as e: print(e) def check_checkbox(id): element = find_element_by_id(id) if element is not None: driver.execute_script('arguments.click();', element) is_selected = element.is_selected() if not is_selected: driver.execute_script('arguments.click();', element) def assert_text(id, value): element = find_element_by_id(id) if element is not None: text = element.text try: assert text == value except AssertionError as e: print(e) # Navigate driver.get('https://mock.codeboxsystems.com') # Fill out form subscribe_input = find_element_by_id('subscribe-email') subscribe_input.clear() subscribe_input.send_keys('firstname.lastname@example.org') check_checkbox('subscribe-terms') # Submit subscribe_button = find_element_by_id('subscribe-submit') subscribe_button.click() assert_text('subscribe-success', 'Success') driver.quit()
Lines 28-35: We define an assert_text() function that accepts an element's id, and a text value to validate against. This is useful for when we perform an operation, and want to verify that some expected text was rendered.
Line 45: We check the checkbox with the "subscribe-terms" ID.
Lines 48-50: We target the subscribe button, click, and verify that an element with an ID of "subscribe-success" appeared with the value "Success".
As a general rule of thumb, before you write Python/Selenium code, you'll want to manually perform the steps you'd like to automate. You should then inspect what gets rendered to the browser, and make note of elements that appear and disappear, their IDs, and their text values, because they give you ways to access and run assertions to verify that you are getting the expected result.
The examples above allow you to target and pass a specified value to any input field with an ID. Future tutorials will teach you additional targeting methods, such as targeting by class name, tag name, or attribute. Subscribe to get notified when they get released.
If you're interested in programs that carry out your computer tasks for you, take our Automation the Easy Way with Python course. This course teaches CSV and Excel file generation, API requests, website scraping, email delivery, task scheduling, and browser click, mouse, and keyboard automation. Automate your daily tasks, free up time, and get ahead, today.