Web Automation with Python Selenium

In this article we want to learn about Web Automation with Python Selenium, so Python Selenium is a powerful and open source framework that allows developers to automate web browsers actions. Using Selenium you can interact with web elements, navigating through pages or extracting data, Selenium empowers users to automate repetitive tasks and save time. Selenium supports different web browsers like Chrome, Firefox, Safari and Edge.


To work the examples of this tutorial you need some requirements, first you should have installed Python in your system, after that we need to install Python Selenium and you can use pip for that like this.



Selenium requires a browser specific driver to interact with web browsers. Download the appropriate driver for your preferred browser like ChromeDriver for Chrome and add it to your system PATH, or you can directly copy the driver and add that where you have your Python code.


You can download the drivers from here.

Chrome: https://chromedriver.chromium.org/downloads
Edge: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
Firefox: https://github.com/mozilla/geckodriver/releases
Safari: https://webkit.org/blog/6900/webdriver-support-in-safari-10/




Python Selenium Automating Simple Actions

Let’s start with a simple example of automating a Google search using Python Selenium:



So in the above code first we have imported the required classes from selenium.



After that we launched Chrome browser:



Using this code we can navigate to Google:



In here find_element() method is used to locate the search input field on the webpage by specifying its attribute name (name) as “q”. send_keys() method is used to enter the desired search query (Web automation with Python Selenium) into the search input field. and lastly Keys.RETURN simulates pressing the Enter key to submit the search.



The implicitly_wait() method sets a waiting period of 5 seconds, and it allows the browser to wait for the search results to load before proceeding to the next step.



The find_elements() method is used to locate multiple elements on the page that match the specified XPath expression (//div[@class=’r’]/a). In this case, it finds all the search result links. get_attribute() method extracts the href attribute from each link element, and the URLs are printed to the console.



So we can say that the above code shows a basic example of web automation using Python Selenium to perform a Google search and retrieve the search results.



Python Selenium Interacting with Dynamic Web Elements

Selenium allows us to interact with dynamic elements on web pages. For example, let’s automate logging into a website:



These lines initialize the Chrome browser using Chrome WebDriver and open the login page of the target website (in this case, “https://demoqa.com/login”).




Using find_element_by_id() method, we locate the username and password input fields on the page using their HTML id attributes. After that we use the send_keys() method to enter the desired username and password values.



For submiting the login form, we locate the login button using its XPath expression and the find_element_by_xpath() method. after that we simulate a button click using click() method.



And lastly we close the browser and terminate the WebDriver session using the quit() method.




Python Selenium Handling Waits and Timeouts

Web pages often have elements that load asynchronously or require time to appear. Selenium provides different methods to handle waits and timeouts:



Learn More on PyQt6

Leave a Comment