Web scraping is nothing but the extraction of data from a website. Once the data is extracted, it can be stored for further analysis.
This type of automation might be essential to collect data and information, such as contacts, email addresses, stock data, cost matrix, telephone numbers, and any other search terms, from various websites. The extracted data can then be stored in a structured format either inside a database or a simple CSV or Excel file for further analysis.
Refer to list of Web Automation activities.
Currently, IntelliBuddies supports web automation using only the Chrome browser.
Make sure URL to be used is open in Chrome before using element selector at design time. Refer to How to use Element selector for selecting appropriate element and extracting the required data.
Currency Conversion Data Extraction Sample
Let us understand the Web Scraping with one more example process.
Here is an example that takes input USD value and gets the conversion rate from the url and displays the final amount after conversion in INR.
Come, let's create this process in Process Designer:
Launch and create a new process in Process Designer
Launch Process Designer from the Windows Start menu located under IntelliBuddies Enterprise
Create New Process
Click on New Process and fill in the new process details
Add variables to hold the input and output details.
Add Open Browser to open the URL in browser.
In Activity panel search for Input Dialog
Takes the input USD value and sets to int variable.
Create a sequence and add activities to get the XPath value from the config file containing a mapping of XPath value and string constant.
CSV and DataTable activities are used in Get_XPath_Value.xaml to get the appropriate XPath value from config.
Export this sequence as Snippet.
Drag and drop this Snippet in the main workflow, which adds invoke workflow activity, pointing to the above conversion workflow.
XPath config file
Add Set Text Field Value to set the text field in the webpage.
Add Get Element Value to get the per USD and converted INR value.
Add Writeline to display the converted value.
Save the process
From the Design tab ribbon, click on Save
Run the process
From the Design tab ribbon, click on Run
Make sure the process executes and pops up an input dialog. Enter the USD value to be converted to INR and click on Ok. Then in the output window, the converted value is displayed.
Download Example (C#)