Description:

This step reads data from a web page using an XPath expression.

The Read Web Data Using XPath workflow step retrieves the HTML content from a specified URL, evaluates the supplied XPath expression against the page, extracts the matching content, and stores the result in a FlowWright Variable or Global Variable.

This step provides an efficient way to automate web data extraction without requiring custom web scraping code. It enables workflows to collect information from web pages for reporting, monitoring, integrations, data validation, content processing, or business process automation.

The extracted value can be consumed immediately by downstream workflow steps for additional processing or decision-making.

The step supports:

• Reading web page content
• XPath-based data extraction
• Dynamic URL configuration
• Storage of extracted data in Variables or Global Variables
• Integration with FlowWright workflow automation
• Automated web content retrieval

This step can be used for:

• Web data extraction
• Price monitoring
• Status monitoring
• Content retrieval
• Business intelligence
• System integrations
• Business process automation

Usage:

The Read Web Data Using XPath step is typically placed before reporting, validation, notifications, integrations, or business rule processing activities that require information from an external website.

The workflow retrieves the specified web page, applies the configured XPath expression, stores the extracted value, and makes the result available to subsequent workflow steps.

A typical workflow might look like this:

Once the web data has been extracted, later workflow steps can:

• Compare retrieved values
• Generate reports
• Trigger notifications
• Update workflow variables
• Call REST APIs
• Execute business rules
• Continue workflow execution

The target website must be accessible before this step is executed.

Example:

Let’s build and execute the “clsWebReadDataXPathDef” example.          

• Create a new process definition named “clsWebReadDataXPathDef” and open it in designer mode.
• Drag a “clsWebReadDataXPath” step to the canvas. 
• Connect the dots between the “Start” step and the “clsWebReadDataXPath” step, as shown above. 
• Click the "clsWebReadDataXPath" step to configure its "Required" properties. Provide the URL to read the data from. Provide the XPath expression to use to read the data. Provide the variable or global reference to store the result. Click the Save button. Note: Click the "AI Predict" button for the Copilot to add new process steps that match your process description. 

• The “Logging” setting configuration is necessary for documentation and for measuring workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.

• Save the Process Definition. Create a new Process Instance and execute it. When the workflow reaches the Read Web Data Using XPath step, FlowWright retrieves the specified web page, evaluates the supplied XPath expression, extracts the matching content, and stores the result in the configured Variable or Global Variable. If the operation completes successfully, the workflow follows the True return path. If the website cannot be reached, the URL is invalid, the XPath expression is invalid, or no matching data can be extracted, the workflow follows the False return path. Click the process step to view its properties. The step should successfully retrieve the requested web content, populate the configured Variable or Global Variable with the extracted value, and return True upon completion.

Tips:

• Verify that the specified URL is accessible from the FlowWright server.
• Test the XPath expression using an HTML inspection tool before deploying the workflow.
• Use workflow variables to dynamically construct URLs or XPath expressions at runtime.
• Keep XPath expressions as specific as possible to improve reliability when web page layouts change.
• Store extracted values in Global Variables when multiple workflow steps require access to the retrieved data.
• Consider implementing retry logic when retrieving data from external websites that may be temporarily unavailable.
• Use the True and False return paths to implement appropriate success and exception handling.
• Combine this step with REST Call, JSONPathToList, Convert HTML to PDF, Send Email, Update Variables, or Conditional Routing workflow steps to automate web data monitoring and integration workflows.

Notes:

• The target website must be reachable from the FlowWright server.
• The XPath expression must match the HTML structure of the retrieved page.
• Changes to the target website's HTML structure may require updates to the configured XPath expression.
• Invalid URLs, inaccessible websites, malformed XPath expressions, or unmatched elements may cause the step to return False.
• The extracted value is stored in the configured Variable or Global Variable.
• This step retrieves web content only and does not modify the target website.
• The workflow should include appropriate handling for both True and False return paths.
• When accessing websites that require authentication or dynamically generated content, additional workflow steps or integration methods may be required.

Definition Sample:

You may download the sample definition(s) from the link provided and later import them (drag-and-drop) into your FlowWright Process Definition page.

Note: Verify and complete any missing configuration after importing the sample, including:

• Target URL
• XPath expression
• Result Variable or Global Variable mapping
• Network connectivity
• Workflow variable definitions
• Environment-specific settings

After verifying the configuration, save the Process Definition before execution.

Click here to download the sample file.