Description:

The Get ETL Job Status step enables a FlowWright workflow to query the status of an ETL job using its ETL process instance ID.

This step is commonly used in workflows that execute ETL jobs asynchronously and need to determine whether the job has completed before continuing with downstream processing. Rather than assuming an ETL process has finished, the workflow can periodically check its execution status and perform appropriate business actions.

The ETL process instance ID is typically obtained from a previous workflow step that starts an ETL job, such as Run ETL Job or Run ETL Validation Job, and is then supplied to this step for status retrieval.

If configured, the current ETL status is stored in a workflow variable or global variable, allowing subsequent workflow steps to evaluate or display the execution status.

Typical uses include:

• Monitoring long-running ETL processes
• Waiting for data imports to complete
• Coordinating dependent workflows
• Tracking batch processing
• Reporting ETL execution progress
• Workflow synchronization
• ETL execution auditing

This step allows workflows to monitor ETL execution without directly interacting with the ETL Designer or administration interface.

Usage:

The Get ETL Job Status step is typically used after a workflow launches an ETL process and needs to determine its current execution state.

During execution:

1. Obtain the ETL process instance ID from a previous workflow step.
2. Supply the instance ID to the Get ETL Job Status step.
3. Query the ETL execution status.
4. Store the status in a workflow variable or global variable if configured.
5. Continue workflow execution through either the True or False return path.

A typical workflow might look like this:

Typical workflow scenarios include:

• Monitoring overnight ETL imports
• Tracking scheduled batch jobs
• Waiting for data migration completion
• Coordinating dependent integration processes
• Monitoring validation jobs
• Displaying ETL progress to users
• Recording ETL execution information for audit purposes

Example:

Let’s build and execute the “clsGetETLJobStatusDef” example.          

• Create a new ETL definition called “clsGetETLJobStatusDef” and open the definition in designer mode. 
• Drag "clsExecuteETLjob, and clsGetETLJobStatus” steps to the canvas.
• Connect the dots between the “Start” and other steps as shown above.
• Define variables or globals required for the execution.
• Click the "clsExecuteETLjob" step to configure its "Required" properties. Enter a name for the step. Select the ETL definition from the drop-down list. Click Save. Note: Click the "AI Predict" button to have the Copilot add new process steps that match your process description.

•  Click the "clsExecuteETLjob" step to configure its "Optional" properties. Click the button to configure multiple variables and globals that will be passed to the ETL instance at execution. Provide the variable or global reference to store the ETL instance ID. Then click the Save button. 

• Click the "clsGetETLJobStatus" step to configure its "Required" properties. Provide a name for the step. Provide the variable or global reference that contains the ETL process instance ID. Click the Save button. Note: Click the "AI Predict" button to have the Copilot add new process steps that match your process description. 

• Click the "clsGetETLJobStatus" step to configure its "Optional" properties. Specify the variable or global reference to store the ETL status. Then click the Save button. 

• The “Logging” configuration is necessary for documentation and also measures workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.

• Navigate to the Process Definition page. Save the process definition, create a new instance, and execute it. Render the process instance. This “clsGetETLJobStatus” step shall fetch the ETL status. 

• Navigate to the ETL Instances page. Render the instance shown above. The ETL process execution is shown below.  

Tips:

• Store the ETL instance ID immediately after starting an ETL job to simplify later status retrieval.
• Use this step together with Wait or timer-based workflow logic when monitoring long-running ETL jobs.
• Store the returned status in a descriptive workflow variable for easier maintenance.
• Implement retry or polling logic when waiting for ETL completion.
• Route the False path to logging and notification steps for easier troubleshooting.
• Record ETL execution status for operational reporting and auditing.
• Combine this step with Run ETL Job, Run ETL Validation Job, Wait, Decision, Send Email, or Log Message steps to build robust ETL monitoring workflows.

Notes:

• A valid ETL process instance ID is required before this step can retrieve the execution status.
• The ETL instance ID is typically obtained from a previous ETL execution step.
• Storing the returned status is optional but recommended for workflow decision-making and reporting.
• The True and False return paths indicate whether the status lookup operation itself succeeded, not whether the ETL job completed successfully.
• The actual ETL execution state should be evaluated using the stored status value.
• Execution details and any lookup errors are recorded in the FlowWright workflow execution log.

Get ETL Job Status vs. Run ETL Validation Job:

Both workflow steps interact with ETL processes, but they serve different purposes.

As a general guideline:

• Use Get ETL Job Status when a workflow needs to monitor or retrieve the status of an ETL process that has already been started.
• Use Run ETL Validation Job when a workflow needs to launch an ETL validation definition and branch according to the validation outcome.

Definition Sample:

You may download the sample workflow definition from the link provided and import it into your FlowWright environment.

Note: Verify and complete any missing configuration after importing the sample, including:

• ETL process instance ID
• Workflow variable or global variable for storing the status
• ETL execution step that generates the instance ID
• Success and error handling
• Workflow branching
• Optional polling or retry logic

After verifying the configuration, save and publish the workflow before execution.

Click here to download the Process sample file.

Click here to download the ETL Process sample file.