Description:

The Evaluate Column Expressions step evaluates one or more configurable expressions using values from existing ETL columns. The transformation is configured through the Configure Expression List property, where ETL designers define expressions that calculate new values based on one or more source columns.

Enterprise ETL processes frequently require derived values that are calculated from existing data rather than directly imported from the source system. Examples include calculating order totals, profit margins, employee tenure, sales commissions, weighted scores, inventory values, and business performance metrics. This transformation enables these calculations through configurable expressions instead of custom code.

Each configured expression is evaluated for every row processed by the ETL pipeline. The calculated results can be stored in new or existing columns and immediately become available to downstream ETL components.

Unlike Evaluate Expressions and Set Values, which is commonly used to assign fixed or calculated values to destination columns as part of broader business-rule processing, Evaluate Column Expressions focuses primarily on calculating values directly from existing ETL column data using configurable expressions.

The Evaluate Column Expressions step is typically placed after importing and validating data but before filtering, grouping, aggregation, reporting, exporting, or database loading.

The step supports:

• Column-based expression evaluation
• Derived field calculations
• Multiple configurable expressions
• Business metric generation
• Conditional calculations
• Enterprise ETL workflows
• Integration with FlowWright ETL pipelines

Typical business uses include:

• Calculating order totals
• Computing profit margins
• Deriving employee tenure
• Calculating commissions
• Generating reporting metrics
• Creating analytical datasets
• Preparing data warehouse loads

Usage:

The Evaluate Column Expressions step is typically inserted after importing and validating source data but before grouping, filtering, aggregation, reporting, exporting, or loading the transformed dataset into downstream systems.

During execution, each configured expression is evaluated using the values from the current ETL row. The resulting calculated values become immediately available for subsequent ETL transformations.

A typical ETL pipeline might look like this:

Typical usage scenarios include:

• Calculating extended prices from quantity and unit cost
• Computing sales commissions
• Deriving customer lifetime values
• Calculating inventory valuations
• Computing performance scores
• Generating reporting metrics
• Preparing analytical datasets for data warehouses

The calculated values can then be filtered, grouped, aggregated, validated, exported, or loaded into downstream business systems.

Prerequisite:

The ETL process engine service should be running to execute the ETL definitions.

Example:

Let’s build and execute the “clsComputeExpressionDef” example.          

• Create a new ETL definition named “clsComputeExpressionDef” and open it in designer mode. 
• Drag “clsInputCSV, clsComputeExpression, and clsOutputCSV” steps to the canvas.
• Connect the dots between the “clsInputCSV” step and other steps, as shown above.
• Define a variable or a global variable to hold the file path.
• Click the "clsInputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the input CSV file on the application server. Select the ETL data schema from the drop-down list. Note: The schema's column names should match those in the input file. Click the Save button. 

• Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The pop-up window displays the schema columns as shown below. This function helps to understand the schema at a glance.  

• Click the "clsComputeExpression" step to configure its "Required" properties. Provide a name for the step. Click the button to configure the transform list, then click Save. 

• Click the "clsComputeExpression" step to configure the transform list. A pop-up window will appear. Click the Add Row (+) button to insert an empty row. Enter the expression to evaluate. Click the Save button. You may add multiple transformation columns by using the Add Row button.  

• Click the "clsOutputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the output CSV file on the application server. Click the Save button. 

• Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name for the step. Provide a variable or global reference to store the virtual output path. Click the Save button. 

• The “Logging” configuration is necessary for documentation and also measures workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown in the images below. Configure the “Logging” using the following properties.

• Save the ETL definition, create a new ETL instance, and execute. Render the ETL instance. Click the ETL step to view its properties. The “clsComputeExpression” step should transform the output file by evaluating the expression result. The variable or global reference contains the output file’s virtual path, as shown below. 

Tips:

• Validate source data before evaluating expressions to minimize calculation errors.
• Keep expressions readable and well documented to simplify maintenance.
• Use meaningful names for calculated columns.
• Preserve original source values whenever auditing or historical comparison is required.
• Test expressions using representative production datasets before deployment.
• Review business formulas whenever reporting or operational requirements change.
• Combine this transform with Validate Data, Transform Number Operations, Filter with Expressions, Group By Transform, Sorting Transform, JSON Output File, or SQL Output components to build comprehensive enterprise ETL workflows.

Notes:

• Multiple expressions can be configured within a single transformation.
• Expressions are evaluated independently for each processed row.
• Source columns referenced within expressions must exist within the ETL schema.
• Calculated values immediately become available to downstream ETL components.
• Invalid expressions or incompatible data types may result in ETL execution errors.
• This transform is commonly used for financial calculations, ERP migration, CRM integration, business intelligence, KPI generation, reporting, analytical processing, and enterprise data warehouse preparation.
• Review downstream calculations whenever business formulas or expression definitions are modified.

Evaluate Column Expressions vs. Evaluate Expressions and Set Values:

Although both transformations evaluate configurable expressions, they are intended for different scenarios:

As a general guideline:

• Use Evaluate Column Expressions when calculations are derived directly from existing ETL column values.
• Use Evaluate Expressions and Set Values when expressions are used to populate destination columns as part of configurable business-rule processing.

Definition Sample:

You may download the sample ETL definition(s) from the link provided and later import them into your FlowWright ETL Designer.

Note: Verify and complete any missing configuration after importing the sample, including:

• Expression definitions
• Source and output column mappings
• Business calculation logic
• ETL schema validation
• Downstream transformation mappings
• Destination component configuration
• Environment-specific settings

After verifying the configuration, save the ETL Definition before execution.

Click here to download the sample file.