Description:

The Output Column Transform step defines the final set of columns that will be passed to subsequent ETL components. Rather than modifying the values in the dataset, this transformation controls the output schema by selecting which columns should remain available for downstream processing.

The columns to be included in the output are configured through the Configure Columns List property. During execution, the transformation generates a dataset containing only the selected columns while preserving the associated data values for each record.

This transform is commonly used near the end of an ETL pipeline to prepare datasets for export, reporting, data warehousing, or integration with external applications. It is also useful for hiding intermediate processing fields that were required earlier in the ETL workflow but are not needed in the final output.

The step supports:

• Output schema selection
• Column projection
• Dataset simplification
• Final output preparation
• Configurable column selection
• Enterprise ETL workflows
• Integration with FlowWright ETL pipelines

This step can be used for:

• Preparing export files
• Creating reporting datasets
• Database loading
• API response preparation
• Data warehouse integration
• Removing intermediate processing fields
• Enterprise data integration

Usage:

The Output Column Transform step is typically placed near the end of an ETL pipeline after data cleansing, validation, enrichment, and aggregation have been completed.

During execution, the transformation projects the dataset to include only the configured output columns. This produces a simplified schema that is ready for export or consumption by downstream systems.

A typical ETL pipeline might look like this:

Typical usage scenarios include:

• Selecting columns for CSV or Excel exports
• Preparing standardized reporting datasets
• Creating destination-specific schemas
• Removing temporary calculation fields
• Preparing API response datasets
• Simplifying database imports
• Producing datasets for business intelligence tools

The resulting dataset contains only the selected output columns while maintaining the original record values.

Prerequisite:

The ETL process engine service should be running to execute the ETL definitions.

Example:

Let’s build and execute the “clsOutputColumnsTransformDef” example.          

• Create a new ETL definition named “clsOutputColumnsTransformDef” and open it in designer mode. 
• Drag “clsInputCSV, clsOutputColumnsTransform, and clsOutputCSV” steps to the canvas. 
• Connect the dots between the “clsInputCSV” step and other steps, as shown above.
• Define a variable or a global to hold the virtual path.
• Click the "clsInputCSV" step to configure its "Required" properties. Provide a name for the step. Specify the path to the input CSV file on the application server. Select the ETL data schema from the drop-down list. Note: The schema's column names must match those in the input file. Click Save. Click here to learn about the ETL Data Schema Designer.

• Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The pop-up window displays the schema columns as shown below. This function helps you understand the schema at a glance.  

• Click the "clsOutputColumnsTransform" step to configure its "Required" properties. Enter the step name, then click the button to configure the columns to transform. Finally, click the Save button. 

• Click the "clsOutputColumnsTransform" step to configure its "Required" properties. Click the column button to open the transform list configuration window. Click the Add Row (+) button to insert an empty row. Select a column from the drop-down list, then click the Save button. You may add multiple columns for transformation using the Add Row button. 

• Click the "clsOutputCSV" step to configure its "Required" properties. Enter a name for the step and the path to the output CSV file on the application server. Click the Save button. 

• Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name for the step and specify the variable or global reference that will hold the virtual output path. Click the Save button. 

• The “Logging” configuration is necessary for documentation and to measure workflow progress and percent complete. This is done by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.

• Save the ETL definition, create a new ETL instance, and execute it. Render the ETL instance. Click the ETL step to view its properties. The “clsOutputColumnsTransform” step should order columns according to the output file’s configuration. The variable or global contains the output file’s virtual path, as shown below. 

Tips:

• Place this transform near the end of the ETL pipeline after all required transformations have completed.
• Include only the columns required by downstream systems or reports.
• Exclude temporary or intermediate processing columns to simplify the final schema.
• Verify that required destination mappings reference only the selected output columns.
• Use descriptive and standardized column names before configuring the output schema.
• Validate the resulting dataset before deploying the ETL definition to production.
• Combine this transform with Rename Column, Set Field Value, Validate Data, Sorting Transform, Pivot Transform, Aggregate Transform, or destination components such as CSV Output File, JSON Output File, or SQL Output to build complete ETL workflows.

Notes:

• This transformation performs only column projection; it does not modify the values in the selected columns.
• Multiple output columns can be configured within a single transformation.
• Only the configured columns remain available to downstream ETL components.
• Columns excluded from the output schema cannot be referenced by subsequent transformations.
• All selected columns must exist within the ETL schema.
• Invalid column mappings or schema inconsistencies may result in ETL execution errors.
• This transform is commonly used in reporting, data migration, API integration, business intelligence, regulatory reporting, and enterprise data preparation workflows.
• Review downstream destination mappings whenever the output schema changes.

Definition Sample:

You may download the sample ETL definition(s) from the link provided and later import them into your FlowWright ETL Designer.

Note: Verify and complete any missing configuration after importing the sample, including:

• Output column selection
• ETL schema validation
• Downstream transformation mappings
• Destination component configuration
• Export format settings
• Environment-specific configuration

After verifying the configuration, save the ETL Definition before execution.

Click here to download the sample file.