Description:

The CSV Input File step is an ETL source component that imports data from a CSV file into a FlowWright ETL definition.

CSV is one of the most commonly used formats for exchanging business data because it is lightweight, portable, and supported by nearly every spreadsheet application, database, ERP system, CRM platform, and business intelligence tool. The CSV Input File step provides an efficient mechanism for ingesting CSV data without requiring custom parsing logic.

The step is configured by specifying:

• The CSV input file location
• The column schema that maps CSV columns to ETL fields
• An optional schema viewer for validating the imported structure

During execution, the ETL engine opens the specified CSV file, reads each record sequentially, applies the configured column schema, and creates an in-memory dataset that becomes the source for downstream ETL components.

The Show Schema property helps ETL designers verify that the imported CSV structure aligns with the expected dataset before the ETL definition is executed.

The CSV Input File step is typically the first component in an ETL pipeline.

The step supports:

• CSV file import
• Column schema mapping
• Dataset generation
• Schema validation
• Enterprise ETL workflows
• Integration with FlowWright ETL pipelines

Typical business uses include:

• Customer data imports
• Sales data processing
• Inventory imports
• ERP data migration
• CRM synchronization
• Financial data loading
• Business intelligence staging

Usage:

The CSV Input File step is typically the first component in an ETL definition.

During execution:

1. The configured CSV file is opened.
2. Each record is read sequentially.
3. The configured column schema is applied.
4. The ETL dataset is created.
5. Processing continues to downstream ETL components.

A typical ETL pipeline might look like this:

Typical usage scenarios include:

• Importing customer master records
• Loading inventory files
• Processing daily sales extracts
• Importing financial transactions
• Migrating historical business data
• Building reporting datasets
• Feeding downstream enterprise integrations

The imported dataset can immediately be validated, transformed, filtered, grouped, aggregated, sorted, or exported to various destinations.

Prerequisite:

The ETL process engine service should be running to execute the ETL definitions.

Example:

Let’s build and execute the “clsInputCSVDef” example.          

• Create a new ETL definition called “clsInputCSVDef” and open the definition in designer mode. 
• Drag “clsInputCSV, clsTransformFormatStringToNumber, and clsOutputCSV” steps to the canvas.
• Connect the dots between the “clsInputCSV” and other steps, as shown above.
• Define a variable or a global variable to store the virtual path. 
• Click the "clsInputCSV" step to configure its "Required" properties. Provide a name for the step. Specify the path to the input CSV file on the application server. Select the ETL data schema from the drop-down list. Note: The schema's column names should match those in the input file. Click the Save button. Click here to learn about the ETL Data Schema Designer.

• Click the "clsInputCSV" step to view its "Optional" properties. Select the Show Schema button. The pop-up window displays the schema columns as shown below. This function helps to understand the schema at a glance.  

• Click the "clsTransformFormatStringToNumber" step to configure its "Required" properties. Enter the step name. Click the button to configure the transform list. Then click the Save button. 

• Click the "clsTransformFormatStringToNumber" step to configure the transform list. The configuration pop-up window appears. Click the Add Row (+) button to insert an empty row. Select the string as the input source column and “integer or value” as the output target column. Click the Save button. You may add multiple columns for transformation using the Add Row button. 

• Click the "clsOutputCSV" step to configure its "Required" properties. Enter a name for the step. Enter the path to the output CSV file on the application server. Click the Save button. 

• Click the "clsOutputCSV" step to configure its "Optional" properties. Provide a name for the step. Provide a variable or global reference to store the virtual output path. Click the Save button. 

• The “Logging” configuration is necessary for documentation and also measures workflow progress and percent complete. This is achieved by configuring the step state and percent fields individually, as shown below. Configure the “Logging” using the following properties.

• Save the ETL definition, create a new instance, and then execute it. Render the ETL instance. Click the Items—Variables menu option to view the variable values. The “clsInputCSV” step transforms the input file. The variable or global holds the output file’s virtual path, as shown below. 

Tips:

• Ensure the CSV file is accessible to the FlowWright ETL server.
• Verify that the selected column schema accurately reflects the CSV file structure.
• Use the Show Schema option to validate imported columns during ETL design.
• Validate imported datasets immediately after reading the CSV file to detect missing or invalid values.
• Keep column ordering consistent across recurring CSV imports.
• Use descriptive schema names that reflect the imported business data.
• Combine this step with Validate Data, Transform Text Case, Transform Date by Format, Filter with Conditions, Group By Transform, Sorting Transform, CSV Output File, SQL Output, or Excel Output File to build complete enterprise ETL workflows.

Notes:

• The CSV file must be accessible by the FlowWright ETL server.
• The selected column schema determines the structure of the generated ETL dataset.
• The Show Schema property helps verify imported columns during ETL design.
• File access issues, invalid CSV formatting, or schema mismatches may prevent successful execution.
• Because this is an ETL source component, it has no incoming connections and serves as the starting point of the ETL pipeline.
• ETL execution statistics, import validation results, and any processing errors are recorded in the ETL execution log.
• This step is commonly used for enterprise data migration, business reporting, ERP integration, CRM synchronization, financial imports, analytics, and operational data processing.

CSV Input File vs. Excel Input File:

Both components import external data into FlowWright ETL, but they are designed for different source formats.

As a general guideline:

• Use CSV Input File when importing data from enterprise systems, databases, or automated integrations that generate CSV files.
• Use Excel Input File when importing business data maintained in Microsoft Excel workbooks.

Definition Sample:

You may download the sample ETL definition(s) from the link provided and later import them into your FlowWright ETL Designer.

Note: Verify and complete any missing configuration after importing the sample, including:

• Input file location
• Column schema selection
• Schema validation
• Downstream transformation mappings
• Destination component configuration
• Environment-specific file paths

After verifying the configuration, save the ETL Definition before execution.

Click here to download the sample file.