Why data should be in your systems not in workflow processes

A best practice on data handling and management in the workflow process

Last published at: April 18th, 2024

Process automation does not exist without the integration and use of data. Any process whether it is manual or automated involves 2 paths. We discuss these 2 path options below:

  • Data flow
  • Control flow

 

Data flow controls how information travels from one step to the next step; and how the information is used/accessed throughout the process. The control flow controls what step you get to next, and how you get there based on workflow conditions or states.

 

Workflow processes use data throughout the lifetime of the process. Some data might be used for computation, some data for decision making, or data can even be transformed. In any of these cases, all data should reside in your systems, for these top reasons:

  • Security
  • Process performance

Your systems that hold data are secured; let data in your systems stay in your systems, but pass a pointer/reference key to the process so that if the process needs to access your data for decision-making, the process knows how to do that using the pointer/reference key.

With performance, the more weight/data that a process has to carry and maintain the greater chance the database and engine execution performance will decrease.  Our team sees this all the time within our customer's workflows; specifically, workflows that wake up every hour and process. These types of workflows execute 24 execution iterations per day where the workflow instance just keeps getting bigger and bigger.  FlowWright does have features around this issue to solve for it: it's called "Single Iteration Execution" where the workflow maintains only the last executed iteration. This makes the process "lean and mean", giving you the best performance.

The bottom line, keep the data in your systems and let workflow automation drive your processes using the data from your systems.