Working Data

 

Working Data

A Working Data collection represents an in-memory data set that has been loaded into the Working Data Container.

Working Data collections can be interacted with via workflow Actions.

Working Data Container

The Working Data Container is the in-memory data storage used by Actions during workflow execution.

Actions
working data actions

Data Point Filter

Required Parameters:

  • Filter Data
  • Match Data
  • Comparison Operator
  • Input 1
  • Input 2

Optional Parameters:

  • Non-Match
  • Remove From Working Data
  • Ignore Case

Description:

The Data Point Filter action filters rows of Working Data collection based on the configured comparison logic into the Match Data collection.

How to Perform a Data Point Filter on a Data Collection
Step Description

1

Select the Working Data collection you wish to filter.

Note - We will be using the Product Sales Sample Dataset as our Filter Data collection.

product sales sample dataset

2

Check Remove from Working Data if you wish to remove the Filter Data collection after performing the filter (optional).

3

Enter a Name for the Match Data collection that will store the results that match the filter criteria.

4

Check Non-Match and enter a Name for the collection that will store the results that do not match the filter criteria (optional).

5

Select an Input 1. This will be the left side of the comparison.

6

Select a Comparison Operator:

  • Less Than
  • Less Than or Equal
  • Equal
  • Greater Than or Equal
  • Greater Than
  • Not Equal
  • Less Than
  • Is Null
  • Is Not Null

7

Select an Input 2. This will be the right side of the comparison.

8

Check Ignore Case if casing does not matter (for strings/characters generally) (optional).

9

Click the green (+) to add the filter criteria to the action.

10

For complex filters...

Select a logical operator

  • And
  • Or
  • Nand
  • Nor
  • Xor

Fill out the filter criteria as outlined in the above steps

Click the green (+) to add the additional filter to the action

11

Click OK.

data-point-filter-how-to-1

Running the action...

data-point-filter-how-to-2

Index Based Row Filter

Required Parameters:

  • Working Data
  • Return From Index
  • To Index
  • Result Name

Description:

The Index Based Row Filter action filters out an index based range of rows (starting from 0) into a new Working Data collection.

How to Filter a Range of Rows from a Data Collection
Step Description

1

Select a Working Data collection.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

There is an added Index Data Point to illustrate the result of the action.

product sales sample dataset

2

Specify a Start Index (0-based).

3

Specify an End Index (up to, but not including).

4

Name the Result Collection.

5

Click OK.

index-based-row-filter-how-to-1

Running the action...

index-based-row-filter-how-to-2

Top N Filter

Required Parameters:

  • Working Data
  • Data Point
  • Filter Type
  • Select Numeric Value For N OR Specify Percentage For N
  • Result Name

Description:

The Top N Filter action filters the top N greatest or least values based on a numeric Data Point into a new Working Data collection.

The user has the option to either:

- Specify a discrete number of rows to select (Select Numeric Value For N).

OR

- Enter a percentage in which the action will dynamically determine the value for N (Specify Percentage For N).

How to Filter the Top N Values of a Data Point in a Data Collection
Step Description

1

Select a Working Data collection.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

There is an added Index Data Point to illustrate the result of the action.

product sales sample dataset

2

Enter a Data Point to base the Top N filter on.

3

Select a Filter Type.

  • Filter Top N
  • Filter Bottom N

4

Specify a Numeric Value For N.

OR

Specify a Percentage For N

5

Name the Result Collection.

6

Click OK.

top-n-filter-how-to-1

Running the action...

top-n-filter-how-to-2

Sample Data

Required Parameters:

  • Target Collection
  • Sample Size
  • Allow Repeats
  • Result Collection Name

Description:

The Sample Data action randomly filters a sample of rows from the Target Collection into a new Working Data collection.

How to Randomly Sample a Data Collection
Step Description

1

Select the Working Data collection you wish to add a new Data Point to.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

There is an added Index Data Point to illustrate the result of the action.

product sales sample dataset

2

Set a Sample Size.

3

Select Yes or No for Allow Repeats.

4

Name the Result Collection.

5

Click OK.

sample-data-how-to-1

Running the action...

sample-data-how-to-2

Pop Chunk

Required Parameters:

  • Collection Name
  • Number of Items
  • Result Collection

Optional Parameters:

  • Use Var

Description:

The Pop Chunk action removes the specified number of rows off the top of the targeted Working Data collection and appends them into either a new, or existing, Working Data collection.

How to Pop a Chunk of Rows off the Top of a Data Collection
Step Description

1

Select the Working Data collection you wish to add a new Data Point to.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

There is an added Index Data Point to illustrate the result of the action.

product sales sample dataset

2

Set a Number of Items.

OR

Check Use Var and select a Variable

3

Name the Result Collection.

4

Click OK.

pop-chunk-how-to-1

Running the action...

pop-chunk-how-to-2
pop-chunk-how-to-3

Remove Data

Required Parameters:

  • Dataset to Remove (at least 1)

Description:

The Remove Data action removes one or more Working Data collections from memory.

How to Remove a Data Collection
Step Description

1

Select the Working Data collection that you wish to remove.

2

Click the green (+) to add the removed Working Data collection to the action.

3

Click OK.

Rename Data

Required Parameters:

  • Working Data
  • New Working Data Name

Description:

The Rename Data action renames one or more Working Data collections.

How to Rename a Data Collection
Step Description

1

Select the Working Data collection you wish to rename.

2

Enter a new name for the Working Data collection.

3

Click the green (+) to add the renamed Working Data collection to the action.

4

Click OK.

Sort

Required Parameters:

  • Working Data
  • Data Point
  • Ascending OR Descending

Description:

The Sort action sorts a Working Data collection based on a targeted Data Point in either ascending or descending order.

How to Sort a Data Collection
Step Description

1

Select the Working Data collection that contains the targeted Data Point(s).

2

Select the targeted Data Point.

3

Select either Ascending or Descending as the sort order.

4

Click the green (+) to add the configured Sort Criteria to the action.

5

Click OK.

Slice

Required Parameters:

  • Working Data
  • Slice Name
  • Data Point
  • New Name

Optional Parameters:

  • Full Copy
  • Remove Source Data
  • Convert To
  • Add Multiple Data Points

Description:

The Slice action slices one or more Data Point(s) within a Working Data collection into a new Working Data collection.

How to Slice a Data Collection
Step Description

1

Select the Working Data collection.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

product sales sample dataset

2

Select Full Copy to slice all Data Points contained within the Working Data collection. (optional)

3

Select Remove Source Data to remove the original Working Data collection that contains the targeted Data Point(s). (optional)

4

Enter a name for the new Working Data collection that will contain the sliced Data Point(s).

5

Select the targeted Data Point.

6

Enter a new name for the sliced Data Point or leave New Name as is to retain the original name.

7

Select a Conversion Type to convert the sliced Data Point to a new Data Type. (optional)

8

Click the green (+) to add the sliced Data Point to the action.

9

Optionally, the user can configure the Slice action to start at a specified Data Point and add the next N Data Points to the action.

Note - The Data Point in the left-most position is 1.

10

Click OK.

slice-how-to-1

Running the action...

slice-how-to-2

Append

Required Parameters:

  • Append From
  • Append Into

Optional Parameters:

  • Remove From Group

Description:

The Append action appends a Working Data collection into another Working Data collection.

The two Working Data collections do not need to have the same Data Points. The result collection will contain all Data Points contained between the two collections.

How to Append a Data Collection Into Another Data Collection
Step Description

1

Select the Working Data collection you wish to append data from.

2

Select the Working Data collection you wish to append data into.

3

Check Remove From Group if you wish to remove the Append From collection (optional).

4

Click OK.

Deduplicate

Required Parameters:

  • Working Data
  • Function
  • Compare Data Point(s)
  • Result Name

Optional Parameters:

  • Compare all datapoints

Description:

The Deduplicate action deduplicates the values contained in a set of Data Point within a Working Data collection.

When muliple duplicate values are found, the top row is selected and inserted into a Result Collection.

How to Deduplicate Data
Step Description

1

Select the Working Data collection you wish to add a new Data Point to.

Note - We will be using the Product Sales Sample Dataset as our Working Data collection.

We have sorted (ascending) the Working Data collection according to the Country Data Point.

product-sales-sample-data-set-sorted-by-country

2

Select a Function.

  • Exact Comparison (Unordered)
  • Exact Comparison (Ordered)

Note - It is strongly suggested to run an Ordered comparison. This decreases the complexity of the action and improves run time. A Sort action on the Compare Data Point(s) is required beforehand.

3

Check Compare all Data Points if you wish to deduplicate rows across all Data Points (optional).

4

Select a Data Point and click the green (+) to add the Compared Data Point to the action.

5

Click OK.

deduplicate-how-to-1

Running the action...

deduplicate-how-to-2

Denormalize

Required Parameters:
  • Parent Set
  • Child Set
  • Join Type
  • Parent Key
  • Child Key
Optional Parameters:
  • Remove Child Set
  • Save Result to New Data Set
Description:

The Denormalize action joins two normalized Working Data collections into a single, denormalized Working Data collection.

How to Denormalize Data
Step Description

1

Select a Parent Set.

2

Select a Child Set.

3

Select a Join Type:

  • Keep Matched and Unmatched = FULL OUTER JOIN
  • Remove Unmatched = INNER JOIN
  • Remove Matched = OUTER JOIN

4

Select a Parent Key.

5

Select a Child Key.

6

Click the green (+) to add the JOIN mapping to the action.

For complex JOINS, you may add additional Data Point mappings to the action.

7

Check Remove Child Set if you wish to remove the Child Set (optional).

8

Check Save Result to New Data Set if you wish to save the denormalized data to a new Working Data collection (optional).

If you do not check this option, the Child Set will be denormalized into the Parent Set.

9

Click OK.

We will use the following data to illustrate the denormalization process:

denormalization-parent-set
denormalization-child-set

Configuring the action as follows:

denormalization-how-to-1

Produces the following result:

denormalization-how-to-1-1

Configuring the action as follows:

deduplicate-how-to-2

Produces the following result:

denormalization-how-to-2-1

Lookup

Required Parameters:

  • Working Data
  • Data Point Name
  • Data Point Type

Optional Parameters:

  • Default Value

Description:

The Add Data Points action creates a new Data Point within a Working Data collection.

How to Lookup Data
Step Description

1

Select the Working Data collection that contains the targeted Data Point.

2

Select the targeted Data Point.

3

Input a new value or leave Replace With blank for a null value.

5

Click OK.