For Each Step

The For Each Step is a container Step that executes exactly one child Step multiple times—once for each item in a provided array. This is particularly handy when you want to apply the same operation (e.g., fetching data, transforming inputs, sending requests) for multiple items.

Unlike a traditional loop, the For Each Step has a concurrency parameter that controls how many iterations can run in parallel. By default, it runs one iteration at a time (i.e., concurrency = 1). Increasing this value can speed up the overall process if parallelism is acceptable for your use case.

Key Points

1. Single Child Step
   The For Each Step must have exactly one child Step defined in its children.

2. Index-Based Invocations
   Each iteration of the child Step is given a unique reference:

   • childStepRef[0] for the first item,
   • childStepRef[1] for the second,
   • and so on.

3. Template Variables
   Within the child Step definition, you can access:

   • {{ foreach.index }} for the current loop index,
   • {{ foreach.value }} for the current item in the array.

4. Concurrency
   You can specify how many iterations to run in parallel by setting concurrency. For example, if you have 100 items and set concurrency = 10, then 10 items will be processed simultaneously until all 100 are complete.

5. Failure Handling
   If any single iteration fails, the For Each Step does not immediately stop other iterations. It processes all items and then marks the Step as failed. This ensures more deterministic behavior, preventing partial completions and leaving tasks “stuck.”

Validations

• Only One Child: If more than one child Step is defined, the For Each Step fails.
• Allowed Child Types: If a forbidden Step type is used as the child, the For Each Step fails.

Typical Usage

1. Add a For Each Step in your process definition.
2. Provide an array via inputArray.
3. Set concurrency to control parallel executions.
4. Define a single child Step that references {{ foreach.value }} and optionally {{ foreach.index }} for dynamic inputs.
5. Process subsequent Steps by referencing each iteration’s output using [index].

Script Step Inside FOREACH

When using a Script Step inside a FOREACH loop, you can access the current iteration's data. JavaScript uses context.foreach while Python uses context.get("foreach").

Workflow Example

The diagram below shows a FOREACH loop containing a Script step that can access the current iteration's data:

Script Step Inside FOREACH

Method 1: Using context.foreach (Recommended)

(steps, context) => {
    return {
        "value": context.foreach.value,
        "index": context.foreach.index
    }
}

Method 2: Direct Access

(steps, context) => {
    return {
        "value": foreach.value,
        "index": foreach.index
    }
}

Recommended Approach

Use context.foreach.value and context.foreach.index for better clarity and consistency with the platform's context system.

Practical Examples

JavaScript:

(steps, context) => {
    const currentItem = context.foreach.value;
    const currentIndex = context.foreach.index;
    
    // Process the current item
    return {
        "processedItem": currentItem,
        "position": currentIndex,
        "message": `Processing item ${currentIndex + 1}: ${currentItem.name || currentItem}`
    };
}

Python:

def main(steps, context):
    foreach_data = context.get("foreach")
    current_item = foreach_data.value
    current_index = foreach_data.index
    
    # Process the current item
    return {
        "processedItem": current_item,
        "position": current_index,
        "message": f"Processing item {current_index + 1}: {current_item.get('name', current_item) if isinstance(current_item, dict) else current_item}"
    }

Multi-Step FOREACH Loops

For complex workflows that require multiple steps within each FOREACH iteration, you can use a Sub Process step as the child of the FOREACH loop. This allows you to define a complete workflow with multiple steps that will be executed for each item in the array.

How It Works

1. Create a Sub Process Definition: Define a separate process with all the steps you want to execute for each iteration
2. Use Sub Process as Child: Set the FOREACH loop's child step to a Sub Process step that calls your sub process definition
3. Pass Iteration Data: The sub process receives the current iteration's data through its input parameters

Example Configuration

Main Process (with FOREACH):

{
  "type": "FOREACH",
  "name": "processItems",
  "input": {
    "inputArray": "{{ context.input.items }}",
    "concurrency": 3
  },
  "children": [
    {
      "type": "SUB_PROCESS",
      "name": "processSingleItem",
      "input": {
        "processName": "itemProcessingWorkflow",
        "waitForCompletion": true,
        "input": {
          "item": "{{ foreach.value }}",
          "index": "{{ foreach.index }}"
        }
      }
    }
  ]
}

Sub Process Definition (itemProcessingWorkflow):

{
  "steps": [
    {
      "type": "HTTP",
      "name": "fetchItemDetails",
      "input": {
        "url": "https://api.example.com/items/{{ context.input.item.id }}",
        "method": "GET"
      }
    },
    {
      "type": "SCRIPT",
      "name": "transformData",
      "input": {
        "script": "javascript",
        "scriptBody": "(steps, context) => { return { transformed: steps.fetchItemDetails.output.data }; }"
      }
    },
    {
      "type": "HTTP",
      "name": "saveResults",
      "input": {
        "url": "https://api.example.com/save",
        "method": "POST",
        "body": "{{ steps.transformData.output }}"
      }
    }
  ]
}

Benefits of Multi-Step FOREACH

• Modularity: Each iteration runs a complete, reusable workflow
• Complexity: Handle sophisticated business logic per item
• Error Isolation: Failures in one iteration don't affect others
• Resource Management: Better control over concurrency and resource usage

Learn More

For detailed information about Sub Process steps, see the Sub Process Step documentation.

Example: Gathering Outputs

After the For Each Step finishes, you often want to collect outputs from each iteration. Suppose the For Each Step is called foreachstep, and its child Step is forloop_http. Here's an example of a JavaScript Step that collects results from all iterations:

(steps, context) => {
   const allOutputs = [];
   for (let i = 0; i < steps.foreachstep.input.inputArray.length; i++) {
      allOutputs.push(
              steps[`forloop_http[${i}]`].output
      );
   }
   return allOutputs;
}

tip

foreachstep is the ref of the for-each step forloop_http is the ref of the child of the for-each

The naming convention of looped task is forloop_http[index] and this script is just using that convention to get the results in a single array!

Referencing Iteration Outputs

• Within an iteration: Use {{ foreach.index }} and {{ foreach.value }}.
• Outside an iteration: Use the child Step’s name plus [index] to retrieve the outcome of a specific iteration (e.g., fetch_process_context[2].output).

Sample Input

Using Template Variables

{
  "inputArray": "{{ context.input.items }}",
  "concurrency": 2
}

Static Array with Objects

{
  "inputArray": [
    {
      "id": 1,
      "name": "Item A"
    },
    {
      "id": 2,
      "name": "Item B"
    },
    {
      "id": 3,
      "name": "Item C"
    }
  ],
  "concurrency": 2
}

Static Array with Scalar Values

{
  "inputArray": [
    1,
    2,
    3
  ],
  "concurrency": 2
}