Passing Variables

Passing context and variables around steps is a core concept in the process execution pipeline. This is done by templating which dynamically replaces placeholders within inputs with values. This enables the seamless passing of variables between different steps of a process, ensuring that each step has access to the necessary data and context to execute correctly.

Overview

Unmeshed facilitates the dynamic injection of data into inputs by recognizing and replacing placeholders with actual values from various sources. This mechanism supports secure handling of sensitive information such as secrets, flexible configuration through variables, and data flow between different steps within a process.

Video Overview

Syntax for Passing Variables

Variables within inputs are denoted using double curly braces ({{ }}). The syntax allows for optional quotes and supports different data sources. Here's the syntax:

{{ variablePath }}

Parameter	Description
variablePath	The path indicating the source of the variable, such as secrets, variables, step references, or process inputs.

Variable Types

Secrets

Syntax:

{{ secrets.secretName }}

Description: References a secret value stored securely. Secrets are typically sensitive information like API keys, passwords, or tokens.

Usage Example:

{{ secrets.databasePassword }}

Example in JSON:

{
  "database": {
    "password": "{{ secrets.databasePassword }}"
  }
}

(Environment) Variables

Syntax:

{{ variables.variableName }}

Description: References a predefined variable within the Unmeshed Cluster. Variables can store configuration values or parameters that are reused anywhere across process definitions or steps. This is the equivalent of an environment variable

Usage Example:

{{ variables.retryCount }}

Example in JSON:

{
  "maxRetries": "{{ variables.retryCount }}"
}

Step References

Syntax:

{{ steps.stepRef.output.key }}

Description: References the output from a previous step in the process. This allows subsequent steps to utilize data generated by earlier steps.

Here is the full list of values you can refer to in each step reference

Field	Description
id	A unique identifier for the step context.
parentId	The identifier of the parent step context, if applicable.
parentExecutionId	The execution ID of the parent step, if applicable.
stepDefinitionSnapshot	A snapshot of the step definition, containing metadata and configuration for the step.
input	Key-value pairs representing the input data for this step.
output	Key-value pairs representing the output or results produced by this step.
status	The current status of the step Step Status.
workerId	The identifier of the worker assigned or responsible for executing the step, if present.
priority	A numeric value indicating the priority of this step, often used to determine execution order or importance.
optional	A boolean flag indicating whether this step is optional
start	A timestamp (in epoch milliseconds) indicating when the step started.
schedule	A timestamp (in epoch milliseconds) indicating when the step is scheduled to start or run.
updated	A timestamp (in epoch milliseconds) indicating the last time the step’s status or data was updated.
type	The type of the step.
ref	A reference to the step.
name	The name of the step.
namespace	The namespace under which this step definition belongs.

Usage Example:

{{ steps.dataProcessing.output.result }}

Example in JSON:

{
  "processedData": "{{ steps.dataProcessing.output.result }}"
}

Process Inputs

Syntax:

{{ context.input.inputKey }}

Description: References the inputs provided when initiating the process. These inputs can influence the behavior and execution of the process.

Here is the full list of values we support in the context

Key	Description
processId	The unique identifier for the process, often the same as processContext.getId().
id	An alias to the process id
requestId	A user supplied identifier for the incoming request triggering or associated with the process.
correlationId	A user supplied identifier used to correlate related processes or requests.
status	The current status of the process Process Status
created	The epoch millis date/time the process or context was initially created.
updated	The epoch millis date/time the process or context was last updated.
input	The input data or parameters provided for the process.
output	The output data or result generated by the process.
name	The name of the process definition.
namespace	The namespace under which this process definition belongs.
version	The version number of the process definition.
type	The type or category of the process definition (e.g., “API_ORCHESTRATION”, “STANDARD”).
authClaims	Claims from the token with which this process was ran.

Usage Example:

{{ context.input.userId }}

Example in JSON:

{
  "userId": "{{ context.input.userId }}"
}

Passing Variables Between Steps

To pass inputs into steps from other steps, use the appropriate syntax based on the source of the data. Here's how you can integrate different types of variables within your process steps:

Define the output in the Source Step: Ensure that the step generating the data exposes the necessary output. Reference the output in the Target Step: Use the steps syntax to access the output from the source step.

Example Workflow

Data Processing

Output from Source:

{
  "result": "Processed Data"
}

Target Input definition

{
  "result": "{{ steps.dataProcessing.output.result }}"
}

Usage Examples

Example 1: Using Secrets

Scenario: Injecting a database password securely into a configuration.

Template:

{
  "database": {
    "password": "{{ secrets.databasePassword }}"
  }
}

Resolved Output:

{
  "database": {
    "password": "s3cr3tP@ssw0rd"
  }
}

Example 2: Using Variables Scenario: Setting the maximum number of retries for a process.

Input Definition:

{
  "maxRetries": "{{ variables.retryCount }}"
}

Resolved Output:

{
  "maxRetries": 5
}

Example 3: Referencing Step Outputs Scenario: Using the result from a data processing step in a subsequent step.

{
  "processedData": "{{ steps.dataProcessing.output.result }}"
}

Resolved Output:

{
  "processedData": "Processed Data"
}

Example 4: Accessing Process Inputs Scenario: Utilizing user-provided input to customize a process step.

{
  "userId": "{{ context.input.userId }}"
}

Resolved Output:

{
  "userId": "user_12345"
}

Best Practices

• Secure Handling of Secrets: Always use the secrets prefix to handle sensitive information. Avoid exposing secrets in logs or error messages.
• Consistent Naming Conventions: Use clear and consistent names for variables, secrets, and step references to enhance readability and maintainability.
• Error Handling: Implement appropriate error handling to manage scenarios where a referenced variable or secret might be missing or invalid.
• Documentation: Document the purpose and usage of each variable and secret to aid team members in understanding the process flow.
• Avoid Hardcoding Values: Utilize variables and secrets instead of hardcoding values to make the process more flexible and secure.
• Validate Inputs: Ensure that the inputs provided to the process are validated to prevent issues during template resolution.
• Monitor and Log: Keep track of variable resolutions and monitor for any discrepancies or failures in the template resolution process.