Unmeshed Logo

Command Runners (Agents)

Linux Agent Installation Guide

This document shows how to install and manage the Linux agent for Unmeshed Job Scheduling Platform. The agent is a Java-based application which runs natively on Linux machines and executes the scheduled jobs as configured on the Unmeshed Platform. The integration between agent and Unmeshed platform is protected by an access key and secret and happens over an HTTPS connection.

Prerequisites

Unmeshed CLI

To add an agent, we require the installation of Unmeshed CLI. You can find the guide here.

Installation

Run the following commands and follow the prompts to complete the agent installation. It will download the required files automatically and place them in your selected folder.0

bash

To uninstall the agent, you can use the CLI

bash

The CLI has other commands that can list agents installed in the host. You'd only need a single agent installation typically, but if you need to install multiple agents for any reason, the CLI supports this in its prompts.

Configuration

Step 1: Configure the Agent Worker

Go to the installation folder for your agent. Edit the processes.yaml file to configure your agent:

bash

The file should contain the following configuration:

yaml

Configuration Parameters

FieldDescription
UNMESHED_AUTH_IDThis is a generated field from the Unmeshed Platform when you create a client application
UNMESHED_AUTH_TOKENThis is a generated field from the Unmeshed Platform when you create a client application
UNMESHED_ENGINE_URLHTTPS URL for the Unmeshed server the agent should connect to
UNMESHED_BASH_CUSTOM_WORKERSAdvanced Configuration: Comma-separated list of custom worker specifications in the format namespace:name:maxConcurrentWorkers:false

Example: default:custom-bash:50,prod:high-priority:100

Format: Each specification contains 3 parts separated by colons:
- namespace: The namespace for this worker (e.g., "default", "prod")
- name: The worker name identifier
- maxConcurrentWorkers: Maximum number of concurrent jobs this worker can handle

Default Behavior: If left empty, the agent registers a default bash worker under the "default" namespace with the name "bash"
UNMESHED_BASH_MAX_CONCURRENT_WORKERSMaximum number of concurrent bash commands that can be executed simultaneously
UNMESHED_ENGINE_EMAIL_ENABLEDEnable/disable email sender worker
UNMESHED_EMAIL_SMTP_HOSTSMTP server hostname for sending email
UNMESHED_EMAIL_SMTP_PORTSMTP server port (usually 25, 465, or 587)
UNMESHED_EMAIL_SMTP_USERSMTP server username for authentication
UNMESHED_EMAIL_SMTP_PASSWORDSMTP server password for authentication
UNMESHED_WORK_RESPONSE_BATCH_SIZENumber of work responses to batch together before sending to the server

UNMESHED_BASH_CUSTOM_WORKERS Configuration

The UNMESHED_BASH_CUSTOM_WORKERS parameter allows you to create multiple custom bash workers with different configurations. This is useful when you need different concurrency limits or want to separate workloads by namespace.

Note

For Linux agents, the ioThread parameter must always be set to false.

Format

Each worker specification follows the format: namespace:name:maxConcurrentWorkers:false

  • namespace: The namespace for this worker (e.g., "default", "prod", "dev")
  • name: The worker name identifier that will be used in job definitions
  • maxConcurrentWorkers: Maximum number of concurrent jobs this worker can handle
  • ioThread: Must be set to false for Linux agents (required parameter)

Examples

Single Custom Worker:

yaml

This configuration will query for jobs in the prod namespace with the name high-priority-bash and can run up to 100 concurrent tasks.

Multiple Custom Workers:

yaml

This configuration creates three workers:

  • default:bash.servername.standard:50:false - Namespace default, jobs named standard-bash, concurrency of 50
  • prod:bash.servername.priority:100:false - Namespace prod, jobs named high-priority-bash, concurrency of 100
  • dev:bash.servername.testing:25:false - Namespace dev, jobs named testing-bash, concurrency of 25
Server Name

Since your agents can run in any server, using the server name in the bash worker name is a good way to distinctly identify the agent running on this host and you can assign tasks by this name to run on this host.

No Custom Workers (Default):

yaml

Verification

Check Agent Logs

The agent creates logs in the working directory. Look for the following log entry to confirm successful startup:

bash

You should see a log entry similar to:

Check Service Status

bash

Operating the Unmeshed Agent

Service Management Commands

bash

Troubleshooting

Common Issues

  1. Service fails to start:

    • Check file permissions: ls -la <directory name used in installation>
    • Verify Java installation: java -version
    • Check service logs: sudo journalctl -u <service name used in installation> -n 50
  2. Agent cannot connect to Unmeshed platform:

    • Verify network connectivity: ping <your-unmeshed-server>
    • Check authentication credentials in processes.yaml
    • Verify firewall settings
  3. Permission denied errors:

    • Ensure the service user owns the <directory used in installation> directory
    • Check file permissions: chmod +x <directory used in installation>/start_monitor.sh
  4. Java memory issues:

    • Adjust JVM memory settings in processes.yaml:
      • -Xms1024m (minimum heap size)
      • -Xmx1024m (maximum heap size)

Log Analysis

The agent creates several log files:

  • worker.log - Main worker process logs
  • monitor.log - Monitor process logs
  • systemd logs - Service management logs
bash

Security Considerations

  1. File Permissions: Ensure only authorized users can access the agent configuration files
  2. Network Security: Use HTTPS connections to the Unmeshed platform
  3. Authentication: Keep your auth ID and token secure and rotate them regularly
  4. Service User: Run the service with minimal required privileges
  5. Firewall: Configure firewall rules to allow only necessary outbound connections

Concurrency Settings

Adjust based on your system's CPU cores and workload:

yaml

Batch Processing

Optimize network communication:

yaml

FAQs & Notes

  1. Service Auto-restart: The systemd service is configured to automatically restart if the process crashes
  2. Log Rotation: Consider implementing log rotation for long-running deployments
  3. Monitoring: Set up monitoring for the service status and resource usage
  4. Backup: Regularly backup your processes.yaml configuration file
  5. Updates: Keep your Linux system updated for security patches
  6. Resource Monitoring: Monitor CPU, memory, and disk usage during operation