Skip to main content

Linux Agent Installation Guide

This document shows how to install and manage the Linux agent for Unmeshed Job Scheduling Platform. The agent is a Java-based application which runs natively on Linux machines and executes the scheduled jobs as configured on the Unmeshed Platform. The integration between agent and Unmeshed platform is protected by an access key and secret and happens over an HTTPS connection.

Prerequisites​

Java Runtime Environment (JRE)​

The Linux agent requires Java 22 or higher. The agent package includes a bundled JRE in the jdk folder, which is the recommended approach for consistent operation.

System Requirements​

  • Linux distribution (Ubuntu, CentOS, RHEL, Amazon Linux, etc.)
  • Minimum 2GB RAM (4GB recommended)
  • At least 4GB free disk space (for logs and temporary files)
  • Network access to the Unmeshed platform

Installation Files​

Prerequisites Package: unmeshed-prerequisites.zip​

This zip file contains the foundational components required for the agent to operate:

  • jdk/ - Java Development Kit folder containing the bundled JRE
  • processes.yaml - Configuration file for the agent worker processes
  • worker-logback.xml - Logging configuration for the worker processes
  • monitor.jar - Monitoring and process management application
  • start_monitor.sh - Shell script to start the monitoring service

Agent Binary Package: unmeshed-agent.zip​

This zip file contains the main agent execution files:

  • workers.jar - Main worker execution engine (will be created/updated)
  • monitor.jar - Monitoring application (optional, may be updated)
Request Access

Contact your Unmeshed support team to request download links for the following agent packages:

  • unmeshed-prerequisites.zip - Prerequisites package
  • unmeshed-agent.zip - Agent binary package

Installation​

The installation process involves two main steps: installing prerequisites and then the agent binary. We'll use /unmeshed/agent/ as the installation directory.

Step 1: Install Prerequisites​

  1. Create the installation directory:

    sudo mkdir -p /unmeshed/agent
    sudo chown $USER:$USER /unmeshed/agent
    cd /unmeshed/agent

  2. Download and extract the prerequisites package:

    # Download the prerequisites zip file
    wget <prerequisites-download-url>

    # Extract the contents
    unzip unmeshed-prerequisites.zip

    # Verify the extraction
    ls -la

    You should see the following files and directories:

    drwxr-xr-x  jdk/
    -rw-r--r--  processes.yaml
    -rw-r--r--  worker-logback.xml
    -rw-r--r--  monitor.jar
    -rw-r--r--  start_monitor.sh

  3. Set proper permissions:

    chmod +x start_monitor.sh

Step 2: Install Agent Binary​

  1. Download and extract the agent binary package:

    # Download the agent binary zip file
    wget <agent-download-url>

    # Extract the contents (this will create/update workers.jar and optionally monitor.jar)
    unzip unmeshed-agent.zip

    # Verify the extraction
    ls -la *.jar

    You should see at least:

    -rw-r--r--  workers.jar
    -rw-r--r--  monitor.jar (optional, may be updated)

Configuration​

Step 1: Configure the Agent Worker​

Edit the processes.yaml file to configure your agent:

vi processes.yaml

The file should contain the following configuration:

- command: "java -Xms1024m -Xmx1024m -Dlogback.configurationFile=./worker-logback.xml -jar workers.jar"
  commandSearch: "jar workers.jar"
  commandFile: "monitorworkers"
  env:
    UNMESHED_AUTH_ID: "<< auth id >>"
    UNMESHED_AUTH_TOKEN: "<< token >>"
    UNMESHED_ENGINE_URL: "<< server url >>"
    UNMESHED_BASH_CUSTOM_WORKERS: ""
    UNMESHED_BASH_MAX_CONCURRENT_WORKERS: 1000
    UNMESHED_ENGINE_EMAIL_ENABLED: "false"
    UNMESHED_EMAIL_SMTP_HOST: ""
    UNMESHED_EMAIL_SMTP_PORT: ""
    UNMESHED_EMAIL_SMTP_USER: ""
    UNMESHED_EMAIL_SMTP_PASSWORD: ""
    UNMESHED_WORK_RESPONSE_BATCH_SIZE: 120

Configuration Parameters​

FieldDescription
UNMESHED_AUTH_IDThis is a generated field from the Unmeshed Platform when you create a client application
UNMESHED_AUTH_TOKENThis is a generated field from the Unmeshed Platform when you create a client application
UNMESHED_ENGINE_URLHTTPS URL for the Unmeshed server the agent should connect to
UNMESHED_BASH_CUSTOM_WORKERSAdvanced Configuration: Comma-separated list of custom worker specifications in the format namespace:name:maxConcurrentWorkers:false

Example: default:custom-bash:50,prod:high-priority:100

Format: Each specification contains 3 parts separated by colons:
- namespace: The namespace for this worker (e.g., "default", "prod")
- name: The worker name identifier
- maxConcurrentWorkers: Maximum number of concurrent jobs this worker can handle

Default Behavior: If left empty, the agent registers a default bash worker under the "default" namespace with the name "bash"
UNMESHED_BASH_MAX_CONCURRENT_WORKERSMaximum number of concurrent bash commands that can be executed simultaneously
UNMESHED_ENGINE_EMAIL_ENABLEDEnable/disable email sender worker
UNMESHED_EMAIL_SMTP_HOSTSMTP server hostname for sending email
UNMESHED_EMAIL_SMTP_PORTSMTP server port (usually 25, 465, or 587)
UNMESHED_EMAIL_SMTP_USERSMTP server username for authentication
UNMESHED_EMAIL_SMTP_PASSWORDSMTP server password for authentication
UNMESHED_WORK_RESPONSE_BATCH_SIZENumber of work responses to batch together before sending to the server

UNMESHED_BASH_CUSTOM_WORKERS Configuration​

The UNMESHED_BASH_CUSTOM_WORKERS parameter allows you to create multiple custom bash workers with different configurations. This is useful when you need different concurrency limits or want to separate workloads by namespace.

note

For Linux agents, the ioThread parameter must always be set to false.

Format​

Each worker specification follows the format: namespace:name:maxConcurrentWorkers:false

  • namespace: The namespace for this worker (e.g., "default", "prod", "dev")
  • name: The worker name identifier that will be used in job definitions
  • maxConcurrentWorkers: Maximum number of concurrent jobs this worker can handle
  • ioThread: Must be set to false for Linux agents (required parameter)

Examples​

Single Custom Worker:

UNMESHED_BASH_CUSTOM_WORKERS: "prod:high-priority-bash:100:false"

This configuration will query for jobs in the prod namespace with the name high-priority-bash and can run up to 100 concurrent tasks.

Multiple Custom Workers:

UNMESHED_BASH_CUSTOM_WORKERS: "default:standard-bash:50:false,prod:high-priority-bash:100:false,dev:testing-bash:25:false"

This configuration creates three workers:

  • default:standard-bash:50:false - Namespace default, jobs named standard-bash, concurrency of 50
  • prod:high-priority-bash:100:false - Namespace prod, jobs named high-priority-bash, concurrency of 100
  • dev:testing-bash:25:false - Namespace dev, jobs named testing-bash, concurrency of 25

No Custom Workers (Default):

UNMESHED_BASH_CUSTOM_WORKERS: ""
# This registers a default bash worker under "default" namespace with name "bash"

Naming Recommendations​

Use descriptive names that represent your host machine's purpose:

  • Environment-based: prod-web-server-bash, staging-db-bash, dev-testing-bash
  • Function-based: batch-processing-bash, real-time-bash, backup-server-bash
  • Location-based: us-east-1-bash, eu-west-1-bash, on-premise-bash

Why unique names matter: Using unique, descriptive names ensures that specific jobs will always run on the intended machine. For example, if you have a backup server with UNMESHED_BASH_CUSTOM_WORKERS: "default:backup-server-bash:10:false", only jobs defined with the name backup-server-bash in the default namespace will be executed on that machine.

Use Cases​

  • Environment Separation: Use different namespaces for dev, staging, and production
  • Priority Management: Create high-priority workers with higher concurrency limits
  • Resource Isolation: Separate different types of workloads (e.g., batch processing vs. real-time tasks)
  • Testing: Dedicated workers for testing with lower concurrency limits

Step 2: Configure the Monitor Script​

Edit the start_monitor.sh file to set the correct working directory:

vi start_monitor.sh

Ensure the CURRENT_DIR variable is set correctly:

export CURRENT_DIR=/unmeshed/agent

Important: Do not modify any other parts of this script unless you have specific requirements.

System Service Setup​

Step 1: Create the Systemd Service​

Create a systemd service file to manage the Unmeshed agent:

sudo vi /etc/systemd/system/unmeshed.service

Add the following content:

[Unit]
Description=Unmeshed Monitor
After=network.target

[Service]
Type=simple
WorkingDirectory=/unmeshed/agent/
ExecStart=/bin/bash -c '/unmeshed/agent/start_monitor.sh'
Restart=always
RestartSec=2
User=ec2-user
StartLimitBurst=0

[Install]
WantedBy=multi-user.target

Important Notes:

  • User Field: Update User=ec2-user to the actual user that should run the process
  • Working Directory: Ensure the path matches your installation directory
  • Permissions: The specified user must have read/write access to the /unmeshed/agent/ directory

Step 2: Enable and Start the Service​

# Reload systemd to recognize the new service
sudo systemctl daemon-reload

# Enable the service to start on boot
sudo systemctl enable unmeshed.service

# Start the service
sudo systemctl start unmeshed.service

Step 3: Verify Service Status​

# Check service status
sudo systemctl status unmeshed.service

# View service logs
sudo journalctl -u unmeshed.service -f

# Check if the agent is running
ps aux | grep java

Verification​

Check Agent Logs​

The agent creates logs in the working directory. Look for the following log entry to confirm successful startup:

tail -f /unmeshed/agent/worker.log

You should see a log entry similar to:

[date - time] [pool-3-thread-1] INFO io.unmeshed.client.UnmeshedClient - Currently running

Check Service Status​

# Service status
sudo systemctl status unmeshed.service

# Service logs
sudo journalctl -u unmeshed.service --no-pager -l

Operating the Unmeshed Agent​

Service Management Commands​

# Start the service
sudo systemctl start unmeshed.service

# Stop the service
sudo systemctl stop unmeshed.service

# Restart the service
sudo systemctl restart unmeshed.service

# Check service status
sudo systemctl status unmeshed.service

# View real-time logs
sudo journalctl -u unmeshed.service -f

Manual Process Management​

If you need to run the agent manually (for debugging purposes):

cd /unmeshed/agent
./start_monitor.sh

Upgrading the Unmeshed Agent​

Follow these steps to upgrade:

  1. Stop the service:

    sudo systemctl stop unmeshed.service

  2. Download and extract the new agent binary:

    cd /unmeshed/agent
    wget <new-agent-download-url>
    unzip unmeshed-agent.zip

  3. Start the service:

    sudo systemctl start unmeshed.service

  4. Verify the upgrade:

    sudo systemctl status unmeshed.service
    tail -f /unmeshed/agent/worker.log

Configuration Updates​

note

The processes.yaml configuration file is not included in agent upgrades. If you need to update your configuration or add new parameters, please contact the Unmeshed support team for guidance on any required changes.

Troubleshooting​

Common Issues​

  1. Service fails to start:

    • Check file permissions: ls -la /unmeshed/agent/
    • Verify Java installation: java -version
    • Check service logs: sudo journalctl -u unmeshed.service -n 50

  2. Agent cannot connect to Unmeshed platform:

    • Verify network connectivity: ping <your-unmeshed-server>
    • Check authentication credentials in processes.yaml
    • Verify firewall settings

  3. Permission denied errors:

    • Ensure the service user owns the /unmeshed/agent/ directory
    • Check file permissions: chmod +x /unmeshed/agent/start_monitor.sh

  4. Java memory issues:

    • Adjust JVM memory settings in processes.yaml:
      • -Xms1024m (minimum heap size)
      • -Xmx1024m (maximum heap size)

Log Analysis​

The agent creates several log files:

  • worker.log - Main worker process logs
  • monitor.log - Monitor process logs
  • systemd logs - Service management logs
# View worker logs
tail -f /unmeshed/agent/worker.log

# View systemd service logs
sudo journalctl -u unmeshed.service -f

# Search for specific errors
grep -i error /unmeshed/agent/worker.log

Security Considerations​

  1. File Permissions: Ensure only authorized users can access the agent configuration files
  2. Network Security: Use HTTPS connections to the Unmeshed platform
  3. Authentication: Keep your auth ID and token secure and rotate them regularly
  4. Service User: Run the service with minimal required privileges
  5. Firewall: Configure firewall rules to allow only necessary outbound connections

Performance Tuning​

Memory Configuration​

Adjust JVM memory settings based on your system resources:

command: "java -Xms2048m -Xmx4096m -Dlogback.configurationFile=./worker-logback.xml -jar workers.jar"

Concurrency Settings​

Adjust based on your system's CPU cores and workload:

UNMESHED_BASH_MAX_CONCURRENT_WORKERS: 2000

Batch Processing​

Optimize network communication:

UNMESHED_WORK_RESPONSE_BATCH_SIZE: 200

FAQs & Notes​

  1. Service Auto-restart: The systemd service is configured to automatically restart if the process crashes
  2. Log Rotation: Consider implementing log rotation for long-running deployments
  3. Monitoring: Set up monitoring for the service status and resource usage
  4. Backup: Regularly backup your processes.yaml configuration file
  5. Updates: Keep your Linux system updated for security patches
  6. Resource Monitoring: Monitor CPU, memory, and disk usage during operation