Making It Real

What you’ll accomplish: Build practical Rundeck jobs that solve real home lab problems — a rolling service restart, a parameterized system update workflow, scheduled backups, and access control for a small team.

You have a working Rundeck instance connected to Ansible with SSH access to your nodes. That’s the foundation. Now let’s build something useful on top of it.

This chapter is where Rundeck stops being a tool you installed and starts being a tool you use. Each example demonstrates a different Rundeck capability, and by the end you’ll have a set of jobs that cover the most common home lab operations tasks.

Example 1: Rolling Service Restart

The scenario: you have a service (say, httpd) running on multiple hosts and you want to restart it one node at a time, with a pause between each, so you don’t take everything down simultaneously.

Create the Job

In your homelab project, go to Jobs > Create a New Job.

Details Tab

1. Job Name: Rolling Service Restart
2. Group: Leave blank (or enter a group name like Operations to organize jobs later).
3. Description: Restart a service across selected nodes one at a time with a configurable delay

Workflow Tab — Job Options

Options make jobs reusable. Instead of hardcoding “httpd”, let the operator specify which service to restart.

Under Options, click Add Option:

Add a second option:

Workflow Tab — Playbook Step

1. Still on the Workflow tab, click Add a Step.
2. Under Node Steps, look under Node Step Plugins and select Ansible Playbook Inline Workflow Node Step.
3. Paste the following playbook into the inline playbook field:

---
- name: Rolling service restart
  hosts: all
  serial: 1
  tasks:
    - name: "Restart {{ service_name }}"
      ansible.builtin.systemd:
        name: "{{ service_name }}"
        state: restarted
      become: true

    - name: "Verify {{ service_name }} is running"
      ansible.builtin.systemd:
        name: "{{ service_name }}"
      register: svc_status
      become: true

    - name: Fail if service did not start
      ansible.builtin.fail:
        msg: "{{ service_name }} failed to restart on {{ inventory_hostname }}"
      when: svc_status.status.ActiveState != "active"

    - name: Wait between nodes
      ansible.builtin.pause:
        seconds: "{{ delay_seconds | int }}"
      when: ansible_play_hosts | length > 1

4. In the Extra Variables field, add the following YAML to pass the job options to the playbook as Ansible variables:

service_name: ${option.service_name}
delay_seconds: ${option.delay_seconds}

5. Leave Authentication and Privilege Escalation at their defaults. SSH key auth and become are handled by Ansible’s own config and the playbook’s become: true directive, not Rundeck’s step-level settings.
6. Click Save to save the step.

Nodes Tab

1. Switch to the Nodes tab.
2. Under Node Filter, enter a filter to target your hosts. For example: tags: rocky or name: web.* or just .* for all nodes.
3. Set Thread Count to 1 — this ensures nodes are processed sequentially, not in parallel.
4. Click Create to save the job.

Test It

Run the job with the defaults. In the execution log, you’ll see Rundeck restart the service on the first node, verify it came back up, wait the configured delay (the delay_seconds option), then move to the next node. If any node fails the verification step, the job stops automatically — you don’t want a bad restart cascading across your fleet.

Example 2: Parameterized System Update

A common home lab task: update packages across some or all of your hosts. You want control over what gets updated and where.

Create the Job

1. Job Name: System Update
2. Description: Update packages on selected hosts with optional targeting

Options

For the dry_run option, set Allowed Values to: true,false.

Workflow

Under Node Steps > Node Step Plugins, select Ansible Playbook Inline Workflow Node Step and paste:

---
- name: System package update
  hosts: "{{ target_group }}"
  become: true
  tasks:
    - name: Check for available updates (dry run)
      ansible.builtin.dnf:
        name: "{{ package }}"
        state: latest
        security: false
      check_mode: true
      register: update_check
      when: dry_run | bool

    - name: Show available updates
      ansible.builtin.debug:
        msg: "Updates available: {{ update_check.results | default(['None']) }}"
      when: dry_run | bool

    - name: Apply updates
      ansible.builtin.dnf:
        name: "{{ package }}"
        state: latest
      register: update_result
      when: not (dry_run | bool)

    - name: Show update results
      ansible.builtin.debug:
        msg: "Updated packages: {{ update_result.results | default(['No changes']) }}"
      when: not (dry_run | bool)

In the Extra Variables field, add:

target_group: ${option.target_group}
package: ${option.package}
dry_run: ${option.dry_run}

I recommend always running with dry_run=true first. Review the output, make sure you’re not about to update something that will break a service, then run again with dry_run=false.

Scheduling Jobs

Rundeck has built-in cron-like scheduling. You don’t need external cron entries or systemd timers — it’s all managed through the UI with execution history and notifications.

Example: Nightly Database Backup

1. Create a new job:

   • Job Name: Rundeck Database Backup
   • Description: Dump the Rundeck MariaDB database to a timestamped file

2. Workflow step (Ansible Playbook, inline):

---
- name: Backup Rundeck database
  hosts: localhost
  connection: local
  become: true
  vars:
    backup_dir: /var/backups/rundeck
    backup_retention_days: 7
  tasks:
    - name: Ensure backup directory exists
      ansible.builtin.file:
        path: "{{ backup_dir }}"
        state: directory
        owner: root
        group: root
        mode: '0750'

    - name: Dump Rundeck database
      ansible.builtin.shell: >
        mysqldump --single-transaction
        -u rundeck -p'{{ vault_rundeck_db_password }}'
        rundeck_db > {{ backup_dir }}/rundeck_db_$(date +%Y%m%d_%H%M%S).sql
      no_log: true

    - name: Remove backups older than retention period
      ansible.builtin.find:
        paths: "{{ backup_dir }}"
        patterns: "rundeck_db_*.sql"
        age: "{{ backup_retention_days }}d"
      register: old_backups

    - name: Delete old backup files
      ansible.builtin.file:
        path: "{{ item.path }}"
        state: absent
      loop: "{{ old_backups.files }}"

3. Under the Schedule tab:
   • Set Schedule to run repeatedly? to Yes
   • Set the cron expression: 0 0 2 * * ? (runs daily at 2:00 AM)
   • Or use the visual scheduler to pick day/time

The job will now run automatically every night. You can see execution history, review output, and get notified on failures.

Notifications

When a scheduled job fails at 2 AM, you want to know about it before you find out the hard way. Rundeck supports notifications via email, webhooks, and plugins.

Email Notifications

1. Edit the backup job (or any job).
2. Go to the Notifications tab.
3. Add a notification:
   • Trigger: On Failure
   • Type: Send Email
   • Recipients: admin@example.com
   • Subject: [Rundeck] Job Failed: ${job.name}

Email notifications require SMTP configuration. Add these to /etc/rundeck/rundeck-config.properties:

# /etc/rundeck/rundeck-config.properties (add to existing file)
grails.mail.host = smtp.example.com
grails.mail.port = 587
grails.mail.username = rundeck@example.com
grails.mail.password = your-smtp-password-here
grails.mail.props.mail.smtp.auth = true
grails.mail.props.mail.smtp.starttls.enable = true

Important: In production, store this password in your Ansible vault as vault_rundeck_smtp_password. Never leave credentials in plaintext config files that touch version control.

Restart Rundeck after adding mail settings. Send a test notification before relying on it — email delivery is notoriously finicky, and you don’t want to discover your alerts are broken during an actual incident.

Webhook Notifications

For Discord, Slack, or any service that accepts webhooks:

1. Trigger: On Failure (or On Success, or both)
2. Type: Send Webhook
3. URL: https://hooks.example.com/your-webhook-url
4. Payload: Rundeck sends a JSON payload with job name, execution status, and output by default.

Webhooks are the more flexible option. You can point them at a Discord channel webhook URL or a Slack incoming webhook without any plugins.

Access Control

If you’re the only person using Rundeck, the admin account is fine. But if you share your home lab with a partner, a roommate, or want to give someone read-only access to trigger predefined jobs, Rundeck’s ACL (Access Control List) system handles this.

The Concept

Rundeck ACL policies are YAML files in /etc/rundeck/. Each file defines what a user or group can do. The default admin.aclpolicy grants full access to users in the admin group.

Creating an Operator Role

Operators can run existing jobs and view output but can’t create, edit, or delete jobs. This is perfect for a “run the update but don’t touch the config” scenario.

First, add the user to realm.properties:

# /etc/rundeck/realm.properties
admin:your-admin-password-here,user,admin
operator:operator-password-here,user,operators

Then create the ACL policy:

sudo vi /etc/rundeck/operators.aclpolicy

# /etc/rundeck/operators.aclpolicy
description: Operators can run and view jobs but not modify them
context:
  project: '.*'
for:
  resource:
    - allow: [read]
  adhoc:
    - deny: [run, kill]
  job:
    - allow: [read, run, kill]
    - deny: [create, update, delete]
  node:
    - allow: [read, run]
by:
  group: operators

---
# Application-level permissions (outside project context)
description: Operators can view projects and their own profile
context:
  application: 'rundeck'
for:
  project:
    - allow: [read]
    - deny: [configure, delete, import, export, admin]
  system:
    - deny: [read, admin]
by:
  group: operators

Restart Rundeck to load the new policy:

sudo systemctl restart rundeckd

Test by logging in as the operator user. You should see jobs and be able to run them, but the “Create Job” and “Edit” options should be hidden or disabled.

API Tokens

For triggering jobs from scripts, CI/CD pipelines, or other automation, use API tokens instead of username/password authentication.

Create a Scoped Token

1. Log in as admin.
2. Go to User Profile (top-right dropdown > Profile).
3. Under API Tokens, click Generate New Token.
4. Set an expiration date and a descriptive name (e.g., ci-pipeline-trigger).
5. Copy the token — you won’t see it again.

Warning: By default, Rundeck limits API tokens to a maximum of 30 days. If you’re using tokens for long-running automation (CI/CD pipelines, cron scripts), they will silently stop working when they expire. To increase the maximum token lifetime, add the following to /etc/rundeck/rundeck-config.properties and restart Rundeck:

# Maximum API token duration (e.g., 0 for no expiration, or a duration like 365d)
rundeck.api.tokens.duration.max=365d

Set a calendar reminder to rotate tokens before they expire regardless of the duration you choose.

Trigger a Job via API

# List jobs in a project
curl -s -H "X-Rundeck-Auth-Token: your-api-token-here" \
  "https://rundeck.example.com/api/14/project/homelab/jobs" \
  -H "Accept: application/json" | python3 -m json.tool

# Run a specific job by ID
curl -s -X POST \
  -H "X-Rundeck-Auth-Token: your-api-token-here" \
  -H "Content-Type: application/json" \
  -d '{"options": {"service_name": "httpd", "delay_seconds": "5"}}' \
  "https://rundeck.example.com/api/14/job/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/run"

Warning: Do not use the admin token for automation. If a script containing the admin token is compromised, the attacker has full control over Rundeck. Create purpose-specific tokens with the minimum required permissions.

What’s Next (Brief Mentions)

There’s more you can build on this foundation, but it’s out of scope for this guide:

• Node tags and filters — Tag nodes with metadata (environment:prod, role:web) and use filters in jobs to target subsets dynamically.
• Job orchestration — Chain jobs together so one job triggers another on success. Useful for multi-stage deployment workflows.
• Webhook triggers — Expose a URL that, when hit, triggers a Rundeck job. Useful for CI/CD integration (GitHub Actions, GitLab CI).
• Log forwarding — Send execution logs to an S3-compatible store or a log aggregator instead of keeping them on local disk.

Each of these is a natural extension of what you’ve built in this chapter. Start with what you have, use it for a few weeks, and add complexity only when you feel a real need.

Verification

At this point, you should have:

• A “Rolling Service Restart” job that takes a service name and delay as parameters
• A “System Update” job with a dry-run option
• A “Rundeck Database Backup” job running on a nightly schedule
• An operators.aclpolicy file that restricts a non-admin user to run-only access
• At least one API token for programmatic access

Run each job at least once to confirm it works. Check the Activity tab to see execution history. The history is one of Rundeck’s biggest advantages over running playbooks from the command line — six months from now, you’ll be able to see exactly what ran, when, with what parameters, and what the output was.

What Can Go Wrong

Job Options Not Passed to Ansible

Symptom: The playbook runs but variables are empty or undefined. Ansible throws “undefined variable” errors.

Cause: The job options aren’t being passed to Ansible. Rundeck does not automatically forward job options to Ansible variables — you must map them explicitly.

Fix: In the Ansible Playbook step configuration, check the Extra Variables field. Each job option must be listed as a YAML key using the ${option.variable_name} syntax (e.g., service_name: ${option.service_name}).

Scheduled Job Doesn’t Run

Symptom: You configured a schedule but the job never executes. No entries in the Activity tab.

Cause: Usually the schedule syntax is wrong, or the Rundeck server’s timezone doesn’t match your expectations.

Fix: Check the job’s schedule tab — Rundeck shows a preview of the next execution time. Verify the server timezone with timedatectl. Rundeck uses the server’s system timezone for scheduling. If the preview shows the right time but the job still doesn’t run, check service.log for scheduler errors.

ACL Policy Not Taking Effect

Symptom: The operator user can still edit jobs, or can’t see anything at all.

Cause: ACL policy syntax is strict. A misplaced indent or wrong context definition silently fails.

Fix: Validate the policy file: check that the context block is correct (project vs. application), that by: group: matches the group name in realm.properties, and that there are no YAML syntax errors. Rundeck logs ACL evaluation in service.log at debug level — temporarily increase logging if you need to trace a specific permission decision.