CLI documentation

Scan a fleet with the OpenCloud Security Scanner

Scan multiple OpenCloud instances with per-instance configuration and visible waivers.

Multiple instances often need different ports, release tracks and waivers. This guide shows when to use a shared command, how to keep a configuration file per instance and how to schedule the resulting checks.

One command, several hosts

--host takes a comma-separated list. The plugin scans them in turn, prints a one-line summary followed by a block per host, and exits with the worst state it found - see Checking multiple hosts.

check-opencloud-security --check-hardening \
  --host opencloud1.example.com,opencloud2.example.com:9200,[2001:db8::1]

This is the right answer when the instances are alike. Everything after --host applies to all of them, so the moment one instance needs --insecure or a waiver the others do not, you have outgrown it.

Aggregating also loses the per-host history your monitoring system would otherwise keep. If you want one red service per broken instance rather than one red service for the group, use one check per host instead.

Add --webhook-digest here and the webhook fires at most once for the whole --host list, instead of once per host that meets --webhook-on - useful when a webhook receiver is a person who would rather get one message about three broken instances than three separate pings. It only combines what happens inside this one process: since it is one flag on one ScanContext, it has no effect on the "configuration file per instance" and "loop over the files" patterns below, where each instance runs as its own separate process with nothing to combine across - each of those still sends its own webhook per instance, exactly as without the flag.

One configuration file per instance

Each instance gets a file, and the file carries everything that makes it different. Nothing is repeated on the command line, so a change is a change in one place.

# /etc/check-opencloud-security/prod-eu.yml
host: opencloud-eu.example.com
check_hardening: true
update_warning: true

scanner:
  target_port: 9200
  release_track: production
  ignore_hardenings:
    # The reverse proxy owns this header; the instance cannot set it.
    - hstsPreload

releases:
  mode: auto
  token: secret://releases_token
check-opencloud-security --config /etc/check-opencloud-security/prod-eu.yml

Precedence is command line > environment variable > configuration file > default, so a per-instance file can still be overridden for one run without editing it. The full syntax, including secret://, is in Configuration file and secrets.

Write the first file with check-opencloud-security --configure --config /etc/check-opencloud-security/prod-eu.yml and copy it for the rest.

A loop over the files

With one file per instance, scanning the fleet is a for loop, and the exit codes are what you report on:

#!/bin/sh
# Scan every configured instance; exit with the worst state seen.
set -u

worst=0
rank() { case "$1" in 2) echo 3 ;; 1) echo 2 ;; 3) echo 1 ;; *) echo 0 ;; esac; }

for config in /etc/check-opencloud-security/*.yml; do
  check-opencloud-security --config "$config" || state=$?
  state="${state:-0}"
  [ "$(rank "$state")" -gt "$(rank "$worst")" ] && worst="$state"
  unset state
done

exit "$worst"

rank exists because Nagios exit codes are not ordered by severity: CRITICAL (2) outranks WARNING (1), which outranks UNKNOWN (3), which outranks OK (0). Sorting numerically would report a host that could not be reached as worse than a host that is end-of-life.

Where the checks should run from

The plugin scans over the network, from wherever it runs, so where you run it decides what it can see:

  • An instance behind a firewall needs a check running inside it, not a check on the monitoring server with a hole punched through.
  • Whether HTTPS is enforced and whether the certificate is trusted depend on the path taken to the instance. Scanning through a load balancer that terminates TLS measures the load balancer.
  • Debug-port probes only mean anything from a network that is supposed not to reach them. From inside the instance's own host they will find ports that no outsider could.

If several monitoring consumers need the same result, run the scan service close to the instances and let them share its cache. The plugin itself never talks to it - it always scans in process - so the service is for dashboards and scripts.

Keeping the waivers honest

Review ignore_hardenings entries regularly so accepted findings do not hide new problems. Waivers have two safeguards:

  • A waiver only ever suppresses the alert. The finding stays in the result document with "ignored": true, and --debug still explains it - see Accepting a finding you are not going to fix.
  • Only a check that actually failed can be waived, so a waiver cannot silently cover a measure that later regresses into a different finding.

Review them by scanning with the waivers off and diffing:

for config in /etc/check-opencloud-security/*.yml; do
  echo "== $config"
  COS_SCANNER_IGNORE_HARDENINGS="" check-opencloud-security --config "$config" --debug \
    | grep -E 'ignored|waived|FAIL'
done

One identifier will never appear in that list, however many instances you run: publicLinkExpirationEnforced is hardcoded by OpenCloud and fails on every instance, so it is recorded but excluded from the alert, the hardenings_missing metric and the webhook. It does not need a waiver; see Measures that are not settings.

Only alerting on what changed

Repeated alerts for unchanged findings can make new problems easy to miss. Give each host a baseline to report regressions:

for config in /etc/check-opencloud-security/*.yml; do
  check-opencloud-security --config "$config" \
      --baseline /var/lib/check_opencloud/baseline.json \
      --warn-on-new
done

One file is enough for the whole fleet: it stores one entry per host, keyed by the host as it was given on the command line. A comma-separated --host list works the same way.

Two things to get right:

  • The monitoring user must own the directory. The file is written atomically with owner-only permissions, and a baseline that cannot be written is reported as a line of output and nothing more - it never changes the verdict.
  • Use the same spelling of the host everywhere. opencloud.example.com and https://opencloud.example.com/ normalise to the same host, but 10.0.0.5 does not match the name that resolves to it, and the check would treat it as a host it has never seen.

A release past its end of life keeps alerting on every run regardless of the baseline, which is the point: it receives no security fixes, so it gets worse every day it stays up. See Reporting only what changed.

Add --self-update-check on one host in the fleet - not all of them - to be told when a newer plugin version is published. It is cached for a day and never changes the exit code.

One dashboard for the whole fleet

Monitoring answers "is this instance broken right now", one host at a time. A fleet review asks something else: which instances run a release nobody patches any more, which waivers run out this month, which finding is broken everywhere, and which instance nobody has looked at lately. Keep the result documents scan writes, and fleet answers all four from the files alone:

dir="/var/lib/opencloud-reports/$(date +%F)"
mkdir -p "$dir"
for host in opencloud1.example.com opencloud2.example.com; do
  check-opencloud-scanner scan "$host" > "$dir/$host.json"
done
check-opencloud-scanner fleet /var/lib/opencloud-reports \
    --inventory /etc/check-opencloud-security/hosts.txt --format html > fleet.html

It scans nothing and stores nothing; the newest report of each host is the one that counts, and the release and the waiver deadlines are measured against today rather than the day the report was written. An inventory makes a host with no report at all show up as missing. See fleet - a dashboard from saved results.

Scheduling the whole thing

Stagger the schedules. Twenty instances scanned at 0 6 * * * means twenty simultaneous scans from one address, and the update check will meet GitHub's anonymous rate limit on the way.


Back to the documentation index | Back to the main README