devguard
Automations

Build your own check

Write a check that the catalog does not have, with the AI or step by step, test it, and publish it so it keeps an evidence record current.

The catalog covers the common settings of common tools. When you need something else, build the check yourself. For example: "our status page answers", or "this admin group has at most three members". A check you build works like a catalog check: it runs on a schedule and keeps one evidence record current. See How a check keeps evidence current.

There are three ways to start:

Then test and publish it. The parts of a check are listed under Steps.

The New automation dialog: under Checks the tiles From the catalog, Describe it to AI and Blank check, and under Ticket rules the button Create ticket rule.
Three ways to start a check. Ticket rules are a separate kind of automation.

Describe it to the AI

  1. Open Automations, choose New automation, then Describe it to AI. The Build with AI page opens.
  2. Write what the check must verify, in plain words. Name the tool and what counts as a pass. For example: "Check that www.example.com and example.com redirect to HTTPS".
  3. The AI drafts the check. Its steps appear on the right.
  4. Ask the AI to test it. The test result appears in the same panel. Ask for changes until the result is right.
  5. Publish the check yourself. See Test and publish. The AI can draft, change and test a check, but it cannot publish one.

Start from a blank check

  1. Choose New automation, then Blank check.
  2. Enter a Check name.
  3. Choose a Connection if the check reads a connected tool. An HTTP or TLS test needs none.
  4. Choose Create and open. The check opens as a Draft, with one Result step.
  5. Add the steps. See Edit the steps.

Change the steps of a catalog check

  1. Open the check and choose Edit steps.
  2. devguard asks Make this check your own copy?. Choose Edit my copy.
  3. Edit the steps. The published version keeps running until you publish.
The dialog Make this check your own copy?: its steps stop receiving catalog updates, edits go to a draft while v1 keeps running, and discarding the draft keeps the catalog version.
devguard asks before your first change. Nothing changes until you publish.

Watch out: once you publish your copy, devguard no longer updates its steps. If your copy uses a connection, it can only call that tool's API.

Edit the steps

The steps run from top to bottom. Select a step to edit it in the panel on the right.

  1. To add a step, choose the + between two steps, or Add a step at the end of a loop or a branch.
  2. Choose the kind of step. See Steps.
  3. Fill in the step in the panel. For a Rule, set the Condition and the Failure summary. Turn on Raise a finding to set the Finding title, Severity and Description.
  4. To move, copy or remove a step, open its menu (...) and choose Move up, Move down, Duplicate or Delete.

Every change is saved to the draft. Edit as JSON shows the same step as text.

A catalog check in edit mode: the Rule step No HTTPS redirect is selected inside a Branch, and the panel on the right shows its condition, failure summary, finding title, description and severity.
The selected rule creates a Medium finding when a host does not redirect to an https:// address.

Test and publish

  1. Choose Test run, or Test run draft when a published version exists. The test runs the draft once. It writes no evidence and does not change the schedule. It appears in All runs with a Test run badge.
  2. Open the result. Check that it passes and fails for the right reason.
  3. Choose Publish (a new check) or Publish v2 (a new version).
  4. A new check asks for a Schedule (required), a Credential and an Owner. Choose Publish.

A new check is now Active and runs on its schedule. A new version keeps the status the check had, so a paused check stays paused. Each publish raises the version number. Discard draft throws the draft away and keeps the published version.

Publishing fails in these cases:

  • The schedule, a required variable or a required connection is missing.
  • The check uses a connection that custom steps cannot use.
  • Your plan has no room for another active check.

Reference

Steps

StepWhat it does
FetchCalls one web address and keeps the answer.
Fetch pagesCalls a web address that returns a long list in parts, and keeps all the parts.
Probe TLSReads the certificate of a host.
For eachRuns the steps inside it once for each item in a list.
AggregateReshapes a list from an earlier step: counts it, filters it, keeps some fields, flattens it, or takes the first item.
BranchRuns one set of steps if a condition holds and another if not.
RuleTests a condition. If it fails, the run fails and can raise a finding.
ResultEnds the check with a short summary. It can save what the check read, and devguard attaches that to the evidence record as a file when the run passes.

Rules for custom steps

  • A credential may only be used in a request header.
  • A check that uses a connection can only call the API of that tool.
  • Connections to Okta, Atlassian, BambooHR, Jamf Pro, Kandji, Zendesk and Grafana Cloud cannot be used in steps you write or change.
  • Test runs are limited to about 20 every 10 minutes.
  • The AI builder needs a Business plan and counts against the monthly AI limit.

Who can build

Owners and admins can build, test and publish checks. Members and auditors can open them in view only.

How is this guide?

Automations

Let devguard check your tools on a schedule and keep the evidence current, and open recurring tickets in GitHub, GitLab, Jira, Asana or Slack.

Records

Write and finalize the documents your management system produces, and keep a locked copy of each version for your auditor.

On this page

Describe it to the AIStart from a blank checkChange the steps of a catalog checkEdit the stepsTest and publishReferenceStepsRules for custom stepsWho can build