Threat Detection
Detection as code in the agentic era: CI/CD strategies for detection engineering teams
Detection as code is managing detection rules the way developers manage software. Rules live in version control, are written in a portable format such as Sigma, are reviewed through pull requests, are tested automatically in a CI/CD pipeline and reach the SIEM only through that pipeline.

Published 7 min read
Detection as code is managing detection rules the way developers manage software. Every rule lives in version control, written in a portable format such as Sigma. Every change goes through a pull request that someone reviews. A CI/CD pipeline tests the rule against events it must catch and events it must ignore, then deploys it to the SIEM. Nobody edits a production rule by hand. Detection engineering teams adopt it because rules managed this way stop breaking silently. The arrival of AI agents in the SOC makes it more urgent, not less: an agent that writes rules needs the same tests and the same reviewer as a person.
Why detection rules break when they live in the console
When we review a SIEM, the same five problems turn up again and again.
- Rules edited in the console. Someone changes a rule by hand to stop an alert. There is no history, no review and no way to roll back, and six months later nobody knows why the rule looks the way it does.
- Rules that break silently. A rule was never tested, so when a log source changes format or a field is renamed, it simply stops firing. A broken rule raises no alert, so nothing tells you it is broken.
- Rules disabled instead of tuned. A noisy rule gets switched off because tuning it takes time. Coverage shrinks a little each time, and nobody notices.
- Unknown coverage. Rules are not mapped to MITRE ATT&CK, so when management asks which attack techniques you can detect, nobody can answer.
- Inconsistent documentation. Some rules have a description, some have a runbook, most have neither. The analyst who receives the alert does not know what it means, how bad it is or who owns the rule.
Noise is the symptom most teams feel. In the SANS 2025 Detection & Response Survey, 73% of organisations named false positives as their top detection challenge, and 20% faced them at very high rates, up from 13% a year earlier. Much of that noise comes from rules that were never tested before they went live.
What a detection-as-code pipeline looks like
A first pipeline does not need to be elaborate. It needs four parts.
A repository, and pull requests for every change. Each rule is a file in Git, with its description, severity, owner, ATT&CK technique and runbook next to the detection logic. A change is a pull request that another engineer reviews. The history now answers who changed what, when and why.
A portable format. Sigma is the open format most teams choose. A Sigma rule is a YAML file that describes what to look for in which log source, and converters built on pySigma turn it into the query language of Splunk, Elastic, Microsoft Sentinel and others. The current Sigma specification, version 2.1.0, also covers correlation rules. If you will only ever run one SIEM, its native format works, provided it is documented and checked by the pipeline.
Automated tests on every change. The pipeline validates and tests each rule before anyone can merge it. The next section lists the tests.
Deployment through the SIEM's API, with rollback. Once merged, the pipeline converts the rule and deploys it. The previous version stays in the repository, so rolling back is one revert. The main platforms support this today. Microsoft Sentinel connects to a GitHub or Azure DevOps repository and deploys analytics rules, hunting queries and other content from it, with Microsoft's advice to edit connected content only in the repository. Elastic publishes its own rules from the detection-rules repository, with commands to validate rules against a schema, run unit tests and move rules between the repository and Kibana. Splunk's contentctl validates, builds and tests detections, starting a Splunk container to run them against real events.
The pipeline itself is ordinary CI/CD, on GitHub Actions, GitLab CI/CD or Azure DevOps. If your developers already have one, the detection team can use the same platform and the same protections, which our secure CI/CD pipelines work puts in place.
The tests that keep detection rules honest
Detection as code is only as good as its quality assurance. These are the checks we would put in a pipeline, roughly in the order they run.
| Check | What it catches |
|---|---|
| Schema validation | A rule file that is malformed, misses a required field or uses a value the format does not allow |
| Policy validation | A rule without an owner, a severity, an ATT&CK technique or a runbook; a rule that breaks your naming or severity conventions |
| Documentation consistency | A description that no longer matches the logic, or a runbook link that leads nowhere |
| Unit tests | Logic errors in a single rule, run against small sample events |
| Golden tests | A rule that no longer fires on the recorded attack events it was written to catch |
| Negative tests | A rule that fires on normal activity it must ignore, the main source of false positives |
| Quality gates | A merge that lowers coverage, fails a test or has no reviewer; the gate blocks it |
| Smoke tests | A deployment that did not arrive, or a rule that does not run, checked in the SIEM right after release |
| Full regression | A change in one rule, parser or log source that quietly breaks others, run on the whole rule set on a schedule and before major changes |
Golden and negative tests are the two most teams skip and most need. A golden test is a small set of recorded events from a real or simulated attack that the rule must always catch. A negative test is a set of normal events that the rule must never match. Together they turn "we think the rule works" into evidence that runs on every change.
Measure every rule after release
A rule that passes its tests can still be wrong in production. Track three things for every rule: how many alerts it raises, how many of those turn out to be false positives, and which ATT&CK techniques it covers. A rule that fires constantly gets tuned in a pull request, not switched off in the console. A rule that has not fired for months gets its golden test rerun, to check that it still can.
Keep the ATT&CK mapping in the repository too, because ATT&CK changes. Version 19, released in April 2026, split the Defense Evasion tactic into two new tactics, Stealth and Defense Impairment, and version 20 followed in October 2026. A team whose mappings sit in files can update them in one reviewed change and regenerate its coverage map the same day. A team whose mappings sit in a spreadsheet, or nowhere, starts again.
The autonomous SOC: what AI agents may do in detection engineering
The SOC is getting AI agents. Microsoft announced Security Copilot agents in March 2025, including one that sorts phishing alerts into real threats and false alarms and learns from analysts' feedback. In April 2025 Google announced an alert triage agent for Google Security Operations that gathers evidence for each alert and records the reasoning behind its verdict. In the SANS survey, 76% of organisations planned to expand their use of AI and machine learning in detection and response. "AI SOC", "agentic SOC" and "autonomous SOC" are now product categories.
Detection as code is what makes that safe. Here is where we would draw the line.
- Let agents triage and enrich. Grouping related alerts, adding user, asset and threat intelligence context, and writing a summary for the analyst saves real time and changes nothing.
- Let agents investigate, read-only. Running searches across logs and threat intelligence to answer an analyst's question is useful, as long as the agent's access is read-only.
- Let agents draft rules, as pull requests. An agent can propose a new rule after an incident, or a tuning change for a noisy one. The proposal goes through the same schema checks, golden and negative tests and human review as any other change. The pipeline does not care who wrote the rule.
- Never let agents act alone. Isolating a host, disabling an account or pushing a rule to production waits for a person to approve it. A human stays in the loop, and every action the agent proposes is logged.
An agent that can change your detections is also something an attacker will try to manipulate, through the alerts, logs and documents it reads. Treat it as an identity with the narrowest permissions it needs, as our article on agentic AI security explains, and watch for attacks on the AI itself, which MITRE ATLAS maps. An autonomous SOC without a tested pipeline behind it is an unreviewed engineer with production access.
Where to start
You do not need to rebuild everything at once.
- Export your current rules into a repository, as they are, and make the repository the only place rules change.
- Add schema and policy checks, so that every rule has an owner, a severity, an ATT&CK technique and a runbook.
- Write golden and negative tests for the ten rules that matter most, then add tests to every rule you touch.
- Connect the pipeline to the SIEM's API and turn off manual editing of deployed rules.
If you are implementing a SIEM, or finishing one that was never set up properly, starting this way costs little. Our SIEM implementation and threat detection service deploys Microsoft Sentinel, Splunk, Elastic Security or Wazuh, writes detection rules mapped to MITRE ATT&CK, tests each one against simulated activity and hands the platform to your team with runbooks, usually in two to four months.
Frequently asked questions
What is detection as code?
It is running detection rules like software. Each rule is a file in a repository, written in a documented format, changed only through a reviewed pull request, tested automatically on every change and deployed to the SIEM by a pipeline. Nobody edits a production rule by hand in the console.
Do we have to write our rules in Sigma?
No. Sigma is the most widely used portable format, and its converters turn a rule into the query language of Splunk, Elastic, Microsoft Sentinel and others, which helps if you change platforms or run more than one. Writing rules in your SIEM's own language works too. What matters is that the format is documented and checked by the pipeline.
Can AI agents write detection rules?
They can draft them, and they are useful at it. The draft should arrive as a pull request and go through the same schema checks, tests and human review as a rule written by a person. An agent should never push a rule to production, isolate a host or disable an account without someone approving it.
Will GUILDA run our SOC?
No. We implement the SIEM, write and test the detection rules and hand the platform over with runbooks. Your team, or a provider you choose, operates it. We don't run a monitoring service.
Sources
- 2025 Detection & Response Survey, SANS Institute, 2025
- Sigma specification, SigmaHQ, 2025
- ATT&CK updates, April 2026 (v19), MITRE, 2026
- Deploy custom content from your repository, Microsoft Sentinel, Microsoft Learn, 2026
- detection-rules, Elastic, 2026
- contentctl, Splunk, 2026
- Microsoft unveils Microsoft Security Copilot agents and new protections for AI, Microsoft, 2025
- Driving secure innovation with AI and Google Unified Security, Google Cloud, 2025
SIEM implementation and threat detection
We deploy a SIEM, connect the log sources that matter, write detection rules mapped to MITRE ATT&CK and hand the platform over with runbooks. Your team operates it afterwards; we don't run a monitoring service.
Explore Threat Detection & SIEM ImplementationRelated reading
Threat Detection
Behavioral biometrics with ML: PERSONA, a multimodal view of every user, device and AI agent
Behavioral biometrics identifies a user by how they act, not by what they know or carry. Combined with a device fingerprint and session context, and built with machine learning, it gives a profile that shows when an account, a device or an AI agent stops behaving like itself. PERSONA is our name for that profile.
7 min read
Threat Detection
Bot detection by rhythm: how Fourier transforms and frequency analysis expose bots
Most bot detection judges each request on its own, so a patient bot gets through. Scripts and malware keep time, and people do not. This article explains how a Fourier transform turns that timing into a visible peak, how jitter tries to hide it, and where rhythm fits beside the other signals.
6 min read
Tell us what you need to detect
Name your systems and the requirement behind the project. We reply within two business days.