Why SOC analysts get inconsistent results from ChatGPT (and how structured workflows fix it)
<p>If you've ever handed a security alert to ChatGPT and gotten a different answer each time — you've hit the real problem.</p> <p>It's not the model. It's the prompt.</p> <p>Most analysts paste an alert and ask "what do you think?" That's like asking a junior analyst to investigate without a runbook. You'll get something back, but the quality depends entirely on how the question was framed.</p> <h2> The real problem: no structure </h2> <p>Experienced SOC analysts don't wing investigations. They follow a process:</p> <ul> <li>Triage the alert</li> <li>Map to MITRE ATT&CK</li> <li>Check for lateral movement</li> <li>Build a containment recommendation</li> <li>Write a ticket summary</li> </ul> <p>The issue is that most AI-assisted workflows skip steps 2–5 and jump straight to "is this ba
If you've ever handed a security alert to ChatGPT and gotten a different answer each time — you've hit the real problem.
It's not the model. It's the prompt.
Most analysts paste an alert and ask "what do you think?" That's like asking a junior analyst to investigate without a runbook. You'll get something back, but the quality depends entirely on how the question was framed.
The real problem: no structure
Experienced SOC analysts don't wing investigations. They follow a process:
-
Triage the alert
-
Map to MITRE ATT&CK
-
Check for lateral movement
-
Build a containment recommendation
-
Write a ticket summary
The issue is that most AI-assisted workflows skip steps 2–5 and jump straight to "is this bad?"
What I built
I spent time building SOC.Workflows — a free collection of structured investigation workflows for SOC analysts. Each workflow breaks an investigation into 4 steps, with specific prompts for each step, designed to run in ChatGPT or Claude.
Current workflows:
-
Phishing Email Investigation
-
AWS VPC Flow Log Analysis
-
PowerShell & Script Analysis
-
Credential Dumping Investigation
-
Ransomware Triage
-
Identity Compromise Investigation
-
URL & Domain Analysis
-
SOC Alert Triage
-
Explain This Alert
How it works
-
Pick a workflow matching your alert type
-
Copy the workflow prompt
-
Paste into ChatGPT or Claude
-
Get structured, step-by-step analysis
No login. No setup. No API keys.
Why structure matters
When I ran the same phishing alert through an unstructured prompt vs. the structured workflow, the difference was clear:
Unstructured: "This looks like a phishing email. Check the sender domain."
Structured: SPF/DKIM validation → header analysis → sender reputation → verdict with risk score → recommended response actions
Same model. Completely different output quality.
Try it
If you work in a SOC or do blue team work, I'd love feedback on which investigation types are missing.
👉 socworkflows.com — free, no login required
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!