Analytics Story: Abuse of GenAI Systems and Applications
Description
Detect and investigate threats targeting or abusing artificial intelligence systems, services and applications. Coverage includes prompt injection, attempts to bypass model safeguards, repeated high-risk guardrail events, and possible sensitive data disclosure through model interactions.
Why it matters
AI applications can expose sensitive information, invoke connected tools, or process untrusted prompts. Attackers and malicious insiders may try to override model instructions, extract restricted data, misuse AI-enabled tools, or repeatedly probe guardrails. Use this story to investigate AI security events across model providers and application integrations. Review the affected application and connection, event action, prompt or response direction, policy, and guardrail categories. An allowed event can indicate exposure, while a blocked event records an attempt; neither alone confirms the full impact. Correlate high-risk events with identity, application, data access, and tool activity where those logs are available.
Detections
| Name | Technique | Type |
|---|---|---|
| Cisco AI Defense Allowed Sensitive Response | None | TTP |
| Cisco AI Defense Repeated High Risk Activity | None | TTP |
| Cisco AI Defense Allowed Sensitive Prompt | None | TTP |
| Cisco AI Defense Security Alerts by Application Name | None | Anomaly |
Data Sources
| Name | Platform | Sourcetype | Source |
|---|---|---|---|
| Cisco AI Defense Alerts | Other | cisco:ai:defense |
not_applicable |
References
- https://owasp.org/www-project-top-10-for-large-language-model-applications/
- https://atlas.mitre.org/
Source: GitHub | Version: 1