Detection: AWS Bedrock High Number List Foundation Model Failures

Updated Date: 2024-12-05 ID: e84b3c74-f742-11ee-9f6e-acde48001122 Author: Bhavin Patel, Splunk Type: TTP Product: Splunk Enterprise Security

Description

The following analytic identifies an high number of AccessDenied attempts to list AWS Bedrock foundation models. It leverages AWS CloudTrail logs to detect when a user or service experiences multiple failures when calling the ListFoundationModels API. This activity is significant as it may indicate an adversary performing reconnaissance of available AI models after compromising credentials with limited permissions. Repeated failures could suggest brute force attempts to enumerate accessible resources or misconfigured access controls. If confirmed malicious, this could represent early-stage reconnaissance before attempting to access or manipulate Bedrock models or knowledge bases.

1`cloudtrail` eventSource=bedrock.amazonaws.com eventName=ListFoundationModels errorCode=AccessDenied  
2| rename user_name as user  
3| stats count min(_time) as firstTime max(_time) as lastTime values(errorCode) as errorCodes values(errorMessage) as errorMessages by src user user_agent vendor_account vendor_product dest signature vendor_region 
4| where count > 9 
5| `security_content_ctime(firstTime)`  
6| `security_content_ctime(lastTime)` 
7| `aws_bedrock_high_number_list_foundation_model_failures_filter`

Data Source

Name Platform Sourcetype Source
AWS CloudTrail AWS icon AWS 'aws:cloudtrail' 'aws_cloudtrail'

Macros Used

Name Value
cloudtrail sourcetype=aws:cloudtrail
aws_bedrock_high_number_list_foundation_model_failures_filter search *
aws_bedrock_high_number_list_foundation_model_failures_filter is an empty macro by default. It allows the user to filter out any results (false positives) without editing the SPL.

Annotations

- MITRE ATT&CK
+ Kill Chain Phases
+ NIST
+ CIS
- Threat Actors
ID Technique Tactic
T1580 Cloud Infrastructure Discovery Discovery
Exploitation
DE.CM
CIS 10

Default Configuration

This detection is configured by default in Splunk Enterprise Security to run with the following settings:

Setting Value
Disabled true
Cron Schedule 0 * * * *
Earliest Time -70m@m
Latest Time -10m@m
Schedule Window auto
Creates Notable Yes
Rule Title %name%
Rule Description %description%
Notable Event Fields user, dest
Creates Risk Event True
This configuration file applies to all detections of type TTP. These detections will use Risk Based Alerting and generate Notable Events.

Implementation

The Splunk AWS Add-on is required to utilize this data. The search requires AWS CloudTrail logs with Bedrock service events enabled. You must install and configure the AWS App for Splunk (version 6.0.0 or later) and Splunk Add-on for AWS (version 5.1.0 or later) to collect CloudTrail logs from AWS.

Known False Positives

Legitimate users may encounter multiple failures during permission testing, role transitions, or when service permissions are being reconfigured. High volumes of API errors may also occur during automated processes with misconfigured IAM policies or when new Bedrock features are being explored through API testing.

Associated Analytic Story

Risk Based Analytics (RBA)

Risk Message:

User $user$ attempted to list AWS Bedrock foundation models $count$ times with failures from $src$

Risk Object Risk Object Type Risk Score Threat Objects
user user 48 src

References

Detection Testing

Test Type Status Dataset Source Sourcetype
Validation ✅ Passing N/A N/A N/A
Unit ✅ Passing Dataset aws_cloudtrail aws:cloudtrail
Integration ✅ Passing Dataset aws_cloudtrail aws:cloudtrail

Replay any dataset to Splunk Enterprise by using our replay.py tool or the UI. Alternatively you can replay a dataset into a Splunk Attack Range

Source: GitHub | Version: 1