Detection: Cisco Isovalent - Kprobe Spike

Updated Date: 2026-01-05 ID: 3df0e9a8-7d5e-4b2f-bcd7-bf93e671d1f2 Author: Bhavin Patel, Splunk Type: Hunting Product: Splunk Enterprise Security

Description

This analytic detects excessive kernel probe (kprobe) events in a Kubernetes cluster over a short period of time. Kprobes are a Linux kernel debugging and instrumentation mechanism that allows dynamic monitoring and tracing of kernel functions and system calls. In containerized or cloud-native environments, kprobes are occasionally used for legitimate low-level diagnostics; however, monitoring a spike in kprobe activity is important because malware or attackers may abuse this mechanism to gain insights into the kernel, attempt privilege escalation, or tamper with host processes. More than 10 kprobe events within 5 minutes may indicate suspicious activity, such as an attacker probing the kernel through repeated system calls (e.g., nsenter, mount, sethostname). Such abnormal volume and frequency of kprobe usage within application pods or on nodes can signal container escape attempts or low-level tampering with the host, thereby representing a potential security threat.

Search

 1`cisco_isovalent` process_kprobe.action!=""
 2
 3| bin _time span=5m 
 4| rename process_kprobe.parent.pod.name as pod_name 
 5
 6| stats count as kprobe_count 
 7        values(process_kprobe.function_name) as functions
 8        values(process_kprobe.process.binary) as binaries
 9        values(process_kprobe.args{}.string_arg) as args
10  by pod_name _time
11
12| where kprobe_count > 10 
13| `cisco_isovalent___kprobe_spike_filter`

Data Source

Name	Platform	Sourcetype	Source
Cisco Isovalent Process Kprobe	Other	`'cisco:isovalent'`	`'not_applicable'`

Macros Used

Name	Value
cisco_isovalent	`sourcetype=cisco:isovalent`
cisco_isovalent___kprobe_spike_filter	`search *`

cisco_isovalent___kprobe_spike_filter is an empty macro by default. It allows the user to filter out any results (false positives) without editing the SPL.

Annotations

- MITRE ATT&CK

+ Kill Chain Phases

+ NIST

+ CIS

- Threat Actors

ID	Technique	Tactic
T1068	Exploitation for Privilege Escalation	Privilege Escalation

Exploitation

DE.AE

CIS 10

Default Configuration

This detection is configured by default in Splunk Enterprise Security to run with the following settings:

Setting	Value
Disabled	true
Cron Schedule	`0 * * * *`
Earliest Time	`-70m@m`
Latest Time	`-10m@m`
Schedule Window	`auto`
Creates Risk Event	False

This configuration file applies to all detections of type hunting.

Implementation

Requires Cisco Isovalent Runtime Security with kprobe tracing enabled and logs forwarded into Splunk. Ensure that your Splunk Technology Add-on (TA) for Cisco Security Cloud parses the kprobe JSON correctly. Tune the threshold based on your workload baseline.

Known False Positives

Busy or noisy pods may legitimately produce bursts of kprobe events during normal operation. Tune thresholds and filter by function_name to reduce false positives.

Associated Analytic Story

Cisco Isovalent Suspicious Activity

References

https://docs.isovalent.com/user-guide/sec-ops-visibility/process-execution/index.html

Detection Testing

Test Type	Status	Dataset	Source	Sourcetype
Validation	✅ Passing	N/A	N/A	N/A
Unit	✅ Passing	Dataset	`not_applicable`	`cisco:isovalent`
Integration	✅ Passing	Dataset	`not_applicable`	`cisco:isovalent`

Replay any dataset to Splunk Enterprise by using our replay.py tool or the UI. Alternatively you can replay a dataset into a Splunk Attack Range

Source: GitHub | Version: 1