Detection: Local LLM Framework DNS Query

Description

Detects DNS queries related to local LLM models on endpoints by monitoring Sysmon DNS query events (Event ID 22) for known LLM model domains and services. Local LLM frameworks like Ollama, LM Studio, and GPT4All make DNS calls to repositories such as huggingface.co and ollama.ai for model downloads, updates, and telemetry. These queries can reveal unauthorized AI tool usage or data exfiltration risks on corporate networks.

Search

 1`sysmon`
 2EventCode=22
 3QueryName IN (
 4    "*huggingface*",
 5    "*ollama*",
 6    "*jan.ai*",
 7    "*gpt4all*",
 8    "*nomic*",
 9    "*koboldai*",
10    "*lmstudio*",
11    "*modelscope*",
12    "*civitai*",
13    "*oobabooga*",
14    "*replicate*",
15    "*anthropic*",
16    "*openai*",
17    "*openrouter*",
18    "*api.openrouter*",
19    "*aliyun*",
20    "*alibabacloud*",
21    "*dashscope.aliyuncs*"
22)
23NOT Image IN (
24    "*\\MsMpEng.exe",
25    "C:\\ProgramData\\*",
26    "C:\\Windows\\System32\\*",
27    "C:\\Windows\\SysWOW64\\*"
28)
29
30| stats count
31    min(_time) as firstTime
32    max(_time) as lastTime
33    by src Image process_name QueryName query_count answer answer_count reply_code_id vendor_product
34
35| `security_content_ctime(firstTime)`
36
37| `security_content_ctime(lastTime)`
38
39| `local_llm_framework_dns_query_filter`

Data Source

Name	Platform	Sourcetype	Source
Sysmon EventID 22	Windows	'XmlWinEventLog'	'XmlWinEventLog:Microsoft-Windows-Sysmon/Operational'

Macros Used

Annotations

Default Configuration

This detection is configured by default in Splunk Enterprise Security to run with the following settings:

Implementation

Ensure Sysmon is deployed across Windows endpoints and configured to capture DNS query events (Event ID 22). Configure Sysmon's XML configuration file to log detailed command-line arguments, parent process information, and full process image paths. Ingest Sysmon event logs into Splunk via the Splunk Universal Forwarder or Windows Event Log Input, ensuring they are tagged with sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational. Verify the sysmon macro exists in your Splunk environment and correctly references the Sysmon event logs. Create or update the unauthorized_local_llm_framework_usage_filter macro in your detections/filters folder to exclude approved systems, authorized developers, sanctioned ML/AI workstations, or known development/lab environments as needed. Deploy this hunting search to your Splunk Enterprise Security or Splunk Enterprise instance and schedule it to run on a regular cadence to detect unauthorized LLM model DNS queries and shadow AI activities. Correlate findings with endpoint asset inventory and user identity data to prioritize investigation.

Known False Positives

Legitimate DNS queries to LLM model hosting platforms by authorized developers, ML engineers, and researchers during model training, fine-tuning, or experimentation. Approved AI/ML sandboxes and lab environments where LLM model downloads are expected. Automated ML pipelines and workflows that interact with LLM model hosting services as part of their normal operation. Third-party applications and services that access LLM model platforms for legitimate purposes.

Associated Analytic Story

• Suspicious Local LLM Frameworks

References

• https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

• https://www.splunk.com/en_us/blog/artificial-intelligence/splunk-technology-add-on-for-ollama.html

• https://blogs.cisco.com/security/detecting-exposed-llm-servers-shodan-case-study-on-ollama

Detection Testing

Replay any dataset to Splunk Enterprise by using our replay.py tool or the UI. Alternatively you can replay a dataset into a Splunk Attack Range

Source: GitHub | Version: 1