---
title: Threat_Hunting_with_better_DataVisualization (BSides Tokyo 2025 version)
tags:  #bsides #bsidestokyo  
author: [Tatsuya-hasegawa](https://convert.docswell.com/user/hackeT)
site: [Docswell](https://www.docswell.com/)
thumbnail: https://bcdn.docswell.com/page/P7RGDNX9J9.jpg?width=480
description: Threat_Hunting_with_better_DataVisualization (BSides Tokyo 2025 version) by Tatsuya-hasegawa
published: December 23, 25
canonical: https://convert.docswell.com/s/hackeT/K44XJ8-bsidestokyo2025
---
# Page. 1

![Page Image](https://bcdn.docswell.com/page/P7RGDNX9J9.jpg)

Threat Hunting
with better Data Visualization
Tatsuya Hasegawa (T.H)
Threat Hunter (T.H)
1


# Page. 2

![Page Image](https://bcdn.docswell.com/page/PJXX1N63JX.jpg)

WHOAMI
Nice to meet BSides Tokyo!
• self-employed Threat Hunter : Contracting “Go Ahead, Inc.” for over 4y
• Data Visualization Developer
• ISACA Nagoya Chapter Director
• Security Research Subcommittee Chair
• SECCON Open Conference
• OSS msticpy contributor
• SNS (HN: hackeT, X: @T_8ase)
• WEB: www.hacket-engine.com
• Certiﬁcations
• CISSP, CISA, GSP#414
• GX-FA, GX-FE, GX-IH, GXPN,GPEN,GMOB,GREM,GCIH,GCFA,GCFE,GNFA,GCFR
2


# Page. 3

![Page Image](https://bcdn.docswell.com/page/3JKW2N1NJD.jpg)

My Custom Visualizations ( D3 series )
https://splunkbase.splunk.com/apps?author=hacket
https://github.com/microsoft/msticpy/releases/tag/v2.15.0
3


# Page. 4

![Page Image](https://bcdn.docswell.com/page/LE314VQZ75.jpg)

My Custom Visualizations ( Vega series )
4


# Page. 5

![Page Image](https://bcdn.docswell.com/page/8EDXQ894JG.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
5


# Page. 6

![Page Image](https://bcdn.docswell.com/page/V7PPL8ZV78.jpg)

Agenda
1. Relationship between Threat Hunting and Data
Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
6


# Page. 7

![Page Image](https://bcdn.docswell.com/page/2JV2QNWRJQ.jpg)

What is Threat Hunting?
Security products can NOT detect all threats =&gt; potential risk/threat of FN
But having productʼs telemetries or logs!✨ We can..
Proactive detection, then respond quickly before risk spreads
Even if you go hunting, there may not be any threats. 😅
Your ﬁndings sometimes havenʼt enough risks to report your organization. 😞
Even with the same analytical perspective, new discoveries if the timing is diﬀerent.
Threat hunting should be incorporated into daily operations.
Similar to security monitoring? 🧐
Þ Yes, but security monitoring 👉 “ﬁxed-point observation” (定点観測)
Þ While, threat hunting 👉 “dynamic observation for weak area”
Check more on my past slide of “What you need to know about Threat Hunting.”
https://www.docswell.com/s/hackeT/K6VM1X-isaca20247sr
7


# Page. 8

![Page Image](https://bcdn.docswell.com/page/5EGRWKD6JL.jpg)

Threat Hunting approach - Hypotheses cards
• Intelligence-Driven Hypo
• Known evil, malicious similarity
• ♦ IoC (Indicator of Compromise)
• ❤ IoA (Indicator of Attack) in progress
• ♣ Situation Awareness Hypo
• Baseline, deviation from benign
♦
Intelligence
-Driven
(IoC)
❤
♠
Intelligence
Domain
Expertise
-Driven
(IoA)
• ♠ Domain Expertise Hypo
• Hunterʼs past experience
• Diﬃcult to document and share
• Cognitive bias? But valuable resource
SANS “Generating Hypotheses for Successful Threat Hunting” 2016
https://www.sans.org/white-papers/37172/
♣
Situation
Awareness
8


# Page. 9

![Page Image](https://bcdn.docswell.com/page/4JQV3N127P.jpg)

Hasegawa-style Drill Hunting
Based on EDA (Exploratory Data Analysis)
Instead of direct excavator, hands-on approach, drilling down layer by layer.
Scoping widely at ﬁrst、then dig deeper with more knowledge
Noise
Noise
FP
FP
FP
Dig! Dig! Dig! with
understanding Noise
Security Product Area
FP
FP
Security Product Area
Dig Wide at First
9


# Page. 10

![Page Image](https://bcdn.docswell.com/page/K74M1GRPJ1.jpg)

What is Data Visualization ?
• The art of turning data into visual context
• What would happen if with it ?
• Facilitate data understanding and speed up decision-making
• Eﬃcient information sharing
• Discover patterns and trends
Promotion of data understanding
Operational Eﬃciency
Total Solar Eclipse (皆既⽇⾷) Orbit Visualization (2017-2080) by Denise
Lu
10
https://www.washingtonpost.com/graphics/national/eclipse/


# Page. 11

![Page Image](https://bcdn.docswell.com/page/LJ18GDQXEG.jpg)

Relation between Hunting and Visualization
My Hunting Process
Data
source
Event
Category
Point of
View A
Point of
View B
Suspicious
Event
Term
Bird’s-eye
Insect’s eye
Visualization
Visualization
🦅
🐜
Deep Analysis
Search
11


# Page. 12

![Page Image](https://bcdn.docswell.com/page/GJWZKYMKJ2.jpg)

How to move from Bird to Insect, no silver bullet!
Look! Look! Inspect!
🦅
Bird eye
Data cut by 1
or 2 features/ﬁelds
Horizon
Normal events
Bubble
Understood data enough
Narrow down
while excluding normal/benign
Then, drill down!
Insect eye
🐜
Anomaly
Look at narrowed down
with many features/ﬁelds
Sunburst
Sankey
Multi-Dim Plot
12
Some icons from elastic


# Page. 13

![Page Image](https://bcdn.docswell.com/page/4EZ1ZXQN73.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
13


# Page. 14

![Page Image](https://bcdn.docswell.com/page/Y76LZ469JV.jpg)

Understanding data enough for Threat Hunting
• Most data for Threat Hunting =&gt; “Event” data
• i.e. Data with a time axis and continue to grow
• Understand all the data -&gt; Look at all the data
• Probably, impossible in enterprise !!
• Key = Data Aggregation
• Divide the data into parts and summarize the majority of the
data
Source
• How to divide?
Preprocessing
1. Data source ﬁltering
2. Term (Time span)
3. Event category ﬁltering
Term
Category
14


# Page. 15

![Page Image](https://bcdn.docswell.com/page/G751WQ6DE4.jpg)

Preprocessing on my hunting for data understanding
Events
Time series
aggregated
Dictionary
with datetime
for referral
Bucket events
with datetime
Lookup table
Long term
over 1mon
period
Short term
Multivariate
1h~1w period
analysis
Anomaly Detection
異常検出
多変量解析
15


# Page. 16

![Page Image](https://bcdn.docswell.com/page/9J21QPNM7R.jpg)

Advice for boundary whether you understand the data enough
1. Data Source
• Choose by your hunting scope
2. Term
• View event data for at least the past six months (half of year)
• 1 year for business which the events changes seasonally
3. Event Category
1.
2.
Check rare values by starting with the ﬁeld having smallest standard deviation
In Splunk, “ﬁeldsummary” command is useful.
4. Aggregation
1.
2.
For IP addresses and port numbers, group by range and attribution type
Reduce the data noise by excluding device and system-related events
Do NOT miss the changes of event data structure by system updating !!
e.g. New ﬁelds, New category values
Monitor the change of data structure itself
16


# Page. 17

![Page Image](https://bcdn.docswell.com/page/DEYZW5PP7M.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
17


# Page. 18

![Page Image](https://bcdn.docswell.com/page/VJN39N5MJ8.jpg)

What is SIEM dashboard used for?
MUST
Security Monitoring
CAN
SIEM
Forensics
Dash
CAN
Dashboard from
ü App built-in
ü Develop by ourselves
Threat Hunting
Board
CAN
SOAR
18


# Page. 19

![Page Image](https://bcdn.docswell.com/page/YE992RYWE3.jpg)

General SIEM Dashboards help our monitoring
+ World Maps
Source: TOP 25 KIBANA DASHBOARD EXAMPLES https://logit.io/blog/post/the-top-kibana-dashboards-and-visualisations/
19


# Page. 20

![Page Image](https://bcdn.docswell.com/page/GE895W6R7D.jpg)

But.. not enough for hunting
• General SIEM Dashboard are for security monitoring.
• During yearly, monthly, weekly and even daily monitoring, the information
is summarized in a large format so that any operator can access the same
information and make the same observations.
💣High missing possibility for the threat due to less information 🙃
In my experience,
any anomalies found in this “large-grained” dashboard are
either already triggering alerts from security devices or are false
positives.
Filtering Paradox
Even if you can fully understand what remained after ﬁltering, can you
understand what disappeared after easy ﬁltering?
20


# Page. 21

![Page Image](https://bcdn.docswell.com/page/LELWYN92JR.jpg)

Better viz dashboard ideas/tips for hunting
Inline
Documentation
Keep track of
how much preprocessing has
been done
Use panel’s title
and meta data
area
Graph
No use Pie chart
without time info
Line Chart
&amp;
Network Node
Graph
Drill down search is also useful.
Table
Keep time
information even
when stacking
(aggregation)
Create Summary
column with 4W
&lt;when&gt; &lt;who&gt;
&lt;what&gt; &lt;where&gt;
Highlight with
icons
✅❌💀🔥 etc.
21


# Page. 22

![Page Image](https://bcdn.docswell.com/page/4JM9NXZ9EW.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
22


# Page. 23

![Page Image](https://bcdn.docswell.com/page/PJRGDN69E9.jpg)

Favorite charts for hunting
• Bird-eye 🦅
• Line Chart
• Network Node Graph with line thickness and color bind
• Single Value Metrics for catching error or abnormal
• Insect-eye 🐜
• Table chart sophisticated with summary and categorical
color bind
• No need 3D, No need dynamic move =&gt; Make your
eye more exhausted 💦👁 👁💦
23


# Page. 24

![Page Image](https://bcdn.docswell.com/page/PEXX1NM3EX.jpg)

Example of viz hunting dashboard (1)🦅
DNS-based Data Exﬁltration
👇 Easy
Hunt !
Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2
24


# Page. 25

![Page Image](https://bcdn.docswell.com/page/3EKW2NGNED.jpg)

Example of viz hunting dashboard (2)🦅
Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2
25


# Page. 26

![Page Image](https://bcdn.docswell.com/page/L7314VNZJ5.jpg)

Example of viz hunting dashboard (3)🐜
👈 Preprocessing documentation (ﬁltering)
Spike timing is diﬀerent from each “src_ip” machine, suspicious of lateral movement.🧐
Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2
26


# Page. 27

![Page Image](https://bcdn.docswell.com/page/87DXQ8W4EG.jpg)

Example of viz hunting dashboard (4)🐜
👇 Drill down
Umm🧐, this isnʼt on 08-24, but just before DNS-based Data Exﬁltration on 08-26!
☞ lack of logging.. the event log was deleted? or 08-24 was more stealth behavior? ⚠FORENSIC
Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2
27


# Page. 28

![Page Image](https://bcdn.docswell.com/page/VJPPL89VJ8.jpg)

Network Forensic attempt for 08-24
Demo data: Splunk BOTSv2 datasets (tz: JST utc+9h) https://github.com/splunk/botsv2
💾 Download some attack tools via FTP
☞ Congratulations🎉
Hunting complete on Weaponization phase!!
28


# Page. 29

![Page Image](https://bcdn.docswell.com/page/2EV2QNMREQ.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative
AI THIS INFORMATION IS BASED ON THE VERSION AT JULY 2025
6. Magic of custom visualization
29


# Page. 30

![Page Image](https://bcdn.docswell.com/page/57GRWKG6EL.jpg)

Limitations of Visualization with GenAI
Data source: easy small Azure NetFlow data https://raw.githubusercontent.com/microsoft/msticpy/refs/heads/main/docs/notebooks/data/az_net_ﬂows.csv
Prompt: 「You are a professional threat hunter who detects anomalies from this Azure Netﬂow log.
Detect anomalies and visualize the abnormal areas with appropriate charts and graphs. We will leave the type of chart and the granularity of visualization to you.」
OpenAI ChatGPT Deep Research (o3)
• Ref: https://help.openai.com/en/articles/8437071-data-analysis-with-chatgpt
• Chart by Matplotlib, Network Node Graph hasn’t been supported yet.
• Cannot be emphasized automatically, needed a specific definition by prompt
• Tends to use traditional methods from the past
• Data conversion (preprocessing) looks black box without asking, and concerns about the accuracy
Microsoft Data Formulator (2023~)
• Ref: https://www.microsoft.com/en-us/research/blog/data-formulator-a-concept-driven-ai-powered-approachto-data-visualization/
• GenAI-powered data visualization tool (OSS), help us to transform data (preprocessing)
• Chart by Vega-Lite, many special charts haven’t been supported yet
• Concerns about the accuracy of data conversion
• Depends on LLM model brain linked by API
30


# Page. 31

![Page Image](https://bcdn.docswell.com/page/4EQV3NX2JP.jpg)

GenAI can&#039;t do it alone, still needs the collaborate with us
Data Formulator can check, modify charts and analyze more deeper interactively and sequentially by “Data Threads” feature,
“with singleness of purpose” (⼆⼈三脚)
31


# Page. 32

![Page Image](https://bcdn.docswell.com/page/KJ4M1G6PE1.jpg)

Agenda
1. Relationship between Threat Hunting and Data Visualization
2. What does it mean to fully understand your data?
3. Dashboard design challenges and practical tips
4. Introducing charts to dig deeper faster
5. Limitations of data visualization in current generative AI
6. Magic of custom visualization
32


# Page. 33

![Page Image](https://bcdn.docswell.com/page/LE18GDVX7G.jpg)

Magic of Custom Visualization
diﬀerent visualization methods with the same data on Splunk
Custom Viz can modify
- Layout
- Color palettes and Transparency
- Font and Size
- Tool tip, bread crumb, drill down options
“AS YOU WANT”
33


# Page. 34

![Page Image](https://bcdn.docswell.com/page/GEWZKYLKE2.jpg)

Magic of Custom Visualization (2)
Can attract audience with the
unfamiliar graph🤩
More interest at your analysis
process😳
Not necessary for hunting😅
Customizing needs design
sense🧐
Need description what this is👄
34


# Page. 35

![Page Image](https://bcdn.docswell.com/page/47Z1ZXMNJ3.jpg)

🦅
Now, Wrap Up!
üPoint to utilize visualization for hunting
🐜
ü🦅 Bird-eye by Graphs
Threat Hunter
ü🐜 Insect-eye by sophisticated Table Chart
ü[ Baby Step] Your Insects are put into positions after your birds see the
whole picture.
üDo NOT recommend just applying the SIEM monitoring
dashboards to Threat Hunting! Letʼs customize to go down a level
of granularity.
üAlong with describing what part of the data in whole is being shown there.
üRemain easy documentation like “Jupyter Notebook” in your dashboard
üCustom Visualization may have a worth for illustrating your
hunting process
35


# Page. 36

![Page Image](https://bcdn.docswell.com/page/YJ6LZ4N9EV.jpg)

Thank you!
36


