Phase 0: Mapping the Landscape – What Existing SIEMs Teach and What They Miss
I’m currently developing Project Jupiter, a self-hosted, multi-tenant SIEM + SOAR platform designed with OCSF-native architecture. The landing page (projectjupiter.in) serves as a showcase of the project’s ongoing research and development, highlighting how ML and AI—through embeddings, vector databases, and LLM-driven analysis—can enhance threat detection, power RAG-based investigations and automate response playbooks.
Before I built anything for Project Jupiter, I wanted to understand the space I was stepping into. There are many SIEM and SOAR tools already available. Some are open source, some are enterprise heavyweights, and almost all of them promise complete visibility and security.
So the first real question I asked myself was simple. If all these tools already exist, why do analysts still struggle with noise, complexity and cost?
That question shaped the second step of Phase 0.
What I Looked At
My research fell into three groups.
Open Source and Community Platforms
I examined tools like Wazuh, Security Onion and the ELK stack.
Each had strengths, but each also had gaps. Some were powerful but heavy. Some flexible but incomplete. Some were simple but lacked depth.
Still, every one of them contributed something to my understanding.
Enterprise Ecosystems and Real Operations
A major part of my learning came from my time at ATOS, where I worked around their internal SIEM ecosystem. I will not describe anything about the platform itself, but the environment around it taught me a lot.
I worked with architecture teams, implementation teams, SOC analysts, threat hunters, production support, big data teams, log onboarding specialists and UI and UX groups.
Seeing how these teams interact in real incidents gave me a strong picture of what enterprise scale looks like and what a SIEM must handle every day.
It showed me that a SIEM is not a single tool. It is a living ecosystem that touches almost everything.
Industry Standards and Frameworks
I also studied frameworks like OCSF, MITRE ATT&CK and more.
They helped me understand what a modern security platform should align with in terms of structure, events and behavior.
What Existing Tools Do Well
Despite their problems, there are many things SIEMs get right.
Parsing and Normalization
Most mature tools handle logs from many different sources with impressive consistency.
Data Handling at Scale
Enterprise SIEMs are designed to handle huge data volumes while keeping correlation and indexing reliable.
Interfaces and Dashboards
Years of iteration and feedback cycles have given these tools polished dashboards, search views and timelines.
Governance and Compliance
Audit views, long term storage, access control and reporting are usually solid and reliable.
These strengths formed the baseline of what Jupiter should eventually grow into.
Where They Struggle
Even with their strengths, SIEMs share a number of common challenges.
Cost
Most SIEM platforms are extremely expensive. Even the open source ones become costly once data volume grows.
Vendor Dependence
Many tools tie you into a specific cloud or ecosystem. Flexibility becomes limited.
Complexity
Running a SIEM often requires several teams. Tuning rules, managing infrastructure, keeping ingestion healthy and handling noise all require constant work.
AI That Does Not Truly Help
Many tools advertise AI features, but most of them do not fundamentally improve triage or analysis. They act more like separate tools, not integrated intelligence.
Limited Self-Hosted Options
There are very few tools that can be installed and used fully on a local machine or home lab. Most require cloud infrastructure or heavy dependencies.
These gaps became the reasons Jupiter should exist at all.
What This Research Taught Me
Looking at all these tools helped me identify the principles that guide Jupiter.
It should run anywhere
Laptop, old hardware, enterprise server, cloud VM or anything in between. The tool should adapt to the environment.
It should be modular
Users should enable or disable components according to their needs. Not everyone requires everything.
OCSF from day one
One schema for all data makes everything more understandable and more future proof.
AI should be integrated correctly
AI should help interpret events, not become another noisy feature.
Local inference when possible, API fallback when needed.
It should be friendly to solo developers and small teams
People should be able to experiment without requiring a dedicated engineering team.
These ideas now act as the foundation for Jupiter’s architecture.
Personal Reflections
What surprised me
Most SIEM tools are far more similar internally than they appear from the outside. The differences tend to be polish, performance and scale, not the underlying logic.
What changed my mind
I initially believed that removing features made a system simpler. I later realized that clarity is what makes a system simple.
Users need to understand why the system produces an alert or a view, not just what it shows.
What Comes Next
The next article in Phase 0 will explore Jupiter’s architecture itself. It will focus on how all these lessons shaped the first structural decisions and why each component exists.
