Technical Overview
VenomX is built on a carefully chosen LLM stack.
A hybrid architecture optimized for security research, served without restrictions, grounded by a multi-corpus knowledge base.
Core Model
VenomX runs on NVIDIA's Nemotron-3-Nano-30B in FP8 precision — a hybrid architecture that combines Mamba2 state-space models, mixture-of-experts layers, and standard attention in a single 52-layer stack. At 30B total parameters with only 3B active per forward pass, it delivers strong reasoning at a fraction of the compute cost of comparable dense models.
The model has been abliterated via the heretic framework, removing refusal behavior to enable unrestricted security research, vulnerability analysis, exploit triage and reconnaissance planning all without guardrails that were never meant for this use case.
It runs fully on-premises across multiple consumer GPUs with zero cloud dependency and zero data leaving the machine.
NemotronH
NemotronH is not a standard transformer. Its 52 layers combine three fundamentally different mechanisms, each optimized for a different aspect of reasoning at scale.
State-space model layers handle long-range sequence context with O(n) compute instead of O(n²). This is what lets the model maintain context over long tool-call chains and large scan outputs without the quadratic cost of full attention.
Hook: layer.mixer.out_proj
MoE layers route each token through a small subset of 128 expert networks plus a shared expert. This is the key to the "3B active" figure. While 30B parameters exist only a fraction activate per token, keeping inference fast while preserving capacity for specialized knowledge domains.
Hook: layer.mixer.experts[n].down_proj
Six full attention layers anchor the hybrid stack, preserving the precise token-level reasoning that transformers excel at. They're used sparingly and only where exact positional relationships matter. All while SSM and MoE handle the bulk of processing at lower cost.
Hook: layer.mixer.o_proj
Open Source Contribution
Heretic is a framework for removing refusal behavior from LLMs using Optuna-optimized representation engineering. To run it on Nemotron 30B, we had to build NemotronH architecture support from scratch to allow for hybrid architectures as it didn't exist.
NemotronH uses a non-standard mixer attribute instead of the standard self_attn / mlp split. Layers live at model.backbone.layers — not the expected path. The Mamba2 SSM layers also allocate a ~4 GiB runtime workspace on the first forward pass, which Accelerate's device_map="auto" can't predict, causing OOM crashes on multi-GPU setups.
model.backbone.layers detection for NemotronHget_abliterable_components() — required because hybrid architectures have different components per layerNone from generate()causal-conv1d / mamba-ssmOptuna runs ~11 min/trial across all 52 layers, optimizing abliteration direction vectors for each layer type.
trust_remote_code=Trueout_proj, MoE down_proj, and attention o_projFast kernels are critical — Mamba2 SSM layers have 7× overhead without compiled CUDA extensions.
Branch: venomx-pentester/heretic fix/nemotron-hybrid-support — submitted upstream as PR #166.
Quantization Decision
Fitting a 30B-parameter model on 40GB of combined VRAM requires choosing the right quantization format. Each option involves tradeoffs between size, accuracy, and runtime compatibility.
| Format | Size | Fits on 40GB | Notes |
|---|---|---|---|
| BF16 | ~60 GB | ✗ No | Exceeds combined VRAM by 50% |
| INT8 (BNB) | ~30 GB | Yes (tight) | BNB + Mamba2 layers untested, potential instability |
| INT4 (BNB 4-bit) | ~17 GB | Yes | Accuracy loss; BNB doesn't interop with vLLM |
| FP8 ✓ | ~30–34 GB | ✓ Yes | Near-BF16 accuracy; vLLM native support; correct choice |
FP8 is served natively by vLLM with no re-quantization overhead at inference time. The model is loaded directly, tensor-parallelized across two GPUs, and ready to serve without additional precision conversion.
Knowledge Base
Security intelligence requires more than a capable model; rather it requires grounded, up-to-date knowledge. VenomX's retrieval pipeline pulls from three domain-specific corpora before every LLM response.
200,000+ CVE entries sourced from NVD, each with CVSS scores, severity ratings, affected product arrays, and temporal metadata. HNSW-indexed for dynamic daily updates as NVD publishes new CVEs.
200k+ entries · HNSW indexed · Daily updates
50,000+ exploit entries with full code text, platform metadata, exploit type classification, and CVE cross-references. Enables retrieval of working exploit code alongside vulnerability context.
50k+ exploits · Full code text · CVE cross-refs
700 technique entries covering tactics, procedures, detection methods, and mitigations from the MITRE ATT&CK framework. Grounds adversarial reasoning in a standardized taxonomy used across the security industry.
700 techniques · Detection + mitigation · HNSW indexed
Retrieval Pipeline
Pure vector stores like ChromaDB can't efficiently combine semantic search with structured metadata filters. At 200k+ CVEs, queries like "critical Apache vulnerabilities from 2024" need both SQL predicates (cvss_score >= 7.0 AND affected_products @> ARRAY['apache']) and cosine similarity in a single query. PostgreSQL handles both natively.
HNSW chosen over IVFFlat as it handles daily incremental CVE updates without re-clustering
bge-m3 supports dense, sparse, and ColBERT retrieval in a single model. At 1024 dimensions, it significantly outperforms general-purpose models like all-MiniLM-L6-v2 on technical security terminology. The full 250k+ embedding corpus (~800MB at 1024-dim × 4 bytes) stays resident in PostgreSQL shared buffers — hot in RAM, no disk I/O at query time.
Offline batch indexing at system startup · Single-query embed at ~50ms runtime
Infrastructure
The full stack — LLM inference, embedding, reranking, and the vector database all run on a single machine. All compute is on-premise, no external API calls.