Skip to content
Private LLM
Theme
← Blog

Heretic vs Abliterated: Compare Uncensored LLM Methods

PublishedUpdated
13 min read

Shop for an uncensored open-weight model in Private LLM and you'll see two labels everywhere: abliterated and heretic. Both strip a model's refusal behavior. They don't tell you the same thing about quality, though.

Heretic vs abliterated is two ways of doing the same surgery. Abliteration projects out a refusal direction by hand. Heretic automates that, then tunes it per layer. A Heretic checkpoint that publishes lower KL divergence at a similar refusal rate kept more of the base model intact.

Abliteration, introduced by Arditi et al. in 2024 and popularized by Maxime Labonne, projects a single "refusal direction" out of model weights. Heretic, released by Philipp Emanuel Weidmann in late 2025, generalizes that technique with per-layer tuning and a TPE optimizer that co-minimizes refusals and KL divergence. This post covers what each method does, which Heretic and Abliterated checkpoints Private LLM ships, and which one to load first on your iPhone, iPad, or Mac.

Key Takeaways

  • An abliterated LLM strips a single refusal direction from the model weights using the technique Andy Arditi and colleagues described in 2024.
  • A heretic LLM applies the 2025 Heretic tool, which automates abliteration with Optuna-based optimization and can lower KL divergence at the same refusal rate.
  • Private LLM currently ships Qwen3 4B Heretic and Qwen3 4B Heretic NoSlop, plus a broader Abliterated model set from 1B to 70B.
  • Qwen3 4B Heretic publishes KL divergence 0.43; many Abliterated model cards do not publish a directly comparable KL/refusal pair.
  • Pick Heretic when you want the Qwen3 4B checkpoint with disclosed KL; pick Abliterated when you want broader model-family coverage.

Table of Contents

What Is an Abliterated LLM, and Where Did Abliteration Come From?

An abliterated LLM is an open-weight model with its refusal behavior edited straight out of the weights. Not retrained away. Edited. The technique finds a refusal direction inside the model's activations and projects it out of selected weights. That's fast. But the same hidden features that carry refusals often overlap with reasoning, math, and instruction following, so the edit can damage those too.

The method starts with the finding that refusal behavior in instruction-tuned language models is mediated by a single direction in the residual stream. Run the model on a batch of harmful prompts and a batch of harmless prompts, record the hidden activations at the last token position, and take the mean difference. That vector is the refusal direction. Project it out of the embedding matrix, every attention output projection, and every MLP output projection, and the model can no longer represent refusal. The technique was written up in "Refusal in LLMs Is Mediated by a Single Direction" by Andy Arditi and colleagues, packaged into a practical notebook by FailSpy, and popularized by Maxime Labonne's abliteration guide.

The Cost of Manual Abliteration

Labonne's own writeup is candid about the tradeoff: "we observe a performance drop in the ablated version across all benchmarks. The ablation process successfully uncensored it but also degraded the model's quality." Recovery normally requires a second training pass. Labonne used DPO on the orpo-dpo-mix-40k dataset to produce NeuralDaredevil 8B, which restored most of the lost capability. That added roughly six hours and 45 minutes of training time on six A6000 GPUs. A raw abliterated LLM, shipped without the DPO recovery step, usually loses math performance first. GSM8K drops are a common tell.

NeuralDaredevil 8B is also one of the abliterated LLMs Private LLM ships, alongside deterministic Huihui-team checkpoints such as Qwen3 4B Abliterated, Llama 3.3 70B Abliterated, DeepSeek R1 Distill Llama 8B Abliterated, and the smaller Llama 3.2 1B and 3B Abliterated variants. These older Abliterated releases matter because they give Private LLM users broader model-family coverage than Heretic currently does.

What Is Heretic, and Why Did It Replace Manual Abliteration for Some Models?

Heretic automates directional ablation instead of running it by hand. It searches per-layer settings to minimize refusal count and KL divergence from the base model at the same time, so the published goal is fewer refusals with less drift on safe prompts.

Heretic is a command-line tool from Philipp Emanuel Weidmann, released under the AGPL v3.0 license in late 2025. It takes the same directional-ablation core and adds three changes that matter:

  • A flexible per-layer weight kernel. Instead of uniform ablation weights across layers, Heretic parameterizes each layer with max_weight, max_weight_position, min_weight, and min_weight_distance. The optimizer picks the shape.
  • An interpolated refusal direction. The direction index is a float rather than an integer. Non-integer values linearly interpolate between adjacent refusal direction vectors, which expands the search space far beyond the one direction per layer the original method exposes.
  • Per-component parameters. Attention out-projections and MLP down-projections get separate ablation weights, because, as the Heretic README puts it, "MLP interventions tend to be more damaging to the model than attention interventions."

On top of that, Heretic wraps the whole thing in an Optuna TPE optimizer that co-minimizes two objectives at once: refusal count on a harmful prompt set, and KL divergence against the original model on a harmless prompt set. No human picks layers. No human tunes weights. Runtime depends on the model size, trial count, prompt set, and GPU used for the run.

Private LLM currently ships Heretic in the Qwen3 4B family: Qwen3 4B Heretic and the Numen fine-tuned Qwen3 4B Heretic NoSlop variant. That narrower Heretic coverage is why the app still carries a broader Abliterated set across Llama, Gemma, Phi, DeepSeek, and Qwen families.

Which Uncensored Method Should You Pick?

Choosing between Heretic and Abliterated checkpoints starts with one number: measured refusal count. Base model quality and community testing settle the rest. Call it abliterated vs heretic, the established 2024 method against its 2025 automated successor. Either way, the decision comes down to the checkpoint: pick whichever refuses less on prompts that match your use case, then let community maturity break the tie.

Heretic Wins When

  • The exact checkpoint publishes fewer refusals than available abliterated variants on comparable prompt sets.
  • The model card discloses refusal counts, prompt-set details, and enough context to judge the run quality.
  • You want Qwen3 4B Heretic or the Numen fine-tuned Qwen3 4B Heretic NoSlop variant in Private LLM.

Abliterated Wins When

  • You need a deterministic, well-tested artifact. Huihui checkpoints are reproducible and heavily used, which surfaces failure modes faster than a fresh Heretic run would.
  • The base model has a known abliterated variant available in Private LLM, and there is no shipped Heretic counterpart for that family.
  • You plan to DPO-heal the model afterwards on domain data. Abliteration plus DPO is the longer-tested recovery path.

Private LLM Heretic and Abliterated Models

Private LLM ships Heretic and Abliterated checkpoints as Apple-device quantized builds, not raw full-precision Hugging Face weights. Pick the uncensored method with the strongest disclosed quality numbers first. Then pick the model size that fits your iPhone, iPad, or Mac.

Heretic and abliterated checkpoints ship as full-precision models on Hugging Face. Running them on a phone requires quantization. Private LLM ships GPTQ and OmniQuant quantization, both originally published research methods (GPTQ from Frantar et al. 2022, OmniQuant from OpenGVLab 2023) tuned per model on Apple hardware, rather than the RTN quantization used by llama.cpp-based apps. That tuning is what makes a 4B abliterated LLM or heretic LLM feel usable on an iPhone 15 Pro instead of a laboratory curiosity. Our own benchmarks put 3-bit OmniQuant quality at parity with 4-bit RTN on the same hardware, which is the part of the stack that decides whether the uncensored LLM you picked is actually faster than Ollama on a Mac.

Here's the full Heretic and Abliterated model list shipping in Private LLM today, sorted by hardware tier:

iPhone Tier (4 GB to 6 GB RAM)

Qwen3 4B Heretic on iPhone providing uncensored creative writing feedback in Private LLM
Qwen3 4B Heretic running on iPhone in Private LLM, reused from the Qwen3 4B Heretic release post.
Qwen3 4B Abliterated on iPhone in Private LLM for local character roleplay
Qwen3 4B Abliterated running on iPhone in Private LLM, reused from the Qwen3 4B Abliterated release post.

iPad and 16 GB Mac Tier (8 GB to 16 GB RAM)

Qwen3 4B Heretic NoSlop on Mac generating noir fiction in Private LLM
Qwen3 4B Heretic NoSlop on Mac, reused from the Qwen3 4B Heretic release post.

Mac-Only Tier (32 GB and 48 GB RAM)

Llama 3.3 70B Abliterated writing mature fiction locally in Private LLM on Mac
Llama 3.3 70B Abliterated running on Mac in Private LLM, reused from the Llama 3.3 70B Uncensored release post.

Heretic coverage in Private LLM stays concentrated in the Qwen3 4B family right now. Abliterated coverage runs wider, since the technique is older and manual checkpoints already cover more model families.

Objection Handling

Two objections come up every time. Should you just run Heretic yourself? Does uncensored behavior hold up in long chats? Both answers are mixed. Heretic is reproducible for model builders, but phone use still needs quantization. Both methods can leave soft refusals or moralizing prose behind.

"Can I not just run Heretic myself on my own hardware?" Yes. You will get a full-precision output that still needs quantization before it runs on an iPhone. Private LLM packages, quantizes, tunes, and tests supported checkpoints for Apple devices. For NoSlop, Numen fine-tuned an exclusive variant to reduce AI-slop prose while preserving the uncensored behavior.

"Will the censorship creep back in long contexts?" Sometimes, on both methods. Refusal directions are a first-order description of alignment, not a complete one. Both Heretic and abliteration leave small numbers of soft refusals, especially on safety-lecture topics like harassment or self-harm. Longer contexts do not resurface hard refusals reliably, but they can resurface moralizing prose. The Heretic NoSlop variant targets that specifically.

FAQ

Here's what readers ask most after comparing the two labels: what changed inside the model, how broadly Heretic applies, whether the models run offline on iPhone, and whether older Abliterated checkpoints still deserve a look. Every answer below stays tied to the checkpoint, not the method name.

What Is the Difference Between Abliterated and Heretic Models?

Abliterated models apply a single ablation weight per layer against a single refusal direction computed by mean difference. Heretic models apply per-layer weight kernels against an interpolated refusal direction, with separate attention and MLP parameters, tuned by an Optuna TPE optimizer that co-minimizes refusals and KL divergence. Same goal, different algorithm, usually lower capability damage at the same refusal rate on Heretic.

Which Heretic Models Does Private LLM Ship?

Private LLM currently ships Qwen3 4B Heretic and Qwen3 4B Heretic NoSlop. The standard Heretic checkpoint comes from p-e-w; the NoSlop variant is fine-tuned by Numen to reduce AI-slop prose while preserving uncensored behavior. Other uncensored families in Private LLM are Abliterated, not Heretic.

Can I Run Heretic Models Offline on iPhone?

Yes. Download Qwen3 4B Heretic or Qwen3 4B Heretic NoSlop inside the Private LLM app once, and every subsequent inference runs on-device. No API key, no account, no internet required after the download. Conversation history stays on the device.

What Is an Abliterated LLM?

In short: a weight-edited open model that no longer refuses. The edit projects a single refusal direction, computed from the mean difference between hidden activations on harmful and harmless prompts, out of the embedding matrix and every attention and MLP output projection. The technique is from Andy Arditi and colleagues' 2024 paper. The cost is capability damage on reasoning and math, which is usually patched with a DPO recovery pass on a clean preference dataset.

What Is the Heretic LLM Tool?

Heretic is an open-source command-line tool from Philipp Emanuel Weidmann, released in late 2025, that automates abliteration with per-layer parameter tuning and a TPE optimizer. It minimizes refusals while also minimizing KL divergence from the base model, and applies different ablation strengths to attention out-projections and MLP down-projections. Outputs full-precision Hugging Face checkpoints; quantization is required before phone use.

Are Abliterated Models Still Worth Using in 2026?

Yes. Abliterated LLM checkpoints remain useful in Private LLM because they cover more model families: Llama 3.2, Llama 3.1, Llama 3.3, Meta Llama 3, DeepSeek R1 Distill, Gemma 3, Phi, and Qwen3. Heretic is the sharper Qwen3 4B option when you want disclosed KL and the NoSlop variant.

Pick the Right Uncensored LLM for Your Apple Device

The right uncensored LLM for your Apple device is the checkpoint that balances refusal removal, quality preservation, and hardware fit, not the label on the model card. Private LLM ships Heretic, the Numen fine-tuned NoSlop variant, and Abliterated options, so you can test those tradeoffs yourself instead of taking a label's word for it.

Heretic and abliterated checkpoints solve the same surface problem: an open-weight model that refuses requests out of the box. Heretic is the Qwen3 4B path in Private LLM when you want disclosed KL and NoSlop. Abliterated is the broader path when you want more base-model choices, including Llama 3.3 70B on Mac. Private LLM ships both methods with OmniQuant and GPTQ quantization tuned for Apple Silicon: one-time purchase, Family Sharing across six people, no subscription. For the wider category, see the best uncensored AI chat apps for iPhone, iPad, and Mac. For wider context on running these checkpoints locally, see Private LLM compared to LM Studio and Ollama.

Every Heretic and Abliterated checkpoint Private LLM ships is on the uncensored models page, sorted by the device it fits.

Ready to test the difference yourself? Download Private LLM. Load a Heretic and an Abliterated variant of the same base model. Compare them side by side. The KL divergence numbers are real, and you'll feel it within a few turns of conversation.