We are excited to announce that the Llama 3 Smaug 8B model by Abacus.AI is now part of Private LLM for iOS and macOS. This powerful new model brings enhanced conversational abilities to your personal AI chatbot that runs locally on your device.

On iOS, we offer a 3-bit quantized version of Llama 3 Smaug 8B, while on macOS, we provide a 4-bit quantized version. The macOS version is compatible with both Intel and Apple Silicon Macs.

It's only been a week since the Llama 3 models were released, and we already have three Llama 3 8B-based downloadable models in Private LLM for iOS. Many more are on the way!

What is Llama 3 Smaug 8B?

Llama 3 Smaug 8B builds upon the strong foundation of Meta's Llama 3 8B model. Abacus.AI applied techniques from the Smaug paper to further optimize the model for engaging in realistic, multi-turn dialogues. The result is a state-of-the-art language model tailored for chatbot interactions.

A key aspect of Llama 3 Smaug 8B is the use of Direct Preference Optimisation (DPO) during fine-tuning. DPO leverages preferred and dispreferred example pairs to train the model to favor more desirable responses.

However, the researchers at Abacus.AI discovered that standard DPO can sometimes inadvertently decrease the likelihood of preferred responses. To address this, they developed DPO-Positive (DPOP), a modified loss function and training approach. DPOP ensures the model's probability of generating preferred completions increases, leading to better performance across many datasets and downstream tasks.

To get the best experience with Llama 3 Smaug 8B, we recommend using an iPhone or iPad with at least 8GB of RAM. This allows the model to fully leverage its capabilities while running on your device.

Screenshot of Llama 3 Smaug 8B Running Locally on iPhone — Llama 3 Smaug 8B Running on iPhone

Screenshot of Llama 3 Smaug 8B Running on Mac — Llama 3 Smaug 8B Running on Mac

Screenshot of Llama 3 Smaug 8B Running on iPad — Llama 3 Smaug 8B Running on iPad

Experience Llama 3 Smaug 8B in Private LLM

Ready to try out Llama 3 Smaug 8B? Simply download the latest version of Private LLM from the App Store or Mac App Store. With this update, you'll have access to a highly capable AI assistant that respects your privacy by operating entirely on your local hardware.

Dive into engaging conversations and experience the impressive language understanding and generation of Llama 3 Smaug 8B. Your personal AI chatbot just got smarter on both iOS and macOS.