What “running locally” means
The model's files are downloaded once and run on your own computer. Your prompts and documents never leave the machine, and it keeps working without an internet connection.
The pieces
- Llama 3 — a family of open-weight models released by Meta in April 2024, first in 8-billion and 70-billion-parameter sizes.
- Ollama — a free tool for downloading and running open models on Windows, macOS and Linux with simple commands.
Getting started
Install Ollama
Download the installer for your system from the Ollama project and install it.
Download and run a model
In a terminal, run
ollama run llama3. The first time, it downloads the model; then you can chat with it directly.Use it from other apps
Ollama also serves a local API on your computer, so other programs on the same machine can use the model.
Hardware
Smaller models such as the 8-billion-parameter version run on an ordinary modern computer with enough memory, and faster with a graphics card. The largest models need far more memory and a powerful GPU. Compressed ("quantised") versions trade a little quality for much lower memory use.
Local AI at ZIBADIS
ZIBADIS AI Lab runs its models on ZIBADIS's own servers — 56 models in 17 categories, with no external API.
Tools for this
Asaiejadoo — everyday calculation and AI guides, part of the ZIBADIS network founded by Masoud Moghaddam in Tehran.
