Running Edge0-35B on macOS with Metal, and chatting with it in Chap
Build the engine, fetch and convert the weights, start a local OpenAI-compatible server, and point Chap at it. Everything runs on your Mac.
Edge0 is an open-source
inference framework for large sparse mixture-of-experts models on consumer hardware. Its engine,
edge0-infer, runs the same model on macOS, iOS, Android and Windows, and includes a terminal
chat and an OpenAI-compatible local server.
As of 13 November 2026, the newer Edge0-145B and Edge0-120B models need an
NVIDIA GPU on Linux or Windows and have no macOS release. Edge0-35B runs on Apple Silicon
through edge0-infer with Metal. This guide sets that up and connects it to Chap.
What you need
- An Apple Silicon Mac (M1 or later) with Homebrew.
- About 42 GB of free disk: 20 GB for the downloaded weights and 22 GB for the converted ones. You can delete the downloaded copy after the conversion.
1. Install the tools
xcode-select --install
brew install cmake ninja uv2. Get the code and build it
mkdir -p ~/projects && cd ~/projects
git clone https://github.com/Edge0-AI/Edge0.git edge0
cd edge0/edge0-infer
cmake --preset mac-metal
cmake --build --preset mac-metal
build/mac-metal/bin/edge0-infer info # the list includes edge0_35b3. Download the weights
Run these from ~/projects/edge0/edge0-infer. The export lines and the
assets_py helper only last for the current terminal, so run them again whenever you open a new one.
export MOE_SOURCE_35B=~/models/edge0-35b-src
export MOE_ASSETS_35B=~/models/edge0-35b-assets
assets_py() { uv run --project tools/assets --offline --no-sync python -I "$@"; }
mkdir -p ~/models
uv sync --project tools/assets
assets_py tools/assets/edge0_35b/fetch_source.py probe
assets_py tools/assets/edge0_35b/fetch_source.py download --out "$MOE_SOURCE_35B"If the download stops, run the last command again. It picks up where it left off.
4. Convert the weights
This takes about three to four minutes.
assets_py tools/assets/edge0_35b/make_assets.py --source "$MOE_SOURCE_35B" --out "$MOE_ASSETS_35B"
It prints Converted and checked … when it succeeds. If it fails with a .partial
error, delete the half-written folder and run the conversion again:
rm -rf ~/models/edge0-35b-assets.partialOnce the conversion has succeeded, you can free 20 GB by deleting the downloaded copy:
rm -rf ~/models/edge0-35b-src5. Run it
edge0-infer has two ways to run the model:
| Command | What it does |
|---|---|
chat | A chat in the terminal. Other apps can't connect to it. |
serve | An OpenAI-compatible API at http://127.0.0.1:8000/v1. This is the one Chap uses. |
# a chat in the terminal
build/mac-metal/bin/edge0-infer chat --model ~/models/edge0-35b-assets --device metal
# or the local API server
build/mac-metal/bin/edge0-infer serve --model ~/models/edge0-35b-assets --device metalThe first start is slower while Metal compiles its shaders. With the server running, check it from a second terminal:
curl http://127.0.0.1:8000/v1/modelsThe answer names the model id you'll pick in Chap, edge0_35b, and its 16,384-token context window:
{"object":"list","data":[{"id":"edge0_35b","object":"model","created":0,"owned_by":"local",
"context_length":16384,"capabilities":{"thinking":true,"thinking_default":false}}]}6. Connect Chap
In Chap, open Settings › Endpoints and choose + Add endpoint. The same Settings page appears in the browser extension and in the desktop app. Fill in the endpoint like this:
| Field | Value |
|---|---|
| Endpoint name | Anything you like, e.g. Edge0 (local) |
| Provider | Custom |
| API style | OpenAI-compatible |
| Base URL | http://127.0.0.1:8000/v1 |
| Authentication | API key. Leave the key empty, or type local. |
Choose Load models and pick edge0_35b (or type the id the curl
check printed). Then choose Test, then Save. Start a new chat and select the
Edge0 model in the model picker.
Chap talks to the server on 127.0.0.1, and the model runs on your Mac, so your messages don't
leave the machine. Once the weights are downloaded, it works offline.
7. Keep the server running in the background
To have the server start when you log in and restart if it stops, give it to launchd. Stop any
serve you started by hand first, or the port is already taken. The command below writes the
service file with your home folder filled in, because launchd doesn't expand ~.
mkdir -p ~/Library/LaunchAgents
cat > ~/Library/LaunchAgents/ai.edge0.serve.plist <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key><string>ai.edge0.serve</string>
<key>ProgramArguments</key>
<array>
<string>$HOME/projects/edge0/edge0-infer/build/mac-metal/bin/edge0-infer</string>
<string>serve</string>
<string>--model</string><string>$HOME/models/edge0-35b-assets</string>
<string>--device</string><string>metal</string>
</array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>StandardOutPath</key><string>/tmp/edge0-serve.log</string>
<key>StandardErrorPath</key><string>/tmp/edge0-serve.log</string>
</dict>
</plist>
EOF
plutil -lint ~/Library/LaunchAgents/ai.edge0.serve.plist
launchctl load ~/Library/LaunchAgents/ai.edge0.serve.plistTo stop it and keep it from starting at login:
launchctl unload ~/Library/LaunchAgents/ai.edge0.serve.plistThe server's output goes to /tmp/edge0-serve.log.
Troubleshooting
| What you see | What to do |
|---|---|
cmake: command not found | brew install cmake ninja |
build/cpu/bin/edge0-infer is missing | On a Mac the binary is in build/mac-metal/bin/. Build it first (step 2). |
llama.cpp reports a missing per_layer_token_embd.weight | llama.cpp can't load Edge0, and the NVFP4 weights are NVIDIA-only. Use edge0-infer. |
assets_py not found, or the variables are empty | You're in a new terminal. Run the export lines and the assets_py line from step 3 again. |
A .partial error during conversion | Delete ~/models/edge0-35b-assets.partial and run the conversion again. |
Chap says it couldn't reach 127.0.0.1:8000 | You started chat instead of serve, or the server has stopped. Start serve (step 5 or 7). |
address already in use | Another server is already on port 8000. lsof -i :8000 shows which process. |