Blog · · Guide

Running Edge0-35B on macOS with Metal, and chatting with it in Chap

Build the engine, fetch and convert the weights, start a local OpenAI-compatible server, and point Chap at it. Everything runs on your Mac.

Edge0 is an open-source inference framework for large sparse mixture-of-experts models on consumer hardware. Its engine, edge0-infer, runs the same model on macOS, iOS, Android and Windows, and includes a terminal chat and an OpenAI-compatible local server.

As of 13 November 2026, the newer Edge0-145B and Edge0-120B models need an NVIDIA GPU on Linux or Windows and have no macOS release. Edge0-35B runs on Apple Silicon through edge0-infer with Metal. This guide sets that up and connects it to Chap.

What you need

1. Install the tools

xcode-select --install
brew install cmake ninja uv

2. Get the code and build it

mkdir -p ~/projects && cd ~/projects
git clone https://github.com/Edge0-AI/Edge0.git edge0
cd edge0/edge0-infer
cmake --preset mac-metal
cmake --build --preset mac-metal
build/mac-metal/bin/edge0-infer info   # the list includes edge0_35b

3. Download the weights

Run these from ~/projects/edge0/edge0-infer. The export lines and the assets_py helper only last for the current terminal, so run them again whenever you open a new one.

export MOE_SOURCE_35B=~/models/edge0-35b-src
export MOE_ASSETS_35B=~/models/edge0-35b-assets
assets_py() { uv run --project tools/assets --offline --no-sync python -I "$@"; }

mkdir -p ~/models
uv sync --project tools/assets
assets_py tools/assets/edge0_35b/fetch_source.py probe
assets_py tools/assets/edge0_35b/fetch_source.py download --out "$MOE_SOURCE_35B"

If the download stops, run the last command again. It picks up where it left off.

4. Convert the weights

This takes about three to four minutes.

assets_py tools/assets/edge0_35b/make_assets.py --source "$MOE_SOURCE_35B" --out "$MOE_ASSETS_35B"

It prints Converted and checked … when it succeeds. If it fails with a .partial error, delete the half-written folder and run the conversion again:

rm -rf ~/models/edge0-35b-assets.partial

Once the conversion has succeeded, you can free 20 GB by deleting the downloaded copy:

rm -rf ~/models/edge0-35b-src

5. Run it

edge0-infer has two ways to run the model:

CommandWhat it does
chatA chat in the terminal. Other apps can't connect to it.
serveAn OpenAI-compatible API at http://127.0.0.1:8000/v1. This is the one Chap uses.
# a chat in the terminal
build/mac-metal/bin/edge0-infer chat  --model ~/models/edge0-35b-assets --device metal

# or the local API server
build/mac-metal/bin/edge0-infer serve --model ~/models/edge0-35b-assets --device metal

The first start is slower while Metal compiles its shaders. With the server running, check it from a second terminal:

curl http://127.0.0.1:8000/v1/models

The answer names the model id you'll pick in Chap, edge0_35b, and its 16,384-token context window:

{"object":"list","data":[{"id":"edge0_35b","object":"model","created":0,"owned_by":"local",
  "context_length":16384,"capabilities":{"thinking":true,"thinking_default":false}}]}

6. Connect Chap

In Chap, open Settings › Endpoints and choose + Add endpoint. The same Settings page appears in the browser extension and in the desktop app. Fill in the endpoint like this:

FieldValue
Endpoint nameAnything you like, e.g. Edge0 (local)
ProviderCustom
API styleOpenAI-compatible
Base URLhttp://127.0.0.1:8000/v1
AuthenticationAPI key. Leave the key empty, or type local.

Choose Load models and pick edge0_35b (or type the id the curl check printed). Then choose Test, then Save. Start a new chat and select the Edge0 model in the model picker.

Chap talks to the server on 127.0.0.1, and the model runs on your Mac, so your messages don't leave the machine. Once the weights are downloaded, it works offline.

7. Keep the server running in the background

To have the server start when you log in and restart if it stops, give it to launchd. Stop any serve you started by hand first, or the port is already taken. The command below writes the service file with your home folder filled in, because launchd doesn't expand ~.

mkdir -p ~/Library/LaunchAgents
cat > ~/Library/LaunchAgents/ai.edge0.serve.plist <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>ai.edge0.serve</string>
  <key>ProgramArguments</key>
  <array>
    <string>$HOME/projects/edge0/edge0-infer/build/mac-metal/bin/edge0-infer</string>
    <string>serve</string>
    <string>--model</string><string>$HOME/models/edge0-35b-assets</string>
    <string>--device</string><string>metal</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
  <key>StandardOutPath</key><string>/tmp/edge0-serve.log</string>
  <key>StandardErrorPath</key><string>/tmp/edge0-serve.log</string>
</dict>
</plist>
EOF
plutil -lint ~/Library/LaunchAgents/ai.edge0.serve.plist
launchctl load ~/Library/LaunchAgents/ai.edge0.serve.plist

To stop it and keep it from starting at login:

launchctl unload ~/Library/LaunchAgents/ai.edge0.serve.plist

The server's output goes to /tmp/edge0-serve.log.

Troubleshooting

What you seeWhat to do
cmake: command not foundbrew install cmake ninja
build/cpu/bin/edge0-infer is missingOn a Mac the binary is in build/mac-metal/bin/. Build it first (step 2).
llama.cpp reports a missing per_layer_token_embd.weightllama.cpp can't load Edge0, and the NVFP4 weights are NVIDIA-only. Use edge0-infer.
assets_py not found, or the variables are emptyYou're in a new terminal. Run the export lines and the assets_py line from step 3 again.
A .partial error during conversionDelete ~/models/edge0-35b-assets.partial and run the conversion again.
Chap says it couldn't reach 127.0.0.1:8000You started chat instead of serve, or the server has stopped. Start serve (step 5 or 7).
address already in useAnother server is already on port 8000. lsof -i :8000 shows which process.
← All posts · New to Chap? Install it · Questions? See the FAQ.