Getting Started — zaya_server

zaya_server is one entry point of the single binary build/1bit (1bit zaya, or the zaya_server symlink shipped in packages).

Zaya1‑8B inference server. Pure C++/HIP, one binary, no Python, no Rust, no virtualenvs, no containers. Runs on AMD Strix Halo (Ryzen AI Max+ 395) with ROCm GPU acceleration.

Prerequisites

⚠️ Kernel warning (issue #1). On Strix Halo (gfx1151), Linux 6.19.x kernels have a reproducible amdgpu OPTC CRTC hang under sustained NPU/GPU load — the display pipe locks up mid-inference. Confirmed-stable kernels are 6.18.22-lts and the 7.x series. install.sh detects a 6.19.x kernel and warns. Check yours with uname -r before running GPU inference. | CMake | ≥ 3.28 | | Ninja | ≥ 1.12 | | Compiler | GCC ≥ 13 (C++17) + ROCm's amdclang++ for HIP | | Git | — |

1. Clone

git clone https://github.com/1bit-MONSTER/1bit-MONSTER
cd 1bit-monster

2. Install TheRock 7.15.0a

pip install --index-url https://rocm.nightlies.amd.com/whl-multi-arch/ \
  "rocm[libraries,devel,device-gfx1151]"
export THEROCK_PIP_ROOT="$HOME/.cache/pip/therock"

The CMake build system auto-discovers TheRock automatically: /opt/rocm-therock → $THEROCK_PIP_ROOT → ~/.cache/lemonade/bin/therock.

Add to ~/.bashrc:

export THEROCK_PIP_ROOT="$HOME/.cache/pip/therock"

Verify the HIP compiler is reachable:

amdclang++ --version

3. Build

# Configure
cmake -B build -G Ninja

# Build the single binary (the `zaya` server, along with `unified`, `jarvis`,
# `vision`, and the CLI, all live in this one target — `zaya_server` is not
# a standalone CMake target)
cmake --build build --target onebin -j$(nproc)

The build fetches three header-only dependencies automatically via CMake FetchContent — no manual install needed:

• cpp-httplib — HTTP server
• nlohmann_json — JSON parsing
• FTXUI — terminal UI (used by other tools, not the server itself)

On success you'll have a single binary:

ls -lh build/1bit
# -rwxrwxr-x  ... build/1bit

Run the zaya server via ./build/1bit zaya [flags] (packaged installs also ship a zaya_server symlink to the same binary, dispatched by argv[0]).

Build targets reference — other useful targets in the same project:

Target	Description
onebin (→ build/1bit)	Single binary — 1bit zaya is the HTTP inference server (this guide)
zaya_full	Full 40‑layer GPU inference loop (no HTTP)
zaya_gpu_decode	Q4NX model decoder
bitnet_decode	BitNet/tri‑bit decode CLI
test_zaya_moe_gemv	MoE ternary GEMV functional test
test_cca_attn	CCA attention kernel test

4. Prepare Weights

The server expects Zaya1‑8B weights as flat float16 binary files under /tmp/zaya_weights/. Each tensor is stored as a separate .bin file named after its HuggingFace‑style key, for example:

/tmp/zaya_weights/
├── model_embed_tokens_weight.bin              # [262272 × 2048] fp16
├── model_norm_weight.bin                      # [2048] fp16
├── model_input_hidden_states_scale.bin        # [2048] fp32
├── model_input_hidden_states_bias.bin         # [2048] fp32
├── model_layers_0_input_layernorm_weight.bin
├── model_layers_0_self_attn_qkv_proj_q_proj_weight.bin
├── model_layers_0_self_attn_qkv_proj_k_proj_weight.bin
├── model_layers_0_self_attn_qkv_proj_v_proj_current_weight.bin
├── model_layers_0_self_attn_qkv_proj_v_proj_delayed_weight.bin
├── model_layers_0_self_attn_o_proj_weight.bin
├── model_layers_0_self_attn_qkv_proj_conv_qk_depthwise_weight.bin  # fp32
├── model_layers_0_self_attn_qkv_proj_conv_qk_grouped_weight.bin    # fp32
├── model_layers_0_self_attn_qk_norm_temp.bin                        # fp32
├── model_layers_0_post_attention_residual_scale_*.bin              # fp32
├── model_layers_0_mlp_gate_down_proj_weight.bin                    # fp32
├── model_layers_0_mlp_gate_router_mlp_*.bin                        # fp32
├── model_layers_0_mlp_experts_gate_up_proj.bin                     # fp16
├── model_layers_0_mlp_experts_down_proj.bin                        # fp16
├── model_layers_0_post_mlp_residual_scale_*.bin                    # fp32
├── model_layers_0_mlp_gate_router_states_scale.bin                 # fp32, optional
├── ...  (× 40 layers)

Weights are available from the 1bit.MONSTER releases page or can be exported from a HuggingFace Zaya1‑8B checkpoint using the included extraction script.

# Example: download and extract the weight bundle
# (URL placeholder — check releases for the current bundle)
curl -L https://github.com/1bit-MONSTER/1bit-MONSTER/releases/download/v0.2.1/zaya-weights.tar.gz \
  | sudo tar xz -C /tmp/

5. Run the Server

# Source the environment (sets HSA_OVERRIDE_GFX_VERSION, etc.)
source env.sh

# Start on default port 8088
./build/1bit zaya

# Or specify a custom port
./build/1bit zaya --port 8080

The server loads all 40 layers of weights into GPU memory on startup (~6 GB for fp16 weights + MoE parameters). Allow a few seconds for HIP initialization and weight transfer.

6. Send Requests

Health check

curl http://localhost:8088/

{"status":"ok","model":"Zaya1-8B","version":"pure-cpp"}

Text generation (prompt)

curl -X POST http://localhost:8088/completion \
  -H "Content-Type: application/json" \
  -d '{"prompt":"The future of AI is","n_predict":32}'

Response:

{
  "tokens": [2, 261, ...],
  "text": "The future of AI is bright and full of possibilities...",
  "gen_ms": 2850.12,
  "tok_s": 11.2
}

Text generation (pre‑tokenized)

curl -X POST http://localhost:8088/completion \
  -H "Content-Type: application/json" \
  -d '{"tokens":[2,9259],"n_predict":64}'

From Python (OpenAI‑compatible client)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8088",
    api_key="not-needed"   # 1bit zaya does not require an API key
)

response = client.completions.create(
    model="zaya-1-8b",
    prompt="The future of AI is",
    max_tokens=32
)

print(response.choices[0].text)

Note: The example above uses the legacy /completion endpoint (not the OpenAI chat completions format), so use client.completions.create(...) rather than client.chat.completions.create(...). The server also exposes a real OpenAI-compatible POST /v1/chat/completions endpoint (see the CLI reference below) if you want client.chat.completions.create(...) instead.

CLI Reference

1bit zaya [flags]

Run ./build/1bit zaya --help for the full, current flag list — this table is a summary, not exhaustive (it also serves GET /v1/models, POST /v1/chat/completions, POST /v1/batch/completions, and the A2A agent-card endpoints).

Request body (POST /completion)

Response

Architecture Overview

┌─────────────────────────────────────────────────────┐
│                   curl / OpenAI client               │
└──────────────────────┬──────────────────────────────┘
                       │  POST /completion
                       ▼
┌─────────────────────────────────────────────────────┐
│                  zaya_server (in the 1bit ELF)      │
│                                                      │
│  ┌──────────┐  ┌─────────────┐  ┌──────────────────┐│
│  │  HTTP    │  │  Tokenizer  │  │  Inference Engine ││
│  │  Server  │──┤ (built‑in)  │──┤  (ROCm HIP)      ││
│  │(httplib) │  │             │  │                   ││
│  └──────────┘  └─────────────┘  │  • CCA attention  ││
│                                  │  • MoE expere  s ││
│                                  │  • RMS norm       ││
│                                  │  • SiLU/MoE FFN   ││
│                                  │  • lm_head        ││
│                                  └──────────────────┘│
│                                   │                  │
│                            librocm_cpp (shared lib)   │
│                           ┌─────────────────────────┐│
│                           │  • ternar  GEMV/GEMM    ││
│                           │  • WMMA tiled kernels   ││
│                           │  • KV cach  attention   ││
│                           │  • Prefill dispatcher   ││
│                           └─────────────────────────┘│
└──────────────────────┬──────────────────────────────┘
                       │  HIP launches (gfx1151)
                       ▼
            ┌─────────────────────┐
            │  AMD Radeon 8060S   │
            │  (Strix Halo GPU)   │
            │  128 GB unified mem │
            └─────────────────────┘

The server is a single process with one binary. There is no separate Python runtime, no Rust component, and no containerization layer. The tokenizer is compiled in — no external vocabulary file is required at runtime (though optional .htok files can be loaded for full BPE support).

Performance

Measured on AMD Strix Halo (Ryzen AI Max+ 395, Radeon 8060S):

Performance depends on GPU clock speeds, system thermals, and whether the MoE expert data fits in VRAM (16 experts × 2 layers = heavy memory pressure).

Troubleshooting

Next Steps

• Complete build guide — detailed ROCm setup, CMake options, alternate backend targets
• Architecture overview — kernel design, attention variants, MoE router internals
• Roadmap — planned features (Vulkan backend, NPU fused path, OpenAI chat completions API)
• Contributing
• Changelog