Deploy: hosted Advanced build offers downloadable builder ZIP + hackathon README
Browse files- saas/gradio/app.py: Advanced "Build" on a physical-drive target now exports
the real builder ZIP for download in a hosted Space (like the Light flow),
instead of dead-ending. Reworded the hosted message (FR+EN).
- README.md: Build Small hackathon Space card with track/badge YAML tags,
rubric-alignment table, links to demo video / social post / field notes.
- submission/FIELD_NOTES.md, submission/thumbnail.svg: report + title card
referenced by the README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- README.md +145 -135
- saas/gradio/app.py +54 -2
- submission/FIELD_NOTES.md +66 -0
- submission/thumbnail.svg +64 -0
README.md
CHANGED
|
@@ -1,202 +1,208 @@
|
|
| 1 |
---
|
| 2 |
title: JackAILocal
|
| 3 |
-
emoji:
|
| 4 |
colorFrom: green
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.16.0
|
| 8 |
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
-
pinned:
|
|
|
|
|
|
|
| 11 |
hf_oauth: true
|
| 12 |
hf_oauth_scopes:
|
| 13 |
- inference-api
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
-
# JackAILocal
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
| 23 |
-
WebUI -> jackailocald -> local Ollama model
|
| 24 |
-
```
|
| 25 |
|
| 26 |
-
|
| 27 |
|
| 28 |
-
##
|
| 29 |
|
| 30 |
-
|
| 31 |
-
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
- Offline speech-to-text through whisper.cpp
|
| 36 |
-
- Offline text-to-speech through Piper
|
| 37 |
-
- Local documents and bilingual Field Manual cards
|
| 38 |
-
- AES-256-GCM encrypted import/export packs for threads, documents, and safe settings
|
| 39 |
-
- Opt-in Phone Access on the local network with pairing token and QR code
|
| 40 |
-
- Hardware status, model availability, benchmark, settings, and privacy-safe support ZIP
|
| 41 |
-
- English and French WebUI
|
| 42 |
|
| 43 |
-
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
backends/whisper.cpp/windows/whisper-cli.exe
|
| 52 |
-
backends/piper/windows/piper.exe
|
| 53 |
-
models/whisper/ggml-base.bin
|
| 54 |
-
models/piper/en_US-libritts_r-medium.onnx
|
| 55 |
-
webui/
|
| 56 |
-
config/
|
| 57 |
-
content/
|
| 58 |
-
content-packs/
|
| 59 |
-
```
|
| 60 |
|
| 61 |
-
|
| 62 |
|
| 63 |
-
|
| 64 |
|
| 65 |
-
|
| 66 |
|
| 67 |
-
|
| 68 |
-
START-HERE.cmd
|
| 69 |
-
```
|
| 70 |
|
| 71 |
-
|
| 72 |
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
|
|
|
| 76 |
|
| 77 |
-
|
| 78 |
|
| 79 |
-
```
|
| 80 |
-
|
| 81 |
```
|
| 82 |
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
## Build Targets
|
| 86 |
-
|
| 87 |
-
Windows launchers:
|
| 88 |
-
|
| 89 |
-
```cmd
|
| 90 |
-
BUILD-LOCAL.cmd
|
| 91 |
-
BUILD-USB.cmd
|
| 92 |
-
BUILD-SSD.cmd
|
| 93 |
-
```
|
| 94 |
|
| 95 |
-
|
| 96 |
|
| 97 |
-
|
| 98 |
-
./BUILD-LOCAL-MAC.command
|
| 99 |
-
./BUILD-USB-MAC.command
|
| 100 |
-
./BUILD-SSD-MAC.command
|
| 101 |
-
```
|
| 102 |
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
-
|
| 106 |
|
| 107 |
-
|
| 108 |
-
2. Validate that the target does not overlap the source package.
|
| 109 |
-
3. Copy the runtime payload (including `legal/`, `licenses/`, and the PC analyzer in `profiler/`).
|
| 110 |
-
4. Install selected Ollama models into the target model store.
|
| 111 |
-
5. Install optional LoRA adapters from `manifest/lora-adapters.json` (see `docs/LORA_ADAPTERS.md`).
|
| 112 |
-
6. Install and verify Voice assets.
|
| 113 |
-
7. Generate a SHA-256 file manifest.
|
| 114 |
-
8. Refuse to report success if required runtime assets or legal documents are missing.
|
| 115 |
|
| 116 |
-
|
| 117 |
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
the configuration agent (Gemma 4 12B, `gemma4:12b`) reviews the plan.
|
| 123 |
|
| 124 |
-
|
| 125 |
|
| 126 |
-
|
| 127 |
-
and applied on the customer machine with `UPDATE-FROM-USB.cmd` (auto-detects
|
| 128 |
-
the update package on removable drives, verifies the RSA signature and all
|
| 129 |
-
SHA-256 checksums, asks for confirmation, keeps a rollback backup). See also
|
| 130 |
-
`updates/JackAILocal-Update-Offline.ps1 -Rollback`.
|
| 131 |
|
| 132 |
-
|
| 133 |
|
| 134 |
-
|
| 135 |
-
tooling: `jackailocald license keygen | issue | verify`. The runtime exposes
|
| 136 |
-
`/api/license`, and the WebUI shows the license state under Settings. Model
|
| 137 |
-
sources and backends (Ollama, Hugging Face GGUF/safetensors, NVIDIA NIM) are
|
| 138 |
-
documented in `docs/MODEL_SOURCES.md`.
|
| 139 |
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
|
| 142 |
-
|
| 143 |
|
| 144 |
-
|
| 145 |
-
pip install -r requirements.txt
|
| 146 |
-
python app.py
|
| 147 |
-
```
|
| 148 |
-
|
| 149 |
-
The Gradio application now starts with a simple flow for non-technical users: choose the intended use, choose a target, then prepare or publish. It can install to this computer, write a USB key or external SSD when running locally on Windows, or publish Windows, macOS, and Linux/Docker appliance ZIPs. Hosted Gradio exports packages only; it does not impersonate a client runtime.
|
| 150 |
|
| 151 |
-
##
|
| 152 |
|
| 153 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
|
| 155 |
-
|
| 156 |
-
powershell -ExecutionPolicy Bypass -File installer/windows/Build-Installer.ps1
|
| 157 |
-
```
|
| 158 |
|
| 159 |
-
|
| 160 |
|
| 161 |
```bash
|
| 162 |
-
|
|
|
|
|
|
|
| 163 |
```
|
| 164 |
|
| 165 |
-
|
| 166 |
|
| 167 |
-
|
| 168 |
|
| 169 |
-
```
|
| 170 |
-
|
| 171 |
-
JACKAILOCAL_MODEL=qwen3.5:4b docker compose up --build -d
|
| 172 |
-
./show-url.sh
|
| 173 |
```
|
| 174 |
|
| 175 |
-
|
| 176 |
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
Run the destructive boot-disk preparation only from Linux and only with the correct target device:
|
| 180 |
-
|
| 181 |
-
```bash
|
| 182 |
-
sudo linux/factory/prepare-linux-boot-disk.sh /dev/sdX
|
| 183 |
-
```
|
| 184 |
-
|
| 185 |
-
The script builds the runtime, stages Ollama, builds whisper.cpp, installs Voice assets, preloads the selected model, creates a Debian UEFI disk, and enables the JackAILocal services.
|
| 186 |
|
| 187 |
-
## Security
|
| 188 |
|
| 189 |
-
- Loopback bind by default
|
| 190 |
- No shell tools exposed to the model
|
| 191 |
- No private update key in exported packages
|
| 192 |
-
-
|
| 193 |
-
- AES transfer pack passphrases are never stored
|
| 194 |
- Support ZIP excludes conversations and documents
|
| 195 |
-
- SHA-256 package manifest
|
| 196 |
|
| 197 |
-
JackAILocal does not claim protection from
|
| 198 |
|
| 199 |
-
|
|
|
|
|
|
|
| 200 |
|
| 201 |
```powershell
|
| 202 |
cargo check
|
|
@@ -206,4 +212,8 @@ node --check webui/app-v15.js
|
|
| 206 |
python qa/static_audit.py
|
| 207 |
```
|
| 208 |
|
| 209 |
-
See `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: JackAILocal
|
| 3 |
+
emoji: 🔒
|
| 4 |
colorFrom: green
|
| 5 |
+
colorTo: gray
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.16.0
|
| 8 |
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
+
pinned: true
|
| 11 |
+
license: other
|
| 12 |
+
short_description: Private AI that runs 100% offline — no cloud, ever.
|
| 13 |
hf_oauth: true
|
| 14 |
hf_oauth_scopes:
|
| 15 |
- inference-api
|
| 16 |
+
models:
|
| 17 |
+
- Qwen/Qwen3.5-4B
|
| 18 |
+
- Qwen/Qwen3.6-27B
|
| 19 |
+
- google/gemma-4-12B-it
|
| 20 |
+
tags:
|
| 21 |
+
- gradio
|
| 22 |
+
- build-small-hackathon
|
| 23 |
+
- track:backyard
|
| 24 |
+
- sponsor:modal
|
| 25 |
+
- achievement:offgrid
|
| 26 |
+
- achievement:offbrand
|
| 27 |
+
- achievement:llama
|
| 28 |
+
- achievement:fieldnotes
|
| 29 |
+
- backyard ai
|
| 30 |
+
- backyard-ai
|
| 31 |
+
- off the grid
|
| 32 |
+
- off-the-grid
|
| 33 |
+
- off brand
|
| 34 |
+
- off-brand
|
| 35 |
+
- llama champion
|
| 36 |
+
- llama-champion
|
| 37 |
+
- llama.cpp
|
| 38 |
+
- tiny titan
|
| 39 |
+
- tiny-titan
|
| 40 |
+
- best agent
|
| 41 |
+
- best-agent
|
| 42 |
+
- best demo
|
| 43 |
+
- best-demo
|
| 44 |
+
- bonus quest champion
|
| 45 |
+
- bonus-quest-champion
|
| 46 |
+
- community choice
|
| 47 |
+
- community-choice
|
| 48 |
+
- modal
|
| 49 |
+
- best use of modal
|
| 50 |
+
- best-use-of-modal
|
| 51 |
+
- field notes
|
| 52 |
+
- field-notes
|
| 53 |
+
- zerogpu
|
| 54 |
+
- local
|
| 55 |
+
- offline
|
| 56 |
+
- privacy
|
| 57 |
---
|
| 58 |
|
| 59 |
+
# 🔒 JackAILocal — your AI, sealed in a box
|
| 60 |
|
| 61 |
+
**Private AI that runs 100% offline. No cloud. No account. No data leaving the machine — ever.**
|
| 62 |
|
| 63 |
+
JackAILocal turns any laptop, USB stick, external SSD, or LAN box into a complete private AI workspace: chat, voice, vision, and documents — all running on small open models on the device itself. This Space is the **one-click builder** that configures and ships that offline workspace for you. The AI you build never phones home.
|
| 64 |
|
| 65 |
+
> 🤗 **Hugging Face Build Small Hackathon** · Track: **Backyard AI** · Every model is ≤32B and the default runtime ships on a **4B** model.
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
---
|
| 68 |
|
| 69 |
+
## 🎬 See it work
|
| 70 |
|
| 71 |
+
| | |
|
| 72 |
+
|---|---|
|
| 73 |
+
| **▶️ Demo video** | `<ADD VIDEO URL>` — real laptop, **Wi-Fi physically off**, answering questions, transcribing voice, and reading an image with zero network. |
|
| 74 |
+
| **🐦 Social post** | `<ADD SOCIAL POST URL>` |
|
| 75 |
+
| **📝 Build report (field notes)** | [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
+
---
|
| 78 |
|
| 79 |
+
## ⏱️ For judges: evaluate in 60 seconds
|
| 80 |
|
| 81 |
+
1. **Open the Builder tab above.** Pick a use case (e.g. *"Private document assistant"*) and a target (e.g. *Windows ZIP*).
|
| 82 |
+
2. Click **Sign in with Hugging Face** → **Prepare / publish**. The **AI configuration agent** (Gemma 4 12B, served on **Modal**) reviews your hardware, picks a compliant small model, and returns a validated build plan.
|
| 83 |
+
3. Open the **audit JSON** — every decision shows the **full agent trace**: the exact prompt sent and the raw model output. Nothing is faked; when no model is configured the panel says so instead of inventing an answer.
|
| 84 |
+
4. Watch the **demo video** to see the *output* of that build — a sealed AI running on a real machine with the network cut.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
+
That is the whole pitch: this Space **configures and ships** the AI; the **video proves** it runs with no cloud.
|
| 87 |
|
| 88 |
+
---
|
| 89 |
|
| 90 |
+
## 🌳 The problem (Backyard AI)
|
| 91 |
|
| 92 |
+
Most "local AI" tools are a thin wrapper around an API key. The moment the Wi-Fi drops — or a clinic, law office, field site, or privacy-conscious household refuses to send data to someone else's server — they stop working.
|
|
|
|
|
|
|
| 93 |
|
| 94 |
+
People who actually need AI off the grid have no good option:
|
| 95 |
|
| 96 |
+
- a **nurse** in a rural clinic who can't upload patient notes to a cloud,
|
| 97 |
+
- a **shop owner** who wants a document assistant but not a subscription that reads their books,
|
| 98 |
+
- a **parent** who wants a homework helper that works on the cabin trip with no signal,
|
| 99 |
+
- a **regulated SMB** that legally cannot let data leave the building.
|
| 100 |
|
| 101 |
+
JackAILocal is built for them. You configure it here, ship it to a USB/SSD/installer/LAN box, and from that point on it is **completely self-contained**. The runtime path is simply:
|
| 102 |
|
| 103 |
+
```text
|
| 104 |
+
WebUI → jackailocald (Rust) → local Ollama / llama.cpp model
|
| 105 |
```
|
| 106 |
|
| 107 |
+
There is **no cloud inference fallback**. A capability is shown as *unavailable* when its local binary or model is missing — it is never quietly replaced by a remote call.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 108 |
|
| 109 |
+
---
|
| 110 |
|
| 111 |
+
## 🏆 Why this wins (rubric alignment)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
+
| Badge / award | How JackAILocal earns it |
|
| 114 |
+
|---|---|
|
| 115 |
+
| 🌲 **Track: Backyard AI** | A real, polished tool that solves a real daily problem: private AI for people and SMBs who can't or won't use the cloud. |
|
| 116 |
+
| 🔌 **Off the Grid** | The shipped runtime does **100% local inference**. The demo video is recorded with the network physically disconnected. |
|
| 117 |
+
| 🐤 **Tiny Titan** (≤4B) | The default runtime chat model is **Qwen3.5 4B** — the everyday experience runs on a genuinely tiny model. |
|
| 118 |
+
| 🎨 **Off Brand** | A custom, themed builder console plus a fully hand-built offline **WebUI** (not default Gradio) shipped with the product. |
|
| 119 |
+
| 🤖 **Best Agent** | The configuration step is a real **multi-step decision engine**: it can `ASK_USER`, `ASK_HUMAN_REVIEW`, `REJECT`, or emit a build action — with a policy gate that has final authority — and every step exposes its agent trace. |
|
| 120 |
+
| 🦙 **llama.cpp** | Ships an optional `llama.cpp` OpenAI-compatible server path alongside Ollama for the runtime. |
|
| 121 |
+
| ☁️ **Best Use of Modal** | The hosted configuration agent (Gemma 4 12B IT) is served from a **Modal vLLM** endpoint. |
|
| 122 |
+
| 📓 **Field Notes** | A full build report is included — see [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md). |
|
| 123 |
+
| 🌟 **Bonus Quest Champion** | One submission, stacking the most bonus criteria at once. |
|
| 124 |
|
| 125 |
+
---
|
| 126 |
|
| 127 |
+
## 🤖 The configuration agent (Best Agent)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
|
| 129 |
+
The Builder does **not** treat the model as a chatbot. It treats it as an **internal decision engine** that must return schema-validated JSON. The UI sends the selected use case and target, the hardware profile, the available backend/model plan, client constraints, and any previous answers. The model must then return one of:
|
| 130 |
|
| 131 |
+
- `ASK_USER` — with concrete questions when critical info is missing,
|
| 132 |
+
- `ASK_HUMAN_REVIEW` — when a human operator should approve,
|
| 133 |
+
- `REJECT` — when the request is unsafe or non-compliant,
|
| 134 |
+
- a **build action** — the validated packaging plan.
|
|
|
|
| 135 |
|
| 136 |
+
A deterministic **policy gate** still has final authority: it blocks USB builds without a real target, backend mismatches, missing secrets, and unsafe manifest patches — even if the model says otherwise. The remote agent can only modify the **packaging manifest**; the shipped runtime stays local and offline once models are preloaded.
|
| 137 |
|
| 138 |
+
Crucially, **no fake decision is ever generated.** If no agent endpoint is configured, the panel reports the missing secrets explicitly rather than fabricating an "AI decision."
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
|
| 140 |
+
---
|
| 141 |
|
| 142 |
+
## 🧠 Small-model stack (everything ≤32B)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
|
| 144 |
+
| Role | Model | Size | Where |
|
| 145 |
+
|---|---|---|---|
|
| 146 |
+
| Default runtime chat | `Qwen/Qwen3.5-4B` | 4B | On device (Ollama) |
|
| 147 |
+
| Power profile (24GB VRAM) | `Qwen/Qwen3.6-27B` | 27B | On device (Ollama) |
|
| 148 |
+
| Configuration agent | `google/gemma-4-12B-it` | 12B | Modal vLLM (build-time only) |
|
| 149 |
+
| Speech-to-text | `whisper.cpp` (ggml-base) | — | On device |
|
| 150 |
+
| Text-to-speech | Piper (`en_US-libritts_r-medium`, CC BY 4.0) | — | On device |
|
| 151 |
+
| Vision / OCR (SCOUT) | any installed vision Ollama model | ≤32B | On device |
|
| 152 |
|
| 153 |
+
No model above the 32B cap is ever offered. The default demo deliberately runs on a 4B model so it works on an ordinary laptop without a 24GB GPU.
|
| 154 |
|
| 155 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 156 |
|
| 157 |
+
## 🧩 What the shipped product includes
|
| 158 |
|
| 159 |
+
- Local chat through installed Ollama models, with an optional `llama.cpp` server path
|
| 160 |
+
- Persistent conversation threads (filter, rename, delete)
|
| 161 |
+
- **SCOUT** image analysis and text extraction via a local vision model
|
| 162 |
+
- Offline **speech-to-text** (whisper.cpp) and **text-to-speech** (Piper)
|
| 163 |
+
- Local documents and a bilingual **Field Manual**
|
| 164 |
+
- **AES-256-GCM** encrypted import/export packs for threads, documents, and safe settings
|
| 165 |
+
- Opt-in **Phone Access** on the local network with pairing token + QR code
|
| 166 |
+
- Hardware status, model availability, benchmark, and a privacy-safe support ZIP
|
| 167 |
+
- **English and French** WebUI
|
| 168 |
+
- Ed25519 offline licensing and signed, rollback-capable offline updates
|
| 169 |
|
| 170 |
+
---
|
|
|
|
|
|
|
| 171 |
|
| 172 |
+
## 🚀 Run locally
|
| 173 |
|
| 174 |
```bash
|
| 175 |
+
pip install -r requirements.txt
|
| 176 |
+
python app.py
|
| 177 |
+
# open http://127.0.0.1:7860
|
| 178 |
```
|
| 179 |
|
| 180 |
+
The hosted Space **only builds and publishes packages** (Windows / macOS / Linux-Docker ZIPs + a standalone PC analyzer). It never pretends to run a client runtime for you — when you ship the package and launch it on a real machine, *that* is where the offline AI lives.
|
| 181 |
|
| 182 |
+
Build targets (run locally on Windows/macOS):
|
| 183 |
|
| 184 |
+
```cmd
|
| 185 |
+
BUILD-LOCAL.cmd BUILD-USB.cmd BUILD-SSD.cmd
|
|
|
|
|
|
|
| 186 |
```
|
| 187 |
|
| 188 |
+
Each builder validates the payload, installs the selected Ollama model(s), installs Voice assets, writes a SHA-256 manifest, and **refuses to report success** if required runtime or legal assets are missing.
|
| 189 |
|
| 190 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 191 |
|
| 192 |
+
## 🛡️ Security boundaries
|
| 193 |
|
| 194 |
+
- Loopback bind by default; pairing token required for any non-loopback access
|
| 195 |
- No shell tools exposed to the model
|
| 196 |
- No private update key in exported packages
|
| 197 |
+
- AES transfer-pack passphrases are never stored
|
|
|
|
| 198 |
- Support ZIP excludes conversations and documents
|
| 199 |
+
- SHA-256 package manifest + signed-manifest hooks
|
| 200 |
|
| 201 |
+
JackAILocal does not claim protection from an already-compromised host OS.
|
| 202 |
|
| 203 |
+
---
|
| 204 |
+
|
| 205 |
+
## ✅ Verification
|
| 206 |
|
| 207 |
```powershell
|
| 208 |
cargo check
|
|
|
|
| 212 |
python qa/static_audit.py
|
| 213 |
```
|
| 214 |
|
| 215 |
+
See [`submission/FIELD_NOTES.md`](submission/FIELD_NOTES.md) for the build story. The full source, the Rust runtime, and the package builders live in the [project's GitHub repository](https://github.com/JacquesGariepy/jackailocal).
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
*Built for the 🤗 Build Small Hackathon. Small models. Big privacy. Off the grid.*
|
saas/gradio/app.py
CHANGED
|
@@ -475,7 +475,7 @@ L10N = {
|
|
| 475 |
"target_ready_started": "Cible prête. Runtime démarré.",
|
| 476 |
"target_ready_attention": "Cible prête. Démarrage à vérifier : {detail}",
|
| 477 |
"console_reset": "Réinitialisation de la console... Démarrage du flux de compilation...",
|
| 478 |
-
"build_hosted_unavailable": "La création d'un disque physique est
|
| 479 |
"err_select_drive": "Sélectionnez un lecteur ou dossier cible réel.",
|
| 480 |
"build_stop_processes": "Arrêt des processus runtime locaux pour libérer les verrous de fichiers...",
|
| 481 |
"build_warn_shutdown": "Avertissement pendant l'arrêt des processus : {detail}",
|
|
@@ -740,7 +740,7 @@ JackAILocal applique des frontières de sécurité strictes entre Internet (SaaS
|
|
| 740 |
"target_ready_started": "Target ready. Runtime started.",
|
| 741 |
"target_ready_attention": "Target ready. Start needs attention: {detail}",
|
| 742 |
"console_reset": "Resetting console... Starting target build flow...",
|
| 743 |
-
"build_hosted_unavailable": "Physical drive build
|
| 744 |
"err_select_drive": "Select a real target drive or folder.",
|
| 745 |
"build_stop_processes": "Ensuring local runtime processes are stopped to release file locks...",
|
| 746 |
"build_warn_shutdown": "Warning during process shutdown: {detail}",
|
|
@@ -3883,6 +3883,52 @@ def make_footer_html(lang: str) -> str:
|
|
| 3883 |
</div>
|
| 3884 |
"""
|
| 3885 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3886 |
def build_ui() -> gr.Blocks:
|
| 3887 |
with gr.Blocks(title=get_txt("app_title", "EN")) as demo:
|
| 3888 |
# States
|
|
@@ -4548,6 +4594,12 @@ def build_ui() -> gr.Blocks:
|
|
| 4548 |
run_physical_usb_build,
|
| 4549 |
inputs=[language, sample, profile_json, package_goal, target_drive, target_mode, decision_answers, decision_notes],
|
| 4550 |
outputs=[console_log, usb_ready, sidebar_status_html, btn_start_usb, btn_stop_usb]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4551 |
).then(
|
| 4552 |
update_chat_models,
|
| 4553 |
inputs=[language, sample, profile_json],
|
|
|
|
| 475 |
"target_ready_started": "Cible prête. Runtime démarré.",
|
| 476 |
"target_ready_attention": "Cible prête. Démarrage à vérifier : {detail}",
|
| 477 |
"console_reset": "Réinitialisation de la console... Démarrage du flux de compilation...",
|
| 478 |
+
"build_hosted_unavailable": "La création d'un disque physique est impossible dans une Space hébergée : le cloud n'a aucun accès à votre clé USB/SSD. Le vrai ZIP builder est préparé ci-dessous pour téléchargement — exécutez-le sur l'ordinateur cible pour graver le disque.",
|
| 479 |
"err_select_drive": "Sélectionnez un lecteur ou dossier cible réel.",
|
| 480 |
"build_stop_processes": "Arrêt des processus runtime locaux pour libérer les verrous de fichiers...",
|
| 481 |
"build_warn_shutdown": "Avertissement pendant l'arrêt des processus : {detail}",
|
|
|
|
| 740 |
"target_ready_started": "Target ready. Runtime started.",
|
| 741 |
"target_ready_attention": "Target ready. Start needs attention: {detail}",
|
| 742 |
"console_reset": "Resetting console... Starting target build flow...",
|
| 743 |
+
"build_hosted_unavailable": "Physical drive build can't run inside a hosted Space: the cloud has no access to your USB/SSD. The real builder ZIP is being prepared for download below — run it on the target computer to write the drive.",
|
| 744 |
"err_select_drive": "Select a real target drive or folder.",
|
| 745 |
"build_stop_processes": "Ensuring local runtime processes are stopped to release file locks...",
|
| 746 |
"build_warn_shutdown": "Warning during process shutdown: {detail}",
|
|
|
|
| 3883 |
</div>
|
| 3884 |
"""
|
| 3885 |
|
| 3886 |
+
def _platform_key_from_profile(raw_profile_json: str) -> str:
|
| 3887 |
+
"""Pick a builder platform (windows/macos) from the hardware profile."""
|
| 3888 |
+
try:
|
| 3889 |
+
prof = json.loads(raw_profile_json) if raw_profile_json and raw_profile_json.strip() else {}
|
| 3890 |
+
os_val = str(prof.get("target_os", "") or prof.get("os", "")).lower()
|
| 3891 |
+
except Exception:
|
| 3892 |
+
os_val = ""
|
| 3893 |
+
if "mac" in os_val or "darwin" in os_val or "osx" in os_val:
|
| 3894 |
+
return "macos"
|
| 3895 |
+
return "windows"
|
| 3896 |
+
|
| 3897 |
+
|
| 3898 |
+
def export_builder_zip_for_hosted(
|
| 3899 |
+
language: str,
|
| 3900 |
+
sample_name: str,
|
| 3901 |
+
raw_profile_json: str,
|
| 3902 |
+
package_goal: str,
|
| 3903 |
+
simple_target: str,
|
| 3904 |
+
target_mode: str,
|
| 3905 |
+
target_drive: str,
|
| 3906 |
+
decision_answers_json: str = "",
|
| 3907 |
+
operator_notes: str = "",
|
| 3908 |
+
):
|
| 3909 |
+
"""Advanced flow, hosted Space: a cloud Space can't write a physical drive,
|
| 3910 |
+
so instead of dead-ending we export the real builder ZIP for download —
|
| 3911 |
+
exactly like the Light one-click flow does. Locally the physical build
|
| 3912 |
+
already ran, so we leave the download slot untouched."""
|
| 3913 |
+
if IS_LOCAL:
|
| 3914 |
+
return gr.update()
|
| 3915 |
+
platform_key = _platform_key_from_profile(raw_profile_json)
|
| 3916 |
+
# Force the publish target to match the inferred platform so the plan and the
|
| 3917 |
+
# actual builder type can't disagree (e.g. mac profile but publish_windows).
|
| 3918 |
+
return export_platform_builder_zip(
|
| 3919 |
+
language,
|
| 3920 |
+
sample_name,
|
| 3921 |
+
raw_profile_json,
|
| 3922 |
+
package_goal,
|
| 3923 |
+
platform_key,
|
| 3924 |
+
f"publish_{platform_key}",
|
| 3925 |
+
target_mode,
|
| 3926 |
+
target_drive,
|
| 3927 |
+
decision_answers_json,
|
| 3928 |
+
operator_notes,
|
| 3929 |
+
)
|
| 3930 |
+
|
| 3931 |
+
|
| 3932 |
def build_ui() -> gr.Blocks:
|
| 3933 |
with gr.Blocks(title=get_txt("app_title", "EN")) as demo:
|
| 3934 |
# States
|
|
|
|
| 4594 |
run_physical_usb_build,
|
| 4595 |
inputs=[language, sample, profile_json, package_goal, target_drive, target_mode, decision_answers, decision_notes],
|
| 4596 |
outputs=[console_log, usb_ready, sidebar_status_html, btn_start_usb, btn_stop_usb]
|
| 4597 |
+
).then(
|
| 4598 |
+
# Hosted Space can't write a physical drive: offer the builder ZIP for
|
| 4599 |
+
# download instead (same as the Light one-click flow). No-op locally.
|
| 4600 |
+
export_builder_zip_for_hosted,
|
| 4601 |
+
inputs=[language, sample, profile_json, package_goal, simple_target, target_mode, target_drive, decision_answers, decision_notes],
|
| 4602 |
+
outputs=[simple_package_file]
|
| 4603 |
).then(
|
| 4604 |
update_chat_models,
|
| 4605 |
inputs=[language, sample, profile_json],
|
submission/FIELD_NOTES.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Field Notes — building JackAILocal for Build Small
|
| 2 |
+
|
| 3 |
+
*A short report on what we built, the small-model choices, and the design decisions that matter.*
|
| 4 |
+
|
| 5 |
+
## The bet
|
| 6 |
+
|
| 7 |
+
Most "local AI" projects are an API key behind a local-looking UI. The interesting, harder, and more useful thing is an AI that **keeps working when the network is gone** — and that you can hand to a non-technical person on a USB stick. So the whole project is organized around one rule:
|
| 8 |
+
|
| 9 |
+
> **No cloud inference fallback. Ever.** A capability is shown as *unavailable* when its local binary or model is missing — it is never silently replaced by a remote call.
|
| 10 |
+
|
| 11 |
+
That single constraint shaped every decision below.
|
| 12 |
+
|
| 13 |
+
## Two surfaces, one product
|
| 14 |
+
|
| 15 |
+
- **The shipped runtime** — a Rust daemon (`jackailocald`) + a hand-built WebUI. This is the offline AI: chat, voice, vision, documents. It runs from a folder, USB, SSD, desktop installer, or LAN appliance, binds to loopback by default, and never makes an outbound inference call.
|
| 16 |
+
- **The Hugging Face Space** — a Gradio **builder/console**. It configures a package for a given use case and hardware, runs the AI configuration agent, and exports the installable package. The Space is honest about its limits: a hosted Space cannot touch a visitor's drives, so local-runtime actions are reported as unavailable rather than faked.
|
| 17 |
+
|
| 18 |
+
Keeping these two cleanly separated is what lets us claim **Off the Grid** truthfully: the thing the judge runs in the cloud only *builds*; the thing in the demo video only *runs locally*.
|
| 19 |
+
|
| 20 |
+
## Small-model choices (everything ≤32B)
|
| 21 |
+
|
| 22 |
+
| Role | Model | Why |
|
| 23 |
+
|---|---|---|
|
| 24 |
+
| Default runtime chat | **Qwen3.5 4B** | Runs on an ordinary laptop with no 24GB GPU. This is the everyday experience → our **Tiny Titan** claim. |
|
| 25 |
+
| Power profile (24GB VRAM) | **Qwen3.6 27B** | Unlocked only when a real hardware profile reports ≥24GB VRAM. Stays under the 32B cap. |
|
| 26 |
+
| Configuration agent | **Gemma 4 12B IT** | Strong instruction-following + JSON adherence for the decision engine. Served on **Modal vLLM** at build time only. |
|
| 27 |
+
| STT / TTS | **whisper.cpp** / **Piper** | Mature, fully offline, CPU-friendly. |
|
| 28 |
+
|
| 29 |
+
A deliberate guardrail: the catalog **refuses** any model whose published parameter count exceeds the cap, and ambiguous models are excluded rather than risked.
|
| 30 |
+
|
| 31 |
+
A licensing note we had to get right: the original Piper `en_US-lessac` voice is **non-commercial** (Blizzard 2013), so the product ships `en_US-libritts_r-medium` (CC BY 4.0) instead. Small detail, real consequence.
|
| 32 |
+
|
| 33 |
+
## The agent as a decision engine, not a chatbot (Best Agent)
|
| 34 |
+
|
| 35 |
+
The configuration step does not chat. It is an **internal decision engine** that must return schema-validated JSON, choosing among `ASK_USER`, `ASK_HUMAN_REVIEW`, `REJECT`, or a concrete **build action**. The UI feeds it the use case, target, hardware profile, available backend/model plan, constraints, and prior answers.
|
| 36 |
+
|
| 37 |
+
Two things make it trustworthy:
|
| 38 |
+
|
| 39 |
+
1. **A deterministic policy gate keeps final authority.** It blocks USB builds with no real target, backend/model mismatches, missing secrets, and unsafe manifest patches — even when the model says otherwise. The agent can only edit the **packaging manifest**, never the shipped runtime's offline behavior.
|
| 40 |
+
2. **Full agent traces are exposed.** Every decision carries the exact prompt and the raw model output in the audit JSON.
|
| 41 |
+
|
| 42 |
+
## The hardening pass that mattered most: no simulated AI
|
| 43 |
+
|
| 44 |
+
An early audit caught the worst possible thing for a privacy product: a code path that **fabricated an "AI decision"** when no model was configured. We removed it. Now, with no agent endpoint configured, the panel reports the missing secrets explicitly. Provenance is labelled honestly throughout (`real_local_llm`, `real_remote_llm`, `not_configured`). For a tool whose entire promise is "you can trust where your data goes," a faked decision anywhere would poison the whole claim.
|
| 45 |
+
|
| 46 |
+
## What we'd do next
|
| 47 |
+
|
| 48 |
+
- Apply the ≤32B guard in the Rust `select_model`/`add_model` paths too, not just the catalog.
|
| 49 |
+
- Tighten `/v1/models` to be fully OpenAI-conformant.
|
| 50 |
+
- Finish the French-accent localization pass across the WebUI and Field Manual cards.
|
| 51 |
+
|
| 52 |
+
## Reproduce
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
pip install -r requirements.txt
|
| 56 |
+
python app.py # the builder, on http://127.0.0.1:7860
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
```powershell
|
| 60 |
+
cargo check
|
| 61 |
+
python -m py_compile app.py saas/gradio/app.py
|
| 62 |
+
node --check webui/app-v15.js
|
| 63 |
+
python qa/static_audit.py
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
*Small models. Big privacy. Off the grid.*
|
submission/thumbnail.svg
ADDED
|
|