Instructions to use DQN-Labs/dqnGPT-v0.1-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DQN-Labs/dqnGPT-v0.1-7B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="DQN-Labs/dqnGPT-v0.1-7B",
	filename="dqn-gpt-mistral-7b.q4_k_m.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use DQN-Labs/dqnGPT-v0.1-7B with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Use Docker

docker model run hf.co/DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

LM Studio
Jan
Ollama
How to use DQN-Labs/dqnGPT-v0.1-7B with Ollama:
```
ollama run hf.co/DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
```

Unsloth Studio

How to use DQN-Labs/dqnGPT-v0.1-7B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DQN-Labs/dqnGPT-v0.1-7B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for DQN-Labs/dqnGPT-v0.1-7B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for DQN-Labs/dqnGPT-v0.1-7B to start chatting

How to use DQN-Labs/dqnGPT-v0.1-7B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use DQN-Labs/dqnGPT-v0.1-7B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use DQN-Labs/dqnGPT-v0.1-7B with Docker Model Runner:
```
docker model run hf.co/DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M
```

Lemonade

How to use DQN-Labs/dqnGPT-v0.1-7B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull DQN-Labs/dqnGPT-v0.1-7B:Q4_K_M

Run and chat with the model

lemonade run user.dqnGPT-v0.1-7B-Q4_K_M

List all available models

lemonade list

🧠 DQN GPT v0.1 7B

High-Performance Local AI — No Datacenter Required.

DQN GPT 7B is a behavior-aligned, locally deployable assistant built on Mistral-7B-Instruct-v0.3, fine-tuned using an efficient QLoRA pipeline.

This release proves that powerful 7B models can be trained and deployed on modest hardware — sustainably, locally, and without enterprise infrastructure.

🚀 Mission

Make high-quality AI practical for everyday hardware.

DQN GPT is designed for:

Students
Indie developers
Researchers
Offline environments
LAN-hosted assistants
Personal AI servers

No subscriptions.
No cloud dependency.
No vendor lock-in.

🧠 Base Model

Architecture: Mistral 7B Instruct v0.3
Parameter Count: 7 Billion
Context Length: 32,768 tokens
Training Method: LoRA (low rank adaptation)
Training Dataset: Simple pipline test (no major fine tune yet)
Export Format: GGUF (llama.cpp / LM Studio compatible)

🔧 Fine-Tuning Approach

This model was fine-tuned using LoRA on consumer hardware (Apple M1).

Pipeline:

4-bit quantized base model for memory efficiency
LoRA adapters trained on behavioral alignment data
Adapter fusion into full-precision base
Exported to FP16
Quantized to Q4_K_M for local deployment

Focus Areas:

Identity consistency
Stable conversational tone
Reduced persona drift
Clean instruction-following
Practical, direct responses

This is not a benchmark-chasing release.
This is a usability-focused build.

💻 Hardware Requirements

Recommended (Q4_K_M)

8GB RAM minimum , 16G RAM comfortable.
CPU inference usable, even on low end devices.
GPU optional, but speeds up inference significantly.

FP16 (for research / re-quantization)

~16GB RAM minimum, 32GB recommended.
Intended for conversion, experimentation, or custom quantization.
Very hard on CPU, GPU highly recommended.

FP16 is not available for download from the HF repo at the moment. If you would like access to the FP16 model, please contact me on Discord at @dqnlabs.

📦 Intended Use Cases

Local AI assistant
Offline productivity tool
Personal coding helper
Educational AI system
Research experimentation
LAN-hosted AI server
Sustainable AI workflows

⚠ Limitations

Early-stage release
Behavior-focused fine-tune (not domain-specialized)
Not optimized for coding benchmarks
Not math-specialized
Not tool-trained

This is a foundation release.

🛣 Roadmap

Coding-specialized variant
Hallucination reduction dataset
Reasoning stability improvements
Public evaluation benchmarks
Larger context experimentation
Structured instruction tuning

🌱 Sustainability Philosophy

Large models do not need large infrastructure.

With modern techniques like QLoRA, 7B models can:

Be trained on modest hardware
Be deployed locally
Be distributed efficiently
Reduce centralized compute dependence

AI should scale down — not just up.

🔓 License

Apache 2.0

Use it.
Modify it.
Deploy it.
Build on top of it.

🧭 DQN Labs

This model is part of the broader DQN Labs initiative:

Practical AI. Local-first. Engineer-minded.

More releases coming soon.

Downloads last month: 2

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DQN-Labs/dqnGPT-v0.1-7B

Base model

mistralai/Mistral-7B-v0.3

Finetuned

mistralai/Mistral-7B-Instruct-v0.3

Quantized

DQN-Labs/mistral-7b-it-0.3-mlx-quantized

Quantized

(2)

this model

Collection including DQN-Labs/dqnGPT-v0.1-7B

dqnGPT v0.1

Collection

Your small-sized, but powerful family of AI assistants. No datacenter? No problem. Take your pick! (BETA, NOT FULLY FINE TUNED AS OF V0.1!) • 4 items • Updated Feb 23 • 2