LLM AI Server with llama.cpp

Unlimited AI, zero cloud. Your phone is the server.

Run powerful LLMs locally, privately, and freely — with a lightweight UI anyone can use.

One Android device becomes your personal AI backend, accessible from PCs, tablets, and other devices on the same network.

---

1. Overview

LLM AI Server with llama.cpp is a fully local LLM server for Android, enabling private, offline text and multimodal generation.

Supported model families: Gemma-4 / Gemma-4 Vision, Qwen / Qwen Vision, Mistral, LLaMA, Phi, Bonsai, and others.

A built-in API server is compatible with Ollama, OpenAI, Anthropic (Claude) Messages, and Google Gemini APIs — providing chat, completion, generation, embedding, tokenize/token-count, and model-listing endpoints.

The bundled WebUI runs on the same port and supports model switching, parameter editing, log viewing, and PWA installation.

MCP and Function Calling enable tool-augmented workflows; structured output via GBNF and JSON Schema.

---

2. Intended Users and Supported Devices

Designed for users who want a private, fully local LLM environment:

- Offline-first users

- Developers integrating a local backend

- Advanced users tuning sampling and performance

- Researchers testing inference behavior

- Privacy-focused users

Performance tunable via context size, threads, batch size, GPU offload, KV cache quantization, and optional MTP decoding.

---

3. Key Features

- Hugging Face GGUF search & download; device-suitability guidance, one-tap mmproj download

- Qwen3-VL vision (bundled default); automatic n_ctx expansion for large images; GPU→CPU auto-fallback

- Local GGUF import; model rename/delete; LoRA adapter support (apply at runtime, manage from Settings)

- MTP speculative decoding; detailed parameters (Mirostat, DRY, XTC, Min-p, Typical-p, penalties, dynatemp)

- Upgraded llama.cpp engine (b10621, v0.3.0) — newer models, Adreno OpenCL GPU

- Ollama API: /api/chat, /api/generate, /api/ps, /api/show, /api/delete, /api/embed(dings), /api/tokenize, /api/tags; keep_alive (unload / preload / duration strings)

- OpenAI API: /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/models

- Anthropic (Claude) API: /v1/messages (streaming, thinking, image/audio, tool_use), /v1/messages/count_tokens, GET /v1/models; document/PDF content blocks; thinking via anthropic-beta header

- Google Gemini API: generateContent/streamGenerateContent; GET /v1beta/models (list/single); countTokens; embedContent/batchEmbedContents; JSON mode (responseMimeType); thinkingBudget

- Swagger UI at /swagger: interactive OpenAPI 3.0 docs, open from Settings → "Open API Docs"

- Connection check panel in Settings: probes all interfaces and shows reachable URLs (LAN, carrier, etc.)

- Bundled LLM AI Server Guide profile; on-device log retrieval via GET /api/diagnostics

- Multi-language WebUI (9 languages), PWA support; MCP and Function Calling

- Profile backup & restore; automatic prompt template selection

Full API docs, examples & interactive tester:

https://micklab.web.app/llama-tester-en.html

---

4. Getting Started

A) Start API/WebUI → Open in browser

1. Tap Start API/WebUI on the main screen.

2. Tap Open in browser to launch the bundled WebUI.

3. Chat and run the model directly from the WebUI.

- Note: On first use, if the model is not downloaded, the WebUI will download it when you send the first message.

B) Direct Run

1. Open the Direct Run section on the main screen.

2. Enter a prompt and tap Send to run the model directly inside the app.

C) or Use Settings to prepare a model first

1. Open Settings.

2. Search Hugging Face for a GGUF model, enter a direct URL, or import a local GGUF file.

3. Adjust parameters (context size, temperature, penalties, MTP, GPU offload, etc.).

4. Tap Save Config, then SAVE & CLOSE to load the model.

5. Optionally create a profile backup for later restoration.