Dart Mobile AIFlutterOn-DeviceMultimodal

llamafu

Run AI models directly on mobile devices. No cloud. No latency. Complete privacy. A Flutter FFI plugin for on-device LLM inference.

View on GitHub

Key Features

📱

100% On-Device

No API keys, no network calls. Works completely offline with full privacy.

👁️

Vision & Multimodal

Support for LLaVA, Qwen2-VL and other vision models. Process images and audio locally.

🔧

Tool Calling

Function calling and structured JSON output with schema validation.

🔄

LoRA Hot-Swapping

Load and swap LoRA adapters at runtime for domain-specific fine-tuning.

⚡

Streaming Support

Real-time token streaming for responsive chat interfaces.

🌍

Cross-Platform

Android (API 21+) and iOS (12.0+) with optimized native code via FFI.

Quick Start

flutter pub add llamafu

import 'package:llamafu/llamafu.dart';

void main() async {
  final llamafu = await Llamafu.init(
    modelPath: '/path/to/model.gguf',
    threads: 4,
    contextSize: 2048,
  );

  final result = await llamafu.complete(
    prompt: 'Explain quantum computing in simple terms:',
    maxTokens: 256,
    temperature: 0.7,
  );

  print(result);
  llamafu.close();
}

Vision / Multimodal

final llamafu = await Llamafu.init(
  modelPath: '/path/to/llava-model.gguf',
  mmprojPath: '/path/to/mmproj.gguf',
);

final result = await llamafu.multimodalComplete(
  prompt: 'Describe this image:',
  mediaInputs: [
    MediaInput(type: MediaType.image, data: '/path/to/image.jpg'),
  ],
  maxTokens: 200,
);

Tool Calling

final result = await llamafu.chatComplete(
  messages: [
    ChatMessage.user('What is the weather in Tokyo?'),
  ],
  tools: [
    Tool(
      name: 'get_weather',
      description: 'Get weather for a location',
      parameters: {
        'type': 'object',
        'properties': {
          'location': {'type': 'string'},
        },
      },
    ),
  ],
);

Why On-Device?

Factor	Cloud API	llamafu (On-Device)
Privacy	Data sent to servers	Data never leaves device
Latency	200-2000ms round trip	Sub-100ms
Cost	Per-token billing	One-time compute
Offline	Requires internet	Works anywhere
Control	Vendor-dependent	You own the stack

Platform Requirements

Platform	Minimum Version
Flutter	3.10.0+
Dart SDK	3.1.0+
Android	API 21+ (Android 5.0), NDK 21+
iOS	12.0+, Xcode 14+

Models must be in GGUF format. Quantized versions (Q4_K_M, Q8_0) recommended for mobile.