Evergreen Tools
|
← Back to AI Tools

Jan Offline Local AI Assistant

An open-source ChatGPT alternative that is designed to run 100% offline on your own computer: it uses llama.cpp for local inference, keeps your data on-device, and can also connect to cloud models, while exposing an OpenAI-compatible local endpoint other apps can reuse

Tool Interface

Interactive tool will be available soon

Features

  • ✓ Runs 100% offline: described as an open-source ChatGPT alternative, with chat and inference happening on your machine even without a connection
  • ✓ Powered by llama.cpp, so it can run quantized open models on ordinary machines without requiring a dedicated GPU
  • ✓ LocalDocs for asking questions about your own files, with the content staying on your computer
  • ✓ Custom assistants let you give each task its own system prompt for a specialised helper
  • ✓ Connect cloud models (such as GPT and Claude APIs) and switch between local and cloud in one interface, plus an OpenAI-compatible local endpoint for other tools

How to Use

  1. Download the installer for your platform from the official repo at https://github.com/janhq/jan or the site https://jan.ai (Windows, macOS, Linux)
  2. On first launch, pick and download a suitable open model from the model manager and wait for it to finish
  3. Start a new chat, or add files to LocalDocs first if you want answers grounded in your own documents
  4. To serve other apps, enable the local OpenAI-compatible endpoint and point third-party tools at that address

FAQ

What is Jan?

Jan is an open-source ChatGPT alternative; the official repository is https://github.com/janhq/jan and describes it as an open-source alternative to ChatGPT that runs 100% offline on your computer. It performs inference locally with llama.cpp, keeps chats and files on-device, and suits privacy-sensitive users or anyone wanting to avoid subscription fees.

Does it need an internet connection?

Per the official description, Jan can run 100% offline, so once a model is downloaded you can chat without a connection. It also supports connecting to cloud model APIs such as GPT and Claude, which naturally do need the internet, so connectivity depends on whether you use a local or a cloud model.

Will my computer handle it?

Built on llama.cpp, Jan can run quantized models on ordinary machines without a discrete GPU; the less VRAM or RAM you have, the smaller and more heavily quantized the model should be. Which model fits depends on your hardware and task, so start small and scale up once it runs smoothly.

Can I ask questions about company documents?

Jan offers LocalDocs: add your files and the assistant answers using your own material, with the project emphasising that your data does not leave your machine. For teams with confidentiality requirements this is a relatively safe route to internal knowledge Q&A.

Can other software use it as an API?

Yes. Jan exposes an OpenAI-compatible endpoint locally (commonly at http://localhost:1337 ). Point any tool that supports a custom OpenAI base URL at that address to run on your local models and avoid API costs.