AI Atlasby Verbito
Explore

LLM Inference & Serving

Grouped by technology, as developers usually describe it.

209 tools

  1. Run AI models like Llama, Qwen and DeepSeek on your own computer
    ★ 186kstars
    ollama by ollama · Free alternative to LM Studio, ChatGPT Plus
    No codingDesktop appHealthy
  2. Run AI chat models fast on an ordinary computer
    ★ 133kstars
    llama.cpp by ggml-org · Free alternative to LM Studio
    Some setupCommand-line toolHealthy
  3. Serve AI models fast to many users at once
    ★ 96.3kstars
    vllm by vllm-project
    Some setupSelf-hosted appHealthy
  4. Chat with AI privately on your own computer, no internet needed
    ★ 77.3kstars
    gpt4all by nomic-ai · Free alternative to ChatGPT, LM Studio
    No codingDesktop appFair
  5. Use 100+ AI providers through one standard connection
    ★ 62.7kstars
    litellm by BerriAI
    Some setupSelf-hosted appHealthy
  6. Manage and secure access to APIs and AI models
    ★ 44.3kstars
    kong by Kong
    Some setupSelf-hosted appHealthy
  7. Serve AI models fast on GPUs
    ★ 38.4kstars
    sglang by sgl-project
    Some setupSelf-hosted appHealthy
  8. Get an AI second brain for your notes and the web
    ★ 38kstars
    khoj by khoj-ai · Free alternative to Perplexity, Notion AI
    Some setupSelf-hosted appHealthy
  9. Manage a Linux server and run AI models from a web panel
    ★ 37.4kstars
    1Panel by 1Panel-dev
    Some setupSelf-hosted appHealthy
  10. Run AI models fast on Apple Silicon Macs from the menu bar
    ★ 23.3kstars
    omlx by jundot
    No codingDesktop appHealthy
  11. Run AI models on phones, laptops and in web browsers
    ★ 23.3kstars
    mlc-llm by mlc-ai
    For developersCode libraryHealthy
  12. Build a multi-platform AI chatbot (Chinese)
    ★ 19.3kstars
    kirara-ai by lss233
    Some setupSelf-hosted appHealthy
  13. Run machine learning efficiently on everyday hardware (for developers)
    ★ 15.6kstars
    ggml by ggml-org
    For developersCode libraryHealthy
  14. Run AI models fastest on NVIDIA GPUs
    ★ 15kstars
    TensorRT-LLM by NVIDIA
    For developersCode libraryHealthy
  15. Talk by voice to an AI with an animated avatar
    ★ 14.6kstars
    Open-LLM-VTuber by Open-LLM-VTuber
    No codingDesktop appFair
  16. Make AI models run fast on any hardware (for developers)
    ★ 13.9kstars
    tvm by apache
    For developersCode libraryHealthy
  17. Pretrain, fine-tune and deploy 20+ language models
    ★ 13.7kstars
    litgpt by Lightning-AI
    Some setupCommand-line toolHealthy
  18. Run AI models faster on NVIDIA graphics cards
    ★ 13.4kstars
    TensorRT by NVIDIA
    For developersCode libraryHealthy
  19. Run early LLaMA models on your computer (historic)
    ★ 12.9kstars
    dalai by cocktailpeanut
    Some setupCommand-line toolUse with care
  20. Keep your build and deploy pipelines reliable with AI
    ★ 12.7kstars
    opensre by Tracer-Cloud
    Some setupCommand-line toolHealthy
  21. Run open models as an OpenAI-compatible API
    ★ 12.6kstars
    OpenLLM by bentoml
    Some setupSelf-hosted appHealthy
  22. Serve AI models in production (for engineers)
    ★ 11.1kstars
    server by triton-inference-server
    Some setupSelf-hosted appHealthy
  23. Run AI models fast on Intel hardware
    ★ 11.1kstars
    openvino by openvinotoolkit
    For developersCode libraryHealthy
  24. Run Mistral's open models
    ★ 10.8kstars
    mistral-inference by mistralai
    For developersCode libraryUse with careArchived
  25. Use Ollama from Python code
    ★ 10.7kstars
    ollama-python by ollama
    For developersCode libraryHealthy
  26. See a giant AI model run on an ordinary CPU
    ★ 9.9kstars
    kimi-k3-in-c by FareedKhan-dev
    Some setupCommand-line toolFair
  27. Run large AI models faster on a consumer graphics card
    ★ 9.9kstars
    PowerInfer by Tiiny-AI
    For developersCode libraryFair
  28. Run deep research privately with local AI models
    ★ 9.5kstars
    local-deep-research by LearningCircuit
    Some setupSelf-hosted appHealthy
  29. Run your own AI search engine with a rich interface
    ★ 9.3kstars
    morphic by miurla
    Some setupSelf-hosted appHealthy
  30. Serve AI models as APIs
    ★ 8.9kstars
    BentoML by bentoml
    For developersCode libraryHealthy
  31. Run AI models locally on Qualcomm phones and laptops
    ★ 8.5kstars
    GenieX by qualcomm
    For developersCode libraryHealthy
  32. Serve AI models at datacenter scale
    ★ 8.4kstars
    dynamo by ai-dynamo
    Some setupSelf-hosted appFair
  33. Install and run AI apps with one click
    ★ 8.4kstars
    pinokio by pinokiocomputer · Free alternative to Paid AI app bundles
    No codingDesktop appHealthy
  34. Use your local AI models while browsing
    ★ 8.3kstars
    page-assist by n4ze3m
    No codingBrowser extensionHealthy
  35. Turn prompts into prototypes and slides
    ★ 8.2kstars
    open-codesign by OpenCoworkAI
    No codingDesktop appFair
  36. Compress and serve language models
    ★ 8.1kstars
    lmdeploy by InternLM
    For developersCode libraryHealthy
  37. Analyze stocks with AI, locally (Chinese)
    ★ 8kstars
    go-stock by ArvinLovegood
    No codingDesktop appHealthy
  38. Run AI models quickly on your own computer
    ★ 7.8kstars
    mistral.rs by EricLBuehler
    Some setupCommand-line toolHealthy
  39. Let AI evolve better code and algorithms
    ★ 7.7kstars
    openevolve by algorithmicsuperintelligence
    For developersCode libraryHealthy
  40. Automatically track what you work on
    ★ 7.4kstars
    Dayflow by JerryZLiu
    No codingDesktop appHealthy

Showing 40 of 209. Search or open the full list for more.