AI Atlasby Verbito
Explore

LLMOps, Evaluation & Observability

Grouped by technology, as developers usually describe it.

149 tools

  1. Monitor your servers and apps with AI-powered alerts
    ★ 81.3kstars
    netdata by netdata
    Some setupSelf-hosted appHealthy
  2. Schedule and monitor data and AI pipelines
    ★ 47.4kstars
    airflow by apache
    Some setupSelf-hosted appHealthy
  3. Audit AI agents to check they can be trusted
    ★ 43.2kstars
    iFixAi by ifixai-ai
    Some setupCommand-line toolHealthy
  4. See and improve what your AI app is doing
    ★ 37kstars
    langfuse by langfuse · Free alternative to LangSmith
    Some setupSelf-hosted appHealthy
  5. Control access to servers securely, with AI assistance
    ★ 31.9kstars
    jumpserver by jumpserver
    Some setupSelf-hosted appHealthy
  6. Find passwords and API keys accidentally left in code
    ★ 30.7kstars
    gitleaks by gitleaks
    Some setupCommand-line toolHealthy
  7. Track, evaluate and manage AI models and agents
    ★ 28.7kstars
    mlflow by mlflow
    Some setupSelf-hosted appHealthy
  8. Label images, text and audio to train AI
    ★ 28.6kstars
    label-studio by HumanSignal · Free alternative to Scale AI labelling
    Some setupSelf-hosted appHealthy
  9. Test your prompts and find security flaws in AI apps
    ★ 26.8kstars
    promptfoo by promptfoo · Free alternative to PromptLayer, LangSmith
    Some setupCommand-line toolHealthy
  10. Debug and evaluate your AI app
    ★ 23.2kstars
    opik by comet-ml · Free alternative to Weights & Biases Weave
    Some setupSelf-hosted appHealthy
  11. Turn data and AI code into web apps
    ★ 19.5kstars
    taipy by Avaiga
    For developersCode libraryHealthy
  12. Test the quality of your AI app's answers
    ★ 19.3kstars
    deepeval by confident-ai · Free alternative to LangSmith evaluations
    For developersCode libraryHealthy
  13. Measure how well your AI app answers
    ★ 16.2kstars
    ragas by vibrantlabsai
    For developersCode libraryFair
  14. Monitor and evaluate AI agents
    ★ 16.2kstars
    RagaAI-Catalyst by raga-ai-hub
    For developersCode libraryFair
  15. Remove Copilot and Recall from Windows 11
    ★ 13.6kstars
    RemoveWindowsAI by zoicware
    Some setupCommand-line toolHealthy
  16. Route requests to 1,600+ AI models with safety checks
    ★ 13.3kstars
    gateway by Portkey-AI
    Some setupSelf-hosted appHealthy
  17. Trace and evaluate your AI app
    ★ 12.1kstars
    phoenix by Arize-ai · Free alternative to Arize, LangSmith
    Some setupSelf-hosted appHealthy
  18. Run, observe and improve your AI app in one platform
    ★ 11.7kstars
    tensorzero by tensorzero
    Some setupSelf-hosted appUse with careArchived
  19. Find and hide personal data in text and images
    ★ 11.7kstars
    presidio by data-privacy-stack
    For developersCode libraryHealthy
  20. Track experiments and models while training AI
    ★ 11.3kstars
    wandb by wandb
    For developersCode libraryHealthy
  21. Find and fix security flaws with OpenAI's Codex Security
    ★ 11.3kstars
    codex-security by openai
    Some setupCommand-line toolHealthy
  22. Organize data science projects for production
    ★ 11kstars
    kedro by kedro-org
    For developersCode libraryHealthy
  23. Learn to build and deploy ML on Amazon SageMaker
    ★ 11kstars
    amazon-sagemaker-examples by aws
    No codingGuide or readingHealthy
  24. Scan AI chatbots for security weaknesses
    ★ 9.8kstars
    garak by NVIDIA · Free alternative to Lakera Red
    Some setupCommand-line toolHealthy
  25. Write reliable AI functions with typed outputs
    ★ 9.5kstars
    baml by BoundaryML
    For developersCode libraryHealthy
  26. Build AI apps in Rust
    ★ 9.1kstars
    rig by 0xPlaygrounds
    For developersCode libraryHealthy
  27. Find and explain Kubernetes problems with AI
    ★ 8.3kstars
    k8sgpt by k8sgpt-ai
    Some setupCommand-line toolHealthy
  28. Monitor and test AI models
    ★ 8kstars
    evidently by evidentlyai
    For developersCode libraryHealthy
  29. Visualize model training progress
    ★ 8kstars
    tensorboardX by lanpa
    For developersCode libraryFair
  30. Monitor apps and find root causes with AI
    ★ 8kstars
    coroot by coroot
    Some setupSelf-hosted appHealthy
  31. Orchestrate ML pipelines and agents at scale
    ★ 7.8kstars
    flyte by flyteorg
    Some setupSelf-hosted appHealthy
  32. Monitor and enforce rules on AI agents
    ★ 7.8kstars
    failproofai by FailproofAI
    Some setupSelf-hosted appFair
  33. Route AI requests through your own gateway
    ★ 7.6kstars
    llm-gateway by mnfst
    Some setupSelf-hosted appHealthy
  34. Manage Kubernetes by asking in plain English
    ★ 7.6kstars
    kubectl-ai by GoogleCloudPlatform
    Some setupCommand-line toolFair
  35. Check and fix what AI models say before users see it
    ★ 7.6kstars
    guardrails by guardrails-ai · Free alternative to Lakera Guard
    For developersCode libraryHealthy
  36. Benchmark language models on many tests
    ★ 7.6kstars
    opencompass by open-compass
    For developersCode libraryHealthy
  37. Monitor your AI app with OpenTelemetry
    ★ 7.5kstars
    openllmetry by traceloop
    For developersCode libraryHealthy
  38. Monitor systems in real time with AI help
    ★ 7.5kstars
    hertzbeat by apache
    Some setupSelf-hosted appHealthy
  39. Add safety rules to AI chatbots
    ★ 7.4kstars
    Guardrails by NVIDIA-NeMo · Free alternative to Lakera Guard
    For developersCode libraryFair
  40. Red-team and secure AI agents and infrastructure
    ★ 7.4kstars
    AI-Infra-Guard by Tencent
    Some setupSelf-hosted appHealthy

Showing 40 of 149. Search or open the full list for more.