Open-Source AI

A huge share of practical AI engineering runs on open tools and open models. The ecosystem gives you alternatives to closed APIs: models you can inspect, host, fine-tune, and run offline — and the libraries that glue everything together.

Open is a real option, not just ideology

Open models close in on closed frontier models for many tasks, and you can run them on your own hardware. That matters for data residency, cost at scale, offline use, and customization — concrete engineering reasons, covered through this section.

In this section

Open Models Open weights vs. open source, the major model families, licenses, and the open-vs-closed trade-off.

Frameworks & Libraries The ecosystem map — Hugging Face, orchestration frameworks, inference servers, and evaluation tools.

Running Models Locally Run models on your own machine — Ollama, llama.cpp, GGUF, hardware needs, and when local makes sense.

What you’ll be able to do

Navigate the open ecosystem, weigh open against closed models for a real project, and run a capable model on your own hardware.

Prerequisites

LLM Engineering; AI Infrastructure helps for the local-hosting material.