Skip to content
explainroo
AI models explained

Open Source AI Models: What You Can Run and Build On

Updated 4 min read

With explainroo Text-to-Video Models vs. Code-Rendered Video for Explainers
On this page
  1. What “open” means for model use
  2. Which models you can run
  3. What local hardware requires
  4. How you can adapt a model
  5. When self-hosting makes sense

If you want to run an AI model on your own computer or adapt one for a product, downloadable models let you choose where data is stored and how the model responds. You also become responsible for hardware, licensing, and ongoing operation.

Many models described as “open source” are more precisely open-weight: their trained numerical parameters are available to download, while their training data and full development process remain private. You can run or customize those weights, but the model’s license determines what uses are allowed.

What “open” means for model use

A downloadable model gives you access to its weights, the values it uses to generate answers. It does not automatically give you access to the training data, training code, or permission to use the model in any way you choose.

Check the license attached to the specific model before building with it. Licenses such as Apache 2.0 and MIT generally allow commercial use under their terms. Some model providers use custom licenses with extra conditions. Their conditions may limit use by user count, location, or revenue. They may also restrict certain uses. The commercial-use license overview describes why these terms matter.

For practical purposes, treat the model card and license as part of the model you choose. Confirm that the license covers your intended use, including commercial deployment, before you invest in integration or fine-tuning.

Which models you can run

The right choice depends on the task, hardware, and license. A smaller model is easier to run locally. A larger model can perform better on some tasks, but needs more memory and serving capacity.

For example, the enterprise benchmark summarized by Enterprise Bot reported that Qwen 2.5 72B matched GPT-4o on that benchmark’s accuracy measure, scoring 86.6%. The same test gave Llama 3.2 90B a lower accuracy score of 83.3%, with a lower cost per completion. These results apply to one benchmark and do not guarantee similar results on other tasks. Model size and benchmark scores alone also do not establish whether a model will answer your specific questions reliably.

Before committing to a model, test it with representative prompts from your intended use. Check factual accuracy, output format, latency, and performance on difficult or ambiguous requests. Keep a human review step for decisions where an incorrect answer carries real consequences.

What local hardware requires

Memory is a major constraint when running a model on your own machine. A Llama 3 hardware guide provides memory estimates for different Llama 3.1 model sizes under the configurations it describes.

Those figures are not universal requirements for every setup. Quantization stores model weights with less numerical precision. This reduces memory needs, usually at some cost to output quality. The specific hardware also depends on how many people will use the model at once and how quickly it must respond.

For your first local trial, choose a smaller model that fits your hardware. Then test whether its answers meet your needs. Tools such as Ollama support local model use, while vLLM supports serving models for production traffic; model repositories such as Hugging Face Hub help developers find downloadable weights. The open-model tooling overview covers these options.

How you can adapt a model

Fine-tuning changes a model’s behavior by training it on examples or other data. It can help with a specialized format, domain vocabulary, or repeated task. It does not automatically make a model more accurate, and poor training data can teach it biased or incorrect patterns.

LoRA, short for low-rank adaptation, trains a small set of added parameters rather than changing all the model’s weights. The LoRA and QLoRA overview reports that LoRA can reduce trainable parameters by 99% or more. QLoRA combines that approach with a four-bit representation of the base model, which lowers GPU memory needs during fine-tuning.

Before training, define what success looks like and prepare examples that reflect the intended task. Compare the adapted model against the original using held-out examples, which are examples not used during training. If the base model already performs well, retrieval can be a simpler way to provide current or private reference material: the system finds relevant documents and supplies them alongside a user’s question.

When self-hosting makes sense

Self-hosting keeps prompts and model responses on infrastructure you control, which helps when sensitive data should not go to a third-party API. That benefit depends on how you secure the machines, logs, and service access. Local hosting alone does not guarantee privacy.

Self-hosting also brings ongoing work. Someone must run the serving software, monitor performance, apply updates, and maintain the hardware. Hosted APIs avoid much of that operational work, while charging based on provider pricing and usage. Compare the full cost of the options. Include hardware and electricity or hosting, along with engineering time and expected traffic.

Open-weight models have also narrowed some performance gaps with closed models, though results vary by model and benchmark. One Epoch AI analysis summary reported an average lag of four months and eight points on its Effective Capability Index measure. That finding reflects one measurement approach and does not rank every model or task.

Free and open source

Let your AI agent make the video

explainroo lets a coding agent like Claude Code or Codex make narrated explainer videos and product demos. The voice, the word timing and the rendering run on your own computer, with no API key and no cost per video. Optional outside services use your own key.

Copy this into your AI agent

Make me a short explainer video about [your topic]. Use explainroo for it: run npx --yes explainroo docs and follow what it prints.

Getting started Example videos GitHub

Made with explainroo: How explainroo makes a video

More on ai models explained

All guides on ai models explained