You do not need to derive gradients or tune hyperparameters. You do need to know what you are buying when you say “we will use AI”—because the shape of the technology determines the shape of the product: what breaks, what costs money, what feels magical, and what will embarrass you in production.
This lesson is a map, not a math class. Every concept below should answer a product question: what to scope, what to promise, what to measure, and when to push back.
You are not building logic; you are building behavior that emerges from data
Traditional software is mostly deterministic. Given the same inputs and the same version of the code, you get the same outputs. That is why we can write tests that pass or fail cleanly.
Most modern AI systems—especially those built on machine learning and large language models—are probabilistic. They learn patterns from data and generalize to new situations. That generalization is powerful. It is also why the same prompt can produce different answers, why a classifier occasionally misfires, and why “it worked in the demo” is not a release criterion.
Product implication: Design for distribution, not for a single happy path. Assume variance. Build guardrails, fallbacks, and user-facing language that does not overclaim certainty. Your job is to turn statistical behavior into a reliable experience—which usually means combining models with product mechanics (constraints, UI, human review, and clear failure states).
Supervised learning is “teach by example”; unsupervised learning is “find structure”
Supervised learning means you have labeled examples: inputs paired with the correct output (or a score). Spam filters trained on “spam / not spam,” fraud models trained on “fraud / legitimate,” and many ranking systems are supervised. The model learns a mapping from features to labels.
Unsupervised learning means you have data without those explicit labels. The system looks for clusters, anomalies, or representations that compress information. Think customer segmentation without predefined segments, or anomaly detection when “normal” is easier to characterize than every possible failure mode.
Product implication: Supervised problems are easier to reason about in roadmaps because success ties to labels you can argue about. Unsupervised outputs are often exploratory—great for internal analytics or suggestions that must be framed as “pointers,” not verdicts. If your team cannot describe what a “good” output is, you are not ready to promise a supervised solution at scale.
Classification sorts the world; generation creates new artifacts
Classification assigns inputs to categories (or probabilities across categories). Is this ticket urgent? Does this document violate policy? Which intent does this user message map to?
Generation produces new content: text, images, code suggestions, summaries that paraphrase, and so on. Large language models (LLMs) are primarily generative—they predict likely next tokens in a sequence, which is why they can draft, rewrite, and role-play.
Product implication: Classification features fail in bounded ways (wrong bucket). Generative features fail in open-ended ways (plausible nonsense, tone drift, policy issues). Your UX and evaluation plan must match the failure mode. A wrong classification might need one tap to fix; a wrong draft might need a full editor and provenance cues.
LLMs are next-token predictors—not databases, not oracles
An LLM is a model trained to predict what comes next in text, given prior context, at massive scale. With enough data and compute, that objective produces surprisingly general capabilities: reasoning-like behavior, tool use when trained for it, multilingual fluency.
What LLMs are not: guaranteed fact stores, legal authorities, or substitutes for your domain-specific validation layer. They can sound confident while wrong.
Product implication: Treat LLM outputs as drafts unless you have independent verification (retrieval from trusted sources, calculators, APIs, human review). If your product promise requires factual precision, architecture matters as much as model choice.
Training teaches the model; inference runs it for users
Training is the expensive offline phase where the model learns from data. Inference is the online phase where the model produces outputs for real requests.
Training can take days or weeks and burn serious GPU budget. Inference shows up in your unit economics on every user action: latency per request, tokens processed, and peak-load behavior.
Product implication: Roadmaps must separate “model work” from “serving work.” A better model you cannot afford to run at your traffic and latency bar is not shippable. PMs who only optimize offline accuracy often ship something users never feel because it is too slow or too costly at scale.
Models, training data, and fine-tuning are levers—each with tradeoffs
A model is the learned artifact plus the architecture that defines its capacity. Bigger models can be more capable and more expensive. Smaller models can be faster and cheaper but brittle outside their training distribution.
Training data is the bedrock. It defines what the system “knows” in practice. Gaps in data become gaps in behavior. Biased or unrepresentative data becomes biased outputs—often in subtle ways.
Fine-tuning adapts a base model to your domain, tone, or task with additional training (smaller than training from scratch). It can improve consistency on your vocabulary and workflows. It does not magically fix missing ground truth or eliminate hallucinations.
Product implication: Ask where your differentiation lives. If it is generic language help, an API might suffice. If it is deep domain behavior, you likely need proprietary data, evaluation harnesses, and possibly fine-tuning—or a hybrid with retrieval.
Embeddings turn meaning into geometry your systems can search
Embeddings are numeric vectors that represent text (or images, or other inputs) in a space where “closeness” approximates semantic similarity. They power semantic search, clustering, recommendations, and retrieval-augmented generation (RAG).
Product implication: Embeddings are how many teams make LLMs “know” private or fresh information without retraining the whole model. Your PM questions: Is our corpus clean enough to index? How do we handle permissions so users only retrieve what they should see? How stale can embeddings be before the product lies?
Tokens are the meter for cost, latency, and context limits
Models process text in tokens—chunks that do not always align with words. Longer prompts and outputs mean more tokens: higher cost, higher latency, and sooner you hit context window limits (how much text the model can attend to at once).
Product implication: Product design choices—how much history you pass in, whether you summarize first, how you chunk documents—directly hit margin and speed. Token discipline is not engineering pedantry; it is P&L and UX.
Accuracy is a headline; precision and recall are the real negotiation
Accuracy (often quoted) can mislead when classes are imbalanced. If 99% of events are benign, a model that always says “benign” is “accurate” and useless.
Precision asks: when the model predicts positive, how often is it right? High precision reduces false alarms—critical when positives trigger expensive actions (auto-charge, account lock, escalations).
Recall asks: of all actual positives, how many did we catch? High recall reduces misses—critical when the cost of missing a case is severe (safety, fraud, abuse).
Product implication: Pick the error you can live with in your domain, then set thresholds and UX accordingly. Sometimes you want a conservative system with human triage. Sometimes you want aggressive automation with easy undo. The metric story should match the user story.
Offline wins do not guarantee online wins—and PMs should know why
Teams iterate fast using offline evaluation: held-out datasets, benchmarks, and simulated scoring. That is necessary. It is also incomplete.
Production traffic differs. Users behave differently than your eval set. Incentives change. Adversaries show up. Latency constraints force shorter prompts or smaller models, which can quietly erode quality.
Product implication: Plan for a bridging phase: shadow mode (run the model without affecting users), canary releases, and A/B tests tied to behavioral outcomes. If leadership wants to ship on offline accuracy alone, you are accepting unknown product risk.
Retrieval augments models; it does not replace accountability
Retrieval-augmented generation (RAG) combines search over trusted documents with an LLM so answers can cite fresher, domain-specific material. It is a common pattern for support, internal knowledge, and compliance-heavy domains.
RAG reduces some hallucination risk. It introduces new product risks: bad chunks, permission leaks, stale documents, and “confident synthesis” that still misreads sources.
Product implication: Ask how citations work in the UX, how document freshness is governed, and what happens when retrieval returns nothing useful. Users blame the product, not the chunking algorithm.
”Temperature” is a product lever—because it trades consistency for variety
Many generative systems expose sampling controls (often summarized as temperature). Higher temperature increases randomness: more creative variation, more inconsistency. Lower temperature is more deterministic: steadier outputs, sometimes more repetitive or brittle phrasing.
Product implication: Decide what your product promises. A brainstorming assistant can tolerate variety. A workflow that generates customer-facing legal language probably cannot. Default settings should reflect your risk posture, not the engineer’s personal preference.
Probabilistic systems change what “done” means
In classical software, done is often “meets spec and passes tests.” In AI products, done is usually “meets risk and performance bars under real traffic with a plan for drift.”
Models degrade when the world changes. Data shifts. Adversaries probe edges. User behavior evolves.
Product implication: Budget for monitoring, retraining triggers, and explicit acceptance of residual error. Your roadmap should include evaluation, not only features. If leadership wants a launch date without a definition of “good enough,” you are set up for a credibility crisis the first time the model misfires in public.
You can lead without doing the math
Your job is to translate business and user constraints into requirements your ML partners can execute: what to optimize, what to protect, what failure looks like, and what tradeoffs are acceptable. The vocabulary in this lesson is enough to ask sharp questions and spot sloppy thinking—on your team and in vendor pitches.
Next in this track: how to decide whether AI is the right tool at all—and how to keep discovery honest when the hype is loud.