Machine learning

Machine learning is the branch of artificial intelligence in which a computer is given examples and a measure of success, and adjusts its own behaviour until it performs well. The programmer supplies the data and the objective; the algorithm supplies the rule. That inversion, from writing instructions to fitting models, is why machine learning displaced hand-coded logic in so many domains.

The phrase covers methods from a linear regression fitted to a few hundred rows to a language model trained on much of the public web. What they share is a procedure that improves performance on a task by exposure to data, and assumptions that determine how well the result generalises beyond the examples it saw.

The field is also where most practical and political questions about AI now arise: what counts as good performance, who bears the cost of error, and how a system that was fitted rather than specified can be audited.

A learning algorithm is given a dataset and a hypothesis space, the family of models it may consider. Training selects the member of that family that best fits the data under a loss function, a numerical measure of how wrong a prediction is. For most modern methods the selection is iterative: gradient descent adjusts parameters in small steps in the direction that most reduces the loss 1.

The choice of hypothesis space encodes assumptions. A linear model assumes the relationship is linear; a decision tree assumes the problem can be partitioned by threshold tests; a neural network assumes very little and therefore needs a great deal of data. Much of the craft lies in matching the model to the amount and structure of the evidence 2.

Two further choices shape the result. Regularisation penalises complexity so that the model prefers simpler explanations of the data, and the loss function decides what counts as an error. A loss that weights all mistakes equally will treat a rare but costly failure the same as a common and trivial one, which is rarely what a deployment needs 1. A fraud model that misses one large transfer and flags a hundred small ones may score well on average while failing at the job it was built for.

Supervised learning trains on labelled examples, learning a mapping from inputs to known outputs. Classification assigns a category; regression predicts a quantity. It is the most widely used paradigm and the basis of most commercial deployment, because labelled data can be bought, curated, and checked 1.

Unsupervised learning works without labels, looking for structure: clusters, low-dimensional representations, or anomalies. It is used for customer segmentation, compression, and the pretraining of large models, where the task is constructed from the data itself — predicting a masked word or the next token turns unlabelled text into a supervised problem.

Reinforcement learning trains an agent that acts in an environment and learns from rewards. It produced notable results in games, robot control, and chip design, and it underpins the fine-tuning stage of modern language models 3. Its weakness is cost: it needs many interactions, and a poorly specified reward invites exploitation, since the agent will find whatever behaviour maximises the number rather than whatever behaviour was intended.

The boundaries have blurred. Self-supervised pretraining followed by supervised fine-tuning is now the standard recipe for language and vision systems, and it borrows from all three traditions.

The central question is not whether a model fits its training data but whether it works on data it has never seen. Practitioners hold out a test set, and usually a validation set for tuning, so that reported performance reflects unseen examples. A model that performs well on training data and badly on test data has overfitted: it has memorised noise rather than learned structure. The trade-off between fitting the training data and generalising is the oldest idea in the field, formalised as the bias-variance decomposition 1.

Evaluation is harder than the split suggests. Leakage — information from the test set reaching training, through duplicated records, temporal ordering, or feature construction — inflates results, and it is common. Benchmarks saturate, and a single accuracy figure hides differences across subpopulations. A model can be right on average and wrong for the group that bears the consequences 4.

NIST treats validity and reliability as properties to be measured and documented rather than assumed, and recommends evaluation against the conditions of intended use rather than against a leaderboard 4. In practice that means test sets drawn from the deployment population, stratified results, and periodic revalidation as the population changes.

Machine learning grew out of two traditions that did not always speak to each other. The first was cybernetics and early neural modelling, which produced the perceptron in 1958 and, after Minsky and Papert's critique of single-layer networks in 1969, a long period in which neural methods were unfashionable. The second was statistics, which supplied regression, discriminant analysis, and the theory of estimation.

The statistical tradition came to dominate in the 1990s. Support vector machines, regularisation, and ensemble methods such as boosting and random forests gave strong results on modest datasets with clear theoretical guarantees. Leo Breiman's 2001 essay described the resulting split between a data-modelling culture, which assumed a stochastic model generated the data, and an algorithmic culture, which treated prediction as the goal and the model as a means 5.

Deep learning reopened the neural programme once computation and data caught up. A convolutional network's win in the 2012 ImageNet competition demonstrated that learned features could beat hand-engineered ones at scale 2, and the transformer of 2017 supplied an architecture that trained efficiently on parallel hardware. The result was a shift in emphasis from model design to data curation and compute allocation.

Deep learning refers to machine learning with neural networks of many layers. Its advantage is that the model learns its own features, replacing the hand-engineered descriptors that earlier systems depended on 2. The same architecture then applied to images, audio, text, and protein sequences.

Scale became a method in its own right. The GPT-3 paper reported that a sufficiently large language model could perform new tasks from instructions and a few examples in the prompt, without retraining 6. Later systems pushed parameters, data, and computation further, with real but uneven returns: strong gains on some tasks, stubborn failures on others.

Scale is not free. Training consumes large amounts of electricity and specialised hardware, and the resulting models are expensive to serve. A growing share of practical work therefore goes into making models smaller, faster, and cheaper — distillation, quantisation, pruning, and better data selection — rather than into making them larger.

The result is a division of labour that did not exist in earlier machine learning. A few organisations pretrain large general models on unlabelled data, and many more adapt those models to specific tasks with small labelled datasets, retrieval, or prompting. That adaptation is itself machine learning, but it depends on artefacts, evaluation sets, and interfaces that the adapter did not build and cannot fully inspect 6.

The part of machine learning that is least visible in papers is the part that decides whether a system works. Datasets must be collected, cleaned, labelled, deduplicated, and versioned; labels must be checked for consistency; and the pipeline that produces features must be kept in step with the pipeline that serves predictions. A 2015 paper by Sculley and colleagues called the result hidden technical debt, noting that the model is a small fraction of a production system and that data dependencies are expensive to maintain 7.

Costs compound. Training a frontier model requires thousands of accelerators for weeks, and serving it requires inference capacity that scales with users. The AI Index reports that the price of inference at a fixed capability level fell by more than 280-fold between 2022 and 2024, which is why features that were uneconomic became routine 8. Efficiency, not raw capability, often determines what gets deployed.

The environmental footprint is real but difficult to compare across providers, because published figures depend on hardware, cooling, utilisation, and the local energy mix. Vendor claims about efficiency are claims; independent audits are rare 8.

Applications include fraud detection, medical imaging, credit scoring, speech recognition, translation, recommendation, predictive maintenance, and scientific discovery. In several, machine learning is genuinely better than the rules it replaced, because the relationship between inputs and outcomes is stable, the data is abundant, and errors are cheap or reviewed by a person 8.

The failures cluster where those conditions break. A credit model trained on historical lending learns historical exclusion; a medical model trained in one hospital may learn the scanner rather than the pathology; a recommender optimised for engagement can learn to promote material that users later regret consuming. These are not exotic bugs but ordinary consequences of fitting to available data and a proxy objective 4.

Scientific use is among the most successful. Models that predict protein structure, screen candidate molecules, or classify astronomical images compress years of manual work into hours, and they are evaluated against experimental ground truth rather than against a benchmark alone.

Recommendation and ranking deserve separate mention, because they are the largest commercial application and the least examined. A ranking model decides what millions of people see, is trained on behaviour that its own output helped produce, and is rewarded for a proxy — clicks, watch time, purchases — that correlates with but is not identical to the welfare of the person scrolling. The resulting feedback loop is a machine-learning problem with a social outcome 4.

Machine-learning systems are only as good as the data and the objective they are given. If the training data under-represents a group, performance will be worse for that group. If the objective is a proxy for what is wanted, the model will optimise the proxy, sometimes in ways that defeat the purpose. These are the ordinary consequence of specifying a task numerically 4.

Distribution shift is the other structural weakness. A model trained on one population, season, or sensor degrades when those conditions change, and it does not announce the degradation. Systems that retrain on their own outputs can drift further, and feedback loops can amplify whatever bias the loop rewards. Deployment therefore requires monitoring, periodic revalidation, and human review of consequential decisions 4.

Accountability is the unresolved part. A model's parameters are not an explanation, and post-hoc explanation methods approximate rather than reveal the model's reasoning. Regulators have responded by requiring documentation, risk assessment, and human oversight for high-stakes uses, shifting the burden from proving that a system is fair to showing that its risks were assessed and managed 4.

Two critiques recur. The first is methodological: much published machine-learning research optimises benchmark scores rather than deployment outcomes, and results are often hard to reproduce because code, data, and tuning details are incomplete. The field has responded with reproducibility checklists and benchmark audits, but the incentive to report the best number has not disappeared 5.

The second is conceptual. Breiman's two-cultures argument predicted a trade-off between predictive accuracy and interpretability, and the deep-learning era resolved it decisively in favour of accuracy for many tasks. Critics argue that this has produced systems whose behaviour cannot be explained to the people affected by them, and that the resulting opacity is a governance problem rather than a technical inconvenience 5.

A third disagreement concerns economics. One account treats machine learning as a general-purpose technology that will diffuse through the economy like electricity, with large productivity gains arriving slowly. Another treats it as a collection of narrow prediction tools whose gains are real but concentrated, and whose costs — data collection, energy, and displaced labour — fall on different people than the benefits 8.

The disagreements are not purely academic. They determine how much regulation is proportionate, how much investment is justified, and how much of the current capability should be described as intelligence rather than as pattern completion. A fourth disagreement concerns who can participate. Training frontier models now requires capital and data at a scale available to a few dozen organisations, while the academic groups that built the field work with smaller models and borrowed infrastructure. Whether that concentration is a temporary feature of the technology or a permanent shift in how research is done is disputed, and the answer shapes which kinds of critique remain possible 8.