Artificial neural networks can appear mysterious, but underneath the apparent intelligence is something much more concrete: numbers being transformed by mathematics. A network repeatedly multiplies values, adds them together, passes the results through functions, measures how wrong its predictions were and adjusts itself accordingly.
You do not need a degree in linear algebra or calculus to understand the basic mechanism. The important ideas are inputs, weights, biases, activation functions, predictions, errors and adjustments. Once those pieces are connected, the learning process becomes much easier to understand.
What a neural network actually does
At its simplest, a neural network converts one collection of numbers into another. Imagine a model designed to estimate the price of a house. We might give it information such as house size, number of bedrooms, distance from a city centre, age of the property and local market data.
Each of those values becomes an input. The network transforms those inputs through one or more layers and eventually produces an output, perhaps a predicted value such as £320,000.
A neural network does not understand a bedroom or a city centre in the human sense. It works with numerical representations and learns how strongly different patterns should influence its output.
Weights and biases: deciding what matters
Every input entering a neuron is associated with a weight. A weight represents how strongly that input should influence the neuron’s calculation.
Suppose a simplified neuron receives three inputs: house size, number of bedrooms and distance from the city centre. Each input has a corresponding weight.
Multiply each input by its weight, add the results together, then add the bias.
The same calculation can be written more compactly:
The symbol Σ means: add together all of the corresponding input × weight calculations.
What does the bias do?
The bias gives the neuron additional flexibility. In the simple expression below, the weight controls how strongly the input affects the output, while the bias shifts the result.
A useful analogy is that the weight controls sensitivity while the bias adjusts the starting position.
Vectors and matrices: doing many calculations at once
Real neural networks do not usually process one input and one neuron at a time. They may process hundreds of inputs, thousands of neurons and very large collections of parameters.
Linear algebra gives us a compact and efficient way to represent these calculations. Inputs can be stored in a vector, weights in a matrix and biases in another vector.
A one-dimensional collection of numerical values, such as x₁, x₂, x₃ and so on.
A structured table of values describing how inputs connect to neurons in the next layer.
Additional adjustable values that allow neurons to shift their response.
A compact way to perform many weighted calculations efficiently and in parallel.
This single expression can represent thousands of individual multiplication-and-addition operations. It is also one reason GPUs are so useful for machine learning: they are designed to perform large numbers of numerical operations in parallel.
Activation functions: introducing non-linearity
If every layer only performed a linear calculation such as z = Wx + b, even a deep network would ultimately behave like a much simpler linear model. Neural networks therefore use activation functions.
ReLU
One of the most widely used activation functions is the Rectified Linear Unit, or ReLU.
If z is negative, the output is 0. If z is positive, the output is z itself.
Sigmoid
The sigmoid function compresses a value into the range between 0 and 1, which can be useful when an output is interpreted as a probability in binary classification.
The output approaches 0 for strongly negative inputs and 1 for strongly positive inputs.
Tanh
The hyperbolic tangent function produces values between −1 and 1. Unlike sigmoid, its output is centred around zero.
Forward propagation: producing a prediction
Forward propagation is the process of sending information through the network from the input towards the output. At each layer, the network performs a weighted calculation, applies an activation function and passes the result to the next layer.
For a two-layer network, the sequence can be written:
The output of one layer becomes the input to the next.
Loss functions: measuring how wrong the model is
Forward propagation produces an answer, but that does not mean the network has learned. Learning requires a way to measure how far the prediction is from the correct answer. That is the role of the loss function.
Mean Squared Error
For regression problems, one common loss function is Mean Squared Error, or MSE.
For each example, the model calculates the difference between its prediction and the correct value, squares that difference, adds the squared errors and divides by the number of examples.
Binary cross-entropy
Classification problems often use cross-entropy-based losses. For a simple binary case, one common form is:
Training needs a numerical target to improve. The loss function turns “that prediction was wrong” into a number the optimisation process can work with.
Backpropagation: learning from mistakes
Once the network has measured its error, it needs to determine which parameters contributed to that error and how they should change. This is where backpropagation comes in.
Backpropagation works backwards through the computational graph and calculates how sensitive the loss is to each adjustable parameter. The mathematical mechanism uses derivatives and the chain rule.
Backpropagation tells the training process which numbers should change, and in which direction, if the model is to reduce its error.
Gradient descent: adjusting the weights
The result of backpropagation is a collection of gradients. A gradient tells us how the loss changes when a parameter changes.
A simplified weight update is:
The optimiser moves the weight in a direction intended to reduce the loss. The learning rate controls the size of the adjustment.
If the learning rate is too large, training may overshoot useful solutions or become unstable. If it is too small, learning can become unnecessarily slow.
The complete neural-network learning cycle
The major pieces can now be connected into one learning loop.
Training repeats this loop across batches of examples, often for many passes through the dataset.
What is an epoch?
One complete pass through the available training dataset is called an epoch. More epochs do not automatically mean a better model. If training continues inappropriately, the model can begin to overfit — performing very well on its training data while generalising poorly to new examples.
What is the network actually learning?
It is tempting to imagine that a neural network stores explicit human-readable rules. In most modern networks, that is not what happens. The model learns large collections of numerical parameters whose interactions collectively encode useful patterns.
That distinction matters because a model can produce an effective prediction without being able to provide a simple human explanation of exactly how every internal parameter contributed to it.
Knowing the mathematics of the learning mechanism does not automatically make every individual model decision easy to interpret.
This becomes particularly important when AI influences financial decisions, healthcare, public services, recruitment, fraud detection, compliance or risk assessment.
Common neural-network architectures
Different network architectures apply the same fundamental building blocks — numerical representations, learned parameters, transformations and optimisation — in different ways.
Basic input-to-output modelling used across classification and regression problems.
Designed to learn local and spatial features, particularly in image and computer-vision tasks.
Use recurrent connections so previous steps can influence later processing.
Gated recurrent architectures designed to preserve useful information across longer sequences.
Use attention mechanisms to model relationships across sequences and underpin many modern large language models.
Train a generator and discriminator in competition to produce synthetic examples.
Used for clustering and visualising relationships in high-dimensional datasets.
Use radial basis functions and have been applied to classification, approximation and control problems.
Transformers are still built from maths
Transformers changed the direction of modern artificial intelligence, particularly language modelling. They rely heavily on attention mechanisms that help a model represent relationships between different elements in a sequence.
Their internal architecture is more sophisticated than the simple neuron described earlier, but the foundations remain recognisable: vectors, matrices, learned weights, transformations, activation functions, loss functions, gradients and optimisation.
From model mathematics to service design and AI governance
Understanding the mathematics explains how a model transforms information. It does not, by itself, tell us whether the model should be used for a particular purpose or whether the surrounding service is safe, usable and accountable.
An operational AI-enabled service may contain data preparation, a model, business rules, a user interface, human decisions, operational actions, logs and monitoring.
Data, features, parameters, training, evaluation, performance and technical behaviour.
Users, journeys, operations, casework, hand-offs, failure routes and human intervention.
Purpose, risk, accountability, evidence, logs, oversight, monitoring and challenge.
That wider service raises questions the mathematics alone cannot answer:
- Is the input data appropriate for this use?
- Who can challenge the model’s output?
- When must a human intervene?
- What happens when the model is wrong?
- What evidence and logs are retained?
- Who owns the final decision?
- How is model performance monitored over time?
- What happens when the service, data or operating context changes?
A mathematically sophisticated model is not automatically a well-designed or well-governed service.
Final reflections: the maths is the mechanism
Artificial neural networks can produce remarkably sophisticated behaviour, but the underlying mechanism is mathematical. At the foundation are multiplication, addition, matrix operations, activation functions, error measurement and parameter adjustment — repeated at enormous scale.
Understanding those foundations helps remove some of the mystery surrounding artificial intelligence. It also reveals why model design, service design and AI governance increasingly need to be considered together.
The mathematics helps us understand how the machine learns. Service design explains how the machine fits into the organisation and user journey. AI governance establishes how the resulting system is controlled, monitored and held accountable.
What comes next?
The next question is no longer simply, “How does a neural network learn?” It is: How should people interact with increasingly capable AI systems?
For large language models, that leads naturally to prompt design and prompt engineering — how instructions, context, examples and constraints influence model output. In the next Insight, we will look at how to structure prompts more effectively, why small changes in wording can alter results, and where prompt engineering fits within the wider design and governance of AI-enabled services.