The Papers

I. What happened?

In June 2017, a team of researchers mostly at Google published “Attention Is All You Need”, proposing an architecture they called the Transformer.¹ It modelled sequences, such as sentences to be translated, using attention alone, without the recurrent or convolutional layers that most models of the time relied on.

II. What is actually known?

The paper reported strong results on standard machine-translation benchmarks while needing less training time than the leading recurrent models it compared against.¹ Within about a year the architecture was adapted beyond translation, for example in BERT, which pre-trained a Transformer on unlabelled text for language understanding.²

III. Why does it matter?

Because the Transformer processes a whole sequence at once rather than one word after another, it trains well on parallel hardware. That property, more than any single benchmark, is why it became the base of today's large language models.

IV. What is the underlying mechanism?

Each word is turned into three vectors: a query, a key and a value. A word's query is compared with every other word's key; the scores, scaled and normalised, decide how much of each word's value flows into the new representation of that word. Several such “heads” run side by side, each free to attend to different relations. Since nothing in this is ordered, position is added to each word as a signal of its own.¹

V. What remains uncertain?

Attention compares every word with every other, so its cost grows quickly with the length of the text. Whether attention weights explain what a model is “thinking” is debated. And whether ever-larger Transformers keep improving at the same pace is an open question, not a law.

VI. What could happen next?

Possibilities, not predictions: cheaper forms of attention for long texts; other architectures that challenge it on efficiency; and, beyond the architecture, the question of who can afford the compute to train such models at all.

I. What happened?

In 2020, researchers described retrieval-augmented generation: a language model paired with a searchable collection of documents, so that what it writes can draw on passages fetched at the moment of asking.¹ The pattern has since become a common way to build question-answering systems.

II. What is actually known?

The original work combined a pre-trained generator with a dense retriever over a large text collection, and reported that this helped on knowledge-heavy tasks compared with the generator alone.¹

III. Why does it matter?

A model's training is frozen at a point in time and cannot cite where an answer came from. Retrieval lets a system use documents it was never trained on, keep them up to date, and point to the passage it relied on, so a reader can check it.

IV. What is the underlying mechanism?

Documents are split into passages and each is turned into a vector, a list of numbers that places similar meanings near one another. A question is turned into a vector the same way; the nearest passages are fetched from a vector index and placed in the model's context with the question; the model then writes its answer from what it was given.

V. What remains uncertain?

The answer is only as good as what is fetched: a stale or poorly split collection gives confident answers from the wrong page. A model can still ignore or misread the passages it is handed. And measuring whether an answer is truly supported by its sources is itself unsettled.

VI. What could happen next?

Possibilities: retrieval that checks its own results, systems that refuse when nothing relevant is found, and standard ways to show readers exactly which passage supports which sentence.

I. What happened?

Robert Baden-Powell, a British Army officer, published Scouting for Boys in 1908, and the Scouting movement grew from it.¹ Its motto, “Be Prepared”, travelled with it.

II. What is actually known?

The ideas Scouting popularised are preparedness, learning by doing, responsibility and service. So are the controversies historians have documented around its founder: colonialism, militarism and racial attitudes among them. Both belong in the record.

III. Why does it matter?

Faced with fast-changing technology, people tend either to fear it or to follow it. Preparedness offers a third way: learn it, practise it, and keep your judgement.

IV. What is the underlying mechanism?

Preparedness works through habit: small, repeated practice that makes a skill available under pressure. It fails through obedience, when discipline replaces thought. The useful lesson keeps the first and refuses the second.

V. What remains uncertain?

How to honour a useful idea without honouring everything about the person who popularised it is a question each reader answers. This paper's answer is to name both.

VI. What could happen next?

Our proposal: be prepared, but do not fear change. Learn by doing, but evaluate the evidence. Develop leadership, but question authority. Learn from history without becoming prisoners of it.

I. What is the question?

Today much AI runs in large data centres and reaches people over the network. A recurring idea is that more of it will run “at the edge”: on phones, sensors, vehicles and machines, close to where the data is made. This report asks how that could unfold, without pretending to know.

II. What is actually known?

Running a model on a device avoids a round trip to a server, can keep data on the device, and keeps working when the network does not. It also meets hard limits of memory, power and heat that a data centre does not.

III. Three scenarios

The Split

Small models on devices handle the routine and the private; large models in the cloud handle the rare and the hard. Most systems quietly become hybrids.

The Swarm

Many devices share what they learn without sharing raw data, and intelligence becomes a property of networks of machines rather than of any single model.

The Centre Holds

Cloud models keep improving faster than devices can follow; the edge stays a thin client, and capability concentrates further in a few data centres.

IV. What would tell them apart?

Signposts to watch: whether capable models keep getting small enough for ordinary devices; whether privacy rules favour data that never leaves the device; whether device makers expose AI hardware to outside developers; and who controls updates once models live on millions of machines.

V. What remains uncertain?

Nearly everything about timing. These scenarios can also mix, differing by industry and by country. Any one of them may arrive in part.

VI. What could happen next?

For readers, the useful preparation is the same in all three: know where your data goes, who can update the model you rely on, and what still works when the network does not.