Knowledge Engine

Do language models gain abilities suddenly as they grow?

·no read data

The idea

In 2022 sixteen authors led by Jason Wei published a paper in Transactions on Machine Learning Research titled “Emergent Abilitiesemergent ability A skill that seems absent in smaller AI models and then appears in larger ones, in a way that trends from the smaller models would not have predicted. of Large Language Modelslarge language model A computer program trained on very large amounts of text to predict and produce written language.”. Their definition was short: an ability is emergent if it is “not present in smaller models but is present in larger models.” The important part is that such an ability cannot be predicted by extrapolating the performance of smaller models.

The paper collected examples from published work. On three-digit addition and subtraction, GPT-3 scored near chance until roughly 13 billion parametersparameter One of the adjustable numbers inside an AI model that training tunes; model size is usually counted in them., then climbed. On a broad knowledge test, MMLU, models of about 10 billion parameters performed at guessing level, while models between 70 and 280 billion did substantially better. Prompting techniques also seemed to switch on: asking a model to reason step by step (chain-of-thoughtchain of thought Step-by-step reasoning that a language model writes out before giving its final answer. prompting) only beat ordinary prompting on maths word problems at about 100 billion parameters. If abilities appear abruptly, then building a bigger model could produce capabilities nobody anticipated.

In 2023 Rylan Schaeffer, Brando Miranda and Sanmi Koyejo at Stanford published a reply at NeurIPS, presented as an oral, titled “Are Emergent Abilities of Large Language Models a Mirage?” Their argument concerned measurement. Many tasks are scored by exact matchexact match A way of scoring an answer that gives full credit only if it is completely right and nothing otherwise.: the answer counts only if every tokentoken A small chunk of text, often a word or part of a word, that a language model reads or writes as one unit. is right. If a model’s chance of getting each digit right rises steadily with size, the chance of getting all five digits right will stay near zero for a long time and then shoot up. The jump is in the scoring rule, not necessarily in the model.

Why it might change a decision

The two positions make different predictions about the same outputs. If emergence is mainly a scoring artefact, then rescoring a model’s answers with a continuous measure, such as how many tokens differ from the correct answer or a Brier scoreBrier score A measure of how far a forecaster's stated probabilities were from what actually happened, where lower is better. on multiple-choice probabilities, should turn an abrupt curve into a smooth one, and smaller models’ progress should be visible in advance. If emergence is real, the jump should survive any reasonable metric.

How solid is this?

Both sides rest on reanalyses of existing models rather than controlled experiments, and neither is a single study in isolation.

Schaeffer and colleagues tested their prediction three ways. On GPT-3 family models doing arithmetic, accuracy showed apparent emergence for longer answers, while token edit distance on the same outputs improved smoothly; with more test questions, every model in the family scored above chance. In a meta-analysismeta-analysis A study that pools the results of many earlier studies on the same question to estimate an overall effect and what makes it vary. of the BIG-Bench benchmarkbenchmark A fixed set of tasks used to score and compare AI systems., they found two metrics, multiple choice grade and exact string match, accounted for more than 92 percent of claimed emergent abilities. They also induced apparent emergence in vision models simply by choosing a harsh metric. They stated that they were not claiming large models cannot have emergent abilities, and noted that several models with claimed emergence, including PaLM and Chinchilla, were not available for them to test.

Wei and colleagues had anticipated part of this. Their paper noted that exact-match scoring might disguise compounding incremental improvements, that a model’s cross-entropy loss improved even where accuracy sat at chance, and that for three generative tasks emergence appeared under several different metrics. They called metric choice an incomplete explanation.

Later work has not settled the matter. Zhengxiao Du and colleagues, at NeurIPS 2024, trained 31 models from scratch, from 300 million to 32 billion parameters, and plotted performance against pre-training losspre-training loss A number showing how badly a language model predicts the next piece of text in its training material, where lower means better prediction. rather than size. On MMLU, its Chinese counterpart C-Eval, and two maths word-problem sets, performance stayed near chance until loss fell below about 2.2, and this threshold held under continuous measures such as Brier score. Other tasks improved smoothly throughout. Their results come from one lab’s models and data, so the specific threshold may not transfer.

Sheng Lu and colleagues, at ACL 2024, ran more than 1,000 experiments and argued that apparent emergent abilities are better explained by in-context learningin-context learning A language model's ability to do a task from instructions or examples given in the prompt, without being retrained., model memory and linguistic knowledge combined, rather than by new abilities appearing.

The popular claim that models unpredictably acquire new skills outruns the data: many reported jumps shrink or vanish under a different metric. The opposite popular claim, that emergence has been shown to be a mirage, also outruns it, since some threshold-like behaviour survives continuous scoring. What counts as the right metric may depend on what users need, which the mirage authors themselves acknowledged.

Sources

  • Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., & Fedus, W. (2022). Emergent abilities of large language models. Transactions on Machine Learning Research. arXiv:2206.07682
  • Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems 36 (NeurIPS 2023). arXiv:2304.15004
  • Du, Z., Zeng, A., Dong, Y., & Tang, J. (2024). Understanding emergent abilities of language models from the loss perspective. Advances in Neural Information Processing Systems 37, 53138-53167. NeurIPS proceedings
  • Lu, S., Bigoulaeva, I., Sachdeva, R., Tayyar Madabushi, H., & Gurevych, I. (2024). Are emergent abilities in large language models just in-context learning? Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5098-5139. doi:10.18653/v1/2024.acl-long.279

Check yourself

Answers are checked in your browser. Nothing is recorded or sent anywhere.

  1. How did Wei and colleagues define an emergent ability?

  2. What is the core of Schaeffer, Miranda and Koyejo's argument?

  3. What did Du and colleagues (2024) report when they plotted performance against pre-training loss instead of model size?

  4. Which is the fairest reading of the evidence described in this post?

Questions without answers

  1. If an ability only matters to users when an answer is fully correct, such as a working line of code, is a smooth improvement in partial credit reassuring or beside the point?
  2. Emergence was plotted against parameters, training compute and pre-training loss in different papers. What would you need to know about a new model before deciding which of those axes is the fair one for a prediction?

Back to the archive