FOR IMMEDIATE RELEASE
Researchers describe a biologically inspired approach designed to help AI models retain previously learned knowledge while acquiring new information.
MOUNTAIN VIEW, Calif., June 10, 2024 — Google Research has announced a new method, referred to as the sleep framework, intended to address the problem of continual learning in large language models, according to Google Research. The approach draws on concepts associated with biological sleep processes to help models retain previously learned knowledge while being updated with new information, Google Research said.
Continual learning refers to the challenge of training a machine learning model on new data over time without degrading its performance on tasks it has already learned, a problem often described in the field as catastrophic forgetting, according to Google Research. The organization stated that the sleep framework represents a new way for large language models to manage this tradeoff, allowing systems to incorporate new knowledge while preserving existing capabilities.
According to Google Research, the framework is named for its conceptual link to the role sleep is believed to play in consolidating memory in biological organisms. Research in neuroscience has long suggested that sleep helps the brain stabilize and integrate newly acquired information with existing knowledge, and Google Research said the new framework draws on this idea as a model for how artificial systems might similarly consolidate learning over time.
Google Research described the work as part of its ongoing research into methods that allow large language models to be updated and improved after their initial training, rather than requiring complete retraining from scratch each time new data becomes available, according to the organization. Retraining large models from the beginning is computationally expensive and resource intensive, a constraint that has driven interest across the AI research community in methods that allow incremental updates, Google Research noted.
The announcement was reported by Google Research as part of its communications regarding ongoing work in machine learning and artificial intelligence research. The organization did not provide additional technical specifics, performance figures, or named individuals associated with the research in the material reviewed for this report.
Continual learning has been identified across the artificial intelligence research community as one of the central obstacles to deploying large language models in settings that require frequent updates, such as applications involving evolving factual information or changing user needs, according to general industry discussion of the topic. Methods that allow models to learn new information without forgetting prior knowledge are considered valuable for reducing the computational and financial cost of maintaining large-scale AI systems over time.
Google Research's announcement of the sleep framework reflects a broader trend in AI research toward approaches inspired by biological systems, including neuroscience-informed methods for memory consolidation, attention, and learning efficiency, according to the organization. The company has in the past published research exploring parallels between biological cognitive processes and computational learning systems, Google Research said.
About
This article was prepared using a release from Google Research, the research division of Google dedicated to advancing computer science and related fields through research publications, open-source tools, and collaboration with the broader scientific community. The communications office that publishes this site issues formal releases covering developments in artificial intelligence, machine learning, and related technology fields as part of its ongoing coverage of research news in the sector. The office compiles information from organizations such as Google Research to provide readers with timely, attributed reporting on developments in the field.
Media Contact
Communications Office This Publication [email protected]
Source: Google Research
