Baidu CTO Wang Haifeng: The Big Language Model Brings Dawn Memory to General Artificial Intelligence | Logic | Language
"The big language model has already possessed the core basic abilities of artificial intelligence such as understanding, generation, logic, memory, etc." Wang Haifeng, chief technology officer of Baidu, recently presented at the WAVE SUMMIT Deep Learning Developers Conference hosted by the National Engineering Research Center for Deep Learning Technology and Applications.
The academic community generally believes that deep learning has strong universality and industrial production characteristics of standardization, automation, and modularization, which will promote artificial intelligence to enter the stage of industrial production. Since 2019, the development of deep learning technology and applications has fully verified this viewpoint - the universality of deep learning technology is becoming stronger, and the standardized, automated, and modular features of deep learning platforms are becoming more and more prominent. The rise of pre trained large models has further expanded the depth and breadth of artificial intelligence applications. This means that artificial intelligence has entered the stage of industrial large-scale production.
Wang Haifeng stated that artificial intelligence has multiple typical abilities, among which understanding, generation, logic, and memory are the core foundational abilities. The stronger these four abilities, the closer they are to general artificial intelligence, and the larger language model possesses these four abilities.
Specifically, the typical abilities of artificial intelligence, such as creativity, programming, problem-solving, and planning, all rely on core foundational abilities such as understanding, generation, logic, and memory, with varying degrees of dependence. Taking problem-solving as an example, the comprehensive application of comprehension, memory, logic, and generative abilities is required from reading and answering the problem to finally writing the answer.
How to acquire these abilities? Wang Haifeng introduced that, taking ERNIE Bot as an example, first of all, we learned from trillions of data and hundreds of billions of knowledge to obtain a large model of pre training. On this basis, we used technologies such as intensive learning and prompting with supervision and fine tuning, and human feedback, and had technical advantages such as knowledge enhancement, retrieval enhancement, and conversational enhancement. Furthermore, through various strategies to optimize data sources and distribution, basic model long text modeling, multi type and multi-stage supervised fine-tuning, multi task adaptive supervised fine-tuning, multi-level and multi granularity reward models and other technological innovations, the basic universal capabilities are comprehensively improved.
Currently, artificial intelligence represented by big language models is penetrating into various industries, accelerating industrial upgrading and economic growth. "In this process, technological innovation and application have formed a virtuous cycle, with continuous improvement in understanding, generation, logic, memory and other abilities. The breadth and depth of industrial applications continue to expand, and the big language model brings dawn to general artificial intelligence," said Wang Haifeng.