Wu Dao 2.0 in 2024: China's Improved Version of GPT-3

In June 2021, the Beijing Academy of Artificial Intelligence (BAAI) launched Wu Dao 2.0, a successor to Wu Dao 1.0, as China’s first super-scale intelligent model system. Wu Dao is a language model that aims to surpass OpenAI’s GPT-3 and Google’s LaMDA in its human-level thinking. Trained on 4.9 terabytes of images and texts, and surpassing the state of the art (SOTA) levels on 9 benchmarks, Wu Dao is closer than any of its peers to reaching artificial general intelligence (AGI) and achieving human-level thinking.

Training

Wu Dao was trained on 4.9 terabytes of high quality images and texts in both English and Chinese:

• 1.2TB Chinese text data in Wu Dao Corpora.
• 2.5TB Chinese graphic data.
• 1.2TB English text data in the Pile dataset.

It was trained with FastMoE, an open-source Mixture of Experts (MoE) system. MoE is a machine learning technique that works by:

• dividing predictive modeling tasks into sub-tasks
• training an expert (learner) model on each sub-task
• developing a gating model that learns which expert to consult based on the input to be predicted, and combines the predictions.

FastMoE enables Wu Dao to consult different expert models in parallel and switch to the one with the best-predicted outcome. For example, if the input is an English text, Wu Dao will use a predictive model that can generate a response in English text.

Capabilities

Wu Dao 2.0 is an intelligent multi-modal AI model, which means that it is not only a language model that generates text and speech, but can also generate images, and has self-improving learning capabilities.

In Beijing Zhiyuan Conference where Wu Dao first debuted, creators displayed Chinese poems and drawings generated by Wu Dao.

Following that event, a virtual student based on Wu Dao’s AI model, Zhibing Hua, announced on her Weibo that she intends to start her education at the Department of Computer Science and Technology at Tsinghua University. The virtual student is powered by Wu Dao, therefore, she can use her knowledge base and learning capabilities to write poems, draw, and compose music. Creators claim that Zhibing Hua will have faster and more efficient learning capabilities than a normal student, however, she will not experience emotions for the time being.

Intelligence benchmarks

As reported by BAAI, Wu Dao 0.2 surpassed state-of-the-art (SOTA) levels on 9 benchmark tasks:

1. ImageNet zero-shot: SOTA surpasses OpenAI CLIP
2. LAMA knowledge detection: surpasses AutoPrompt
3. LAMABADA Cloze: Ability surpasses Microsoft Turing NLG
4. SuperGLUE few-shot FewGLUE: surpasses GPT-3 and obtains the current best few-shot learning results
5. UC Merced Land-Use zero-shot: SOTA surpasses OpenAI CLIP
6. MS COCO text generation diagram: surpasses OpenAI DALL·E
7. MS COCO English graphic retrieval: surpasses OpenAI CLIP and Google ALIGN
8. MS COCO multilingual graphic retrieval: surpasses the current best multilingual and multimodal pre-training model UC2, M3P
9. Multi 30K multilingual graphic retrieval: surpasses the current best multilingual and multimodal pre-training model UC2, M3P

Wu Dao vs. GPT-3

Here’s a comprehensive comparison between Wu Dao and GPT-3:

What is the future of human-level thinking AI?

Artificial general intelligence (AGI), or singularity, is AI’s capability of human-level thinking. ~90% of AI experts expect AI to reach singularity by 2075, however, some find it out of reach as modeling the human brain is impossible. Learn more about artificial general intelligence from our in-depth article Will AI reach singularity by 2060? 995 experts’ opinions on AGI.

If you want to explore AI trends and applications, feel free to read:

• Top 5 Expectations Regarding the Future of NLP
• 141 Myth-Busting Statistics on Artificial Intelligence (AI)
• 100+ AI Use Cases & Applications: In-Depth Guide

Contact AIMultiple to find the right AI solution and vendor for your organization.