A vast artificial intelligence system has been created by Microsoft and chip manufacturer Nvidia to mimic human language more convincingly than ever before.
Known as the Megatron-Turing Natural Language Generation (MT-NLG), the new neural network has 530 billion parameters that is more than tripling the scale of OpenAI’s groundbreaking GPT-3 neural network that was regarded as the state-of-the-art up until now. The new model is capable of exceptional accuracy, especially in terms of reading comprehension and complex sentence formation.
Specialized in natural language, the Megatron-Turing Natural Language Generation can understand texts, reason, and make deductions to form a complete and precise sentence. Microsoft and Nvidia created their own dataset of about 270 billion “tokens” to train MT-NLG from English-language websites. The websites included academic sources educational websites such as Wikipedia or Github, academic sources such as Arxiv and Pubmed, and news articles and even messages on social networks.
Microsoft and Nvidia remarked the use of MT-NLG “must ensure that proper measures are put in place to mitigate and minimise potential harm to users”. This issue is required to be tackled before the world can benefit from the revolutionary model.