Chinese server maker Inspur on Tuesday released Yuan 1.0, one of the most advanced deep learning language models that can generate coherent Chinese texts.
The model is trained with 245.7 billion parameters—the number of weights in an artificial neural network, according to the company. This is more than the Elon Musk-backed GPT-3 language model for English, which has 175 billion parameters. Inspur said the Yuan model was trained with 5 terabytes of datasets.
The release of Yuan is a milestone for the Chinese natural language processing (NLP) industry. NLP is an important branch of artificial intelligence (AI) and a backbone for computers to understand human language.
Inspur said in a statement that its AI Research Institute had to develop “a unique development approach compared to English” to train the model, including addressing challenges such as “the lack of a prior high-quality Chinese-language corpus.” A corpus is a collection of texts used to train language models.
Inspur said the language model is “extremely adept” at natural language generation (NLG) tasks—the processing of generating natural language text using computers. The company said in a paper published in October that “only less than 50% of the time” human testers could distinguish between text generated by the model and human-written ones.
Inspur said in a press release that Yuan can be used to generate texts including news articles and poems.
In May 2021, a group of US researchers at Georgetown University found powerful NLG tools can be used to fuel disinformation. Inspur acknowledges such a possibility and vows to oversee the appropriate use of the model.
“Since the model can generate articles that are difficult to detect whether they are human written or not, the risk of misuse becomes higher,” an Inspur spokesperson told TechNode on Tuesday. “Therefore we need to regulate the application of the model in the future.”
The company is planning to open the model as an application programming interface (API) to developers, meaning that they can access Inspur’s platform and utilize the language model in their applications, the spokesperson said.
Jinan-based Inspur is the world’s third-largest maker of servers, according to market research firm IDC. The 72-year-old state-owned company used to be a manufacturer of electronic devices. It first entered the server market in 1995 as a partner of US chipmaker Intel.
In June 2020, the US Defence Department added Inspur, alongside 19 other Chinese companies, to a list that it deemed to be owned or controlled by the Chinese military, paving the way for further financial or export sanctions. In July 2020, Intel reportedly restored shipment to the company.