Who is holding back China's ChatGPT?

ChatGPT has become an undoubted phenomenal product in the global information technology industry.

In the name of “general artificial intelligence”, it chats with human beings, helps people draft emails and lawyer letters, answers some esoteric ultimate philosophical questions, writes a usable Python code, answers some questions that seem to require complex and progressive logic, writes a movie script based on some character settings, writes a beautiful love poem, catches thesis assignments of college students… It seems that there is no such a versatile AI species in human history. Bill Gates said that the significance of the emergence of ChatGPT is “no less than the birth of the Internet”. Microsoft CEO Satya Nadella said it was comparable to the industrial revolution. Artificial intelligence oral enthusiasts once again exclaimed that the “singularity” is coming. Ordinary people are once again worried that their jobs will be replaced by all-round AI assistants like ChatGPT… From IBM’s “Deep Blue”, to Google’s AlphaGo, and then to OpenAI’s ChatGPT, 25 years have passed. Mental maturity, this is really something that makes AI happy.

I have used ChatGPT to do many indescribable things, and found that it is not always handy, but it can give better answers and solutions to some seemingly more difficult problems. For example, if you ask whether BYD can beat Tesla, it may give some clear and unremarkable statements with many factual fallacies and no personality; but if you ask it how autonomous driving will change the industrial design of a car, it can give a discussion full of imagination from the inside out in terms of chassis innovation, interior changes, digital entertainment, and appearance breakthroughs. On the whole, ChatGPT is quite imperfect, especially in terms of delivering convincing accuracy, but it often surprises humans in areas such as providing structured information discourse, opening imagination and liberating creativity. You can’t tell what specific use it has for you, but it can help you achieve and complete some trivial, redundant and even creative things.

It is such a seemingly useless but useful ChatGPT, which seems to be useful but useless, that has driven its parent company OpenAI to receive an additional investment of more than 10 billion US dollars from Microsoft. It took it two days to break through 1 million users, and Facebook took 305 days; The large-scale neural network containing 175 billion parameters-when it is preferentially licensed to Microsoft to integrate office software and search engine services such as Office and Bing, it really becomes a “usable” product.

But that’s enough to make China’s AI unicorns jealous.

With 500 employees, the company’s overall valuation is close to 30 billion US dollars. This is OpenAI; with thousands of people, the company’s valuation/market value is at best 1-2 billion US dollars. This is a number of AI “small giants” in China.

Because of the huge gap between human efficiency and value, and because of ChatGPT’s sudden influence on the real society of all human beings, the birth of ChatGPT has greatly stimulated the field of artificial intelligence in China. Many people jumped out again, feeling that the gap between China and the United States in artificial intelligence has further widened, and China has a long way to go to catch up with this wave. There are also some people who are keen to discuss why China does not have its own ChatGPT, and the conclusion is still that “China lacks the soil for innovation” and “Chinese Internet companies are engaged in live broadcasting and grocery shopping”, which are both irresponsible and ignorant of the facts.

Chinese Internet companies are not all engaged in live broadcasting and grocery shopping. They are engaged in semiconductor development, AI model research, and autonomous driving. American Internet companies are also very popular in live broadcasting, grocery shopping, and especially Internet finance.

As the Chinese Internet company with the most accumulation in the field of artificial intelligence and natural semantic processing, Baidu has been working on its own deep learning large model “Paddle Paddle” (Paddle Paddle) for the past five years, and even used its own general-purpose AI chip “Kunlun Core” to train its own model-they are the basic environment and premise for Baidu to train its own “ChatGPT”. Alibaba, ByteDance and Didi also have natural semantic training models based on their own needs. It can be said that in terms of training complex natural semantic models with tens of billions of parameters, Chinese companies and research institutions are not weak, and their starting point is not lower than that of their American counterparts—at least around 2016. In recent years, the gap between the Chinese and American artificial intelligence circles in the field of large-scale models is not a matter of awareness, starting point, and ability, but a matter of roads and methods.

The gap between China and the United States in the field of ChatGPT-like human-computer dialogue models is not caused by so-called regulation. If you have had frank exchanges with ChatGPT on some richer religious, cultural, ethnic, and geopolitical issues, you will realize that behind its seemingly refusal and prudent discussion of these issues, there are certain specific position tendencies that are subtly coincident with the mainstream values generally recognized in American society. It can be said that for any complex model of natural semantics, the process of model construction, corpus collection, training, and parameter adjustment is a process of “content review” based on a specific value system, and all have the awareness of maintaining their value system. It is not a question of whether we should “generate” China’s value position in the natural semantic model, but how it should be generated so as to truly check and balance the world view and cultural hegemony that English dominates the global Internet corpus, strengthen the weight of Chinese language understanding benchmarks in the global natural semantic processing system, and provide cultural diversity for the development of artificial intelligence and human-computer dialogue in the world.

I also seriously disagree with the statement that the quality of Chinese Internet information is too bad that the source of the Chinese ChatGPT model corpus is “polluted”. This is also a lazy and smart judgment. Because of the total amount of information on the Internet, English content is undoubtedly the most in the world, and the extreme content of worrying quality is also the most, all of which will affect the process and results of natural semantic model training. In the early training, ChatGPT gave priority to the highly praised content on the social forum Reddit with high content quality, which has a specific corpus selection tendency. If China prioritizes knowledge communities such as Zhihu and Dede, and mainstream media prioritizes the corpus of semantic models, there will be no problem of corpus contamination. Not to mention the foreign language proficiency and reading breadth of most people who hold that “the quality of Chinese content is low” is not enough to support their conclusions.

But in any case, the birth of ChatGPT is indeed a kind of stimulation and a conceptual challenge for me, who has been calling for “farewell to Silicon Valley worship” for many years.

This is not because I think the gap between China and the United States in the field of artificial intelligence has widened, but because a general artificial intelligence human-computer dialogue model such as ChatGPT is a tool that can truly promote social production collaboration and civilization from the perspective of all human beings—rather than a specific field or industry. Its significance is greater than the emergence of the mobile Internet, comparable to the birth of email and search engines. As an artificial intelligence powerhouse, China is no longer a country with a poor information technology industry when emails and search engines were born. However, we did not let this kind of general artificial intelligence innovation that can affect the progress of human civilization first occur in China, and train a model whose basic corpus is constructed from Chinese culture and value system.

What’s more, ChatGPT’s model training method largely relies on the parameter upgrade of “strengthening miracles”, repeated training, and continuous iterative optimization of the model based on the feedback of generated content-this was originally the working method that the Chinese team was best at. When an American start-up company uses the money raised from Microsoft to invest huge computing power costs at all costs, employs a large number of data workers in Africa and the Middle East for information labeling, and uses the most efficient iterations to conduct an “arms race” with giants like Google for self-developed semantic processing models, you still have a very unreal feeling-whether this is a San Francisco company or a Shenzhen company.

A natural semantic processing model like ChatGPT should have been born in China but it was not born in China. The reason has to start with China’s technology companies engaged in artificial intelligence—no matter what the giants or startups are doing these years.

A problem that many people may never realize is that a super-large-scale general-purpose natural semantic processing model like ChatGPT is most likely to produce miracles when built by an AI startup company, and better results are usually not achieved within a technology giant. This is why Google’s LaMDA dialogue application model and Bard, which has recently rushed into battle, have not shined, and it is also the challenge that Baidu will inevitably face next.

Why? The first is because general natural semantic processing modeling is too expensive. In fact, burning money is usually not the skill of big companies, but the privilege of startups. Tech giants are almost all listed companies. The investment of tens of billions of dollars is invested in something that will not see a return for a long time. The pressure on the chief financial officer in the face of the board of directors and shareholder meetings is very high, and they are often punished by the stock price. This makes large companies dare not take big risks. What is “Making Miracles Vigorously”? It is to spend a lot of money and make great efforts first, and then pray for miracles to happen, instead of acquiescing that a miracle must happen, and then decide to spend money and make great efforts.

Unfortunately, big companies can only be the latter. This is why even Microsoft, which has benefited a lot from ChatGPT, only dared to start from $1 billion at the beginning, which lasted four years, until this year’s $10 billion, and continued to increase investment one by one to support OpenAI in Microsoft’s “in vitro”, training the GPT model for many years. The equity acquired by Microsoft through investing in OpenAI enjoys the priority of integrating the ChatGPT model ability into its Office and search engines. It may not be easy to say whether it will eat OpenAI in the future, but at least Microsoft, which has a market value of nearly one trillion U.S. dollars and an annual income of tens of billions of U.S. dollars, absolutely does not dare to “make miracles vigorously” at the beginning and train this model on its own.

Second, because people are less tolerant of tech giants engaging in innovation, and more tolerant of mistakes and deviations in startups. In order to cope with the pressure of ChatGPT, Google hastily launched the human-computer dialogue test version Bard. It was found that some dialogues had basic factual errors, so it was infinitely magnified, and the market value evaporated hundreds of billions of dollars overnight. In fact, it’s not that Google doesn’t know this. If it wasn’t pushed into a hurry, it wouldn’t be so rash. The LaMDA model announced by Google in 2021 has significantly higher parameter levels and information search capabilities than the GPT-3 trained by OpenAI at that time, but Google has been reluctant to test its effect publicly because it is afraid that it will make mistakes, causing public distrust and stock price decline.

What Google cares about, OpenAI doesn’t care about. From the first day of ChatGPT’s release, it has publicly stated that it has no information retrieval capabilities, and its corpus is only up to December 2021. It cannot answer many questions about value and moral judgments, and often makes factual mistakes. The testers tolerantly accepted ChatGPT’s self-“bad performance”, and were amazed at its ability in information association, emotional expression, logical structure, and thinking coherence in the fields of programming, literary creation, formatted writing, and medical consultation, and lightly ignored the mistakes it made.

In March 2019, after the unprecedented success of the GPT-2 model, the four-year-old OpenAI decided to transform from a non-profit foundation to a commercial company. After all, no foundation can stand its chief scientist’s annual salary of $1.5 million. In May 2019, Sam Altman (Sam Altman) became the CEO of OpenAI. Then, OpenAI received a $1 billion investment from Microsoft. In May 2020, the GPT-3 model launched by OpenAI has parameters that have risen sharply from 1.5 billion in GPT-2 to 175 billion, forming an unprecedentedly powerful automatic learning system.

It can be seen that an artificial intelligence start-up company that was born with a golden spoon in its mouth, raised a huge amount of money, and is bundled with giant business, is engaged in the construction and development of general-purpose artificial intelligence natural semantic models, and invests in model training regardless of cost, which is the most ideal state. The imaginative and commercial returns that come with the most powerful models are enough to spur Microsoft and other investors.

So, why doesn’t this logic work in China? Did China ever have a powerful general-purpose natural semantic artificial intelligence model, even if it was just a prototype?

To answer this question, look at when Microsoft first invested in OpenAI: July 2019. Four months after Microsoft bet on OpenAI’s GPT model, that is, in November 2019, Shen Xiangyang, the global senior vice president of Microsoft who is in charge of the Bing search business and the top person in charge of Microsoft’s artificial intelligence, and a computer scientist from Hong Kong, China, announced that he had left Microsoft for more than 20 years. And Shen Xiangyang’s last contribution to Microsoft’s general artificial intelligence model is the chat robot Xiaobing developed by Microsoft Asia Internet Engineering Institute in 2014.

In July 2020, Xiaoice became independent from Microsoft and became a Chinese artificial intelligence start-up company. Shen Xiangyang served as the chairman, and Li Di, the former executive vice president of Microsoft Asia Internet Engineering Academy, served as the CEO. When Xiaoice became independent, it had developed to more than the sixth generation. Its product forms involved conversational artificial intelligence robots, intelligent voice assistants, content providers created by artificial intelligence, and a series of vertical field solutions. Xiaoice once sparked public discussion, in addition to being a chat robot full of emotions and feminine sexuality, it also has its amazing performance in the field of Chinese poetry creation-she published a collection of poems “Sunshine Lost the Glass Window”, which received a lot of praise and more controversies.

There is no doubt that a XiaoIce robot that can write poems and carry out simple emotional and commonsense-based conversations was the best-performing conversational general artificial intelligence model in the world a few years ago.

It is impossible for the team led by Shen Xiangyang to understand search, let alone artificial intelligence. And Shen Xiangyang’s departure from Microsoft and Xiaobing’s “independence”, coupled with Microsoft CEO Nadella’s investment and cooperation with OpenAI, is actually the top artificial intelligence trader in China and the United States. A formal parting of ways in the field of general artificial intelligence models.

So, does Xiaobing still write poetry today? what is it doing

In the past two years, Xiaobing has long stopped writing poems. It’s busy commercializing. It established a game studio to provide NPC scripted dialogue content for games; it cooperated with the Winter Olympics to provide a visual scoring system for freestyle skiing aerial skills; it provided Wind Information with artificial intelligence-generated text summaries of listed company announcements;

In a word, in the past, the artificial intelligence team that represented the higher level of the general natural semantic artificial intelligence model, and the Chinese supported the whole structure, has now become an artificial intelligence supplier that mixes generative artificial intelligence and decision-making artificial intelligence and provides specific solutions for specific scenarios.

You can’t say that this is Xiaoice’s “fallen”, after all, it has only raised hundreds of millions of yuan from the capital market. According to ChatGPT’s model training method, the money will be spent in one day. Without Microsoft’s protection, Xiaobing has to take care of his own life. However, I have never heard of Baidu, Tencent or ByteDance. I thought about investing in Xiaoice and supporting it to continue to develop a large-scale model of general natural semantic artificial intelligence.

Not just Xiaobing. In the past few years, there have also been other entrepreneurial teams in China engaged in automatic modeling of general artificial intelligence and heterogeneous computing, allowing 7-8 types of chips at home and abroad to be connected to software through this model. Chinese investment institutions have never shown interest in general artificial intelligence models, and even a little bit of imagination.

“More than 85% of the investors asked us to introduce the product scenario. We said that we help the GPU connect with the software ecosystem, and even Nvidia uses our model. Investors said that this is not a scenario. We said that we also have customers, research on satellites, docks, smart cities and smart industries. They said that your work is too scattered, so we don’t invest.” This is what I have heard from entrepreneurs who are doing general artificial intelligence models.

As we all know, VCs in China like to “educate” entrepreneurs the most, and of course educate scientists who are engaged in artificial intelligence entrepreneurship. “You have to have some data in this industry”, this is their favorite sentence to educate AI entrepreneurs.

There is data in a certain industry, and it is necessary to focus on providing solutions in a certain segmented field. This is the mindset of most VCs and PEs in China who claim to invest in artificial intelligence. Then we look at “how big is the scene”, the scene of the security camera is big enough, so the valuation model becomes the size of China, how many cameras can be installed? How much is each camera? How big is the total camera plate? Well, the plate is big enough, and we voted in the subdivision of the camera. Let’s look at port smart logistics again. How many ports are there in China? How many are deep water ports? How much can each port terminal pay for AI solutions? It turned out that we paid such a small amount of money. It seems that the scene of “port” is not big enough, so we will not vote. AI virtual digital human as customer service? It can be linked to the metaverse, it has a story and imagination, well, we can give it a try.

So, what you see is that China’s artificial intelligence “four tigers” are basically engaged in the business of cameras and face recognition, and they have all become AI project implementers and integrators. The business model is the same as that of Neusoft and iSoftStone 30 years ago.

For quite a long period of time, few investors in the field of artificial intelligence sincerely believed that a general model could be reused in various industries. Occasionally, there are a few who are a little patient and interested in the general model, and they are basically RMB funds. The US dollar fund is really not interested in the Chinese team’s attempt to develop a general model. Do you think that by comparing the difficulty and level of model training of companies like OpenAI and Google, they feel that there is a gap between the Chinese team in this matter? Then you really think too much. They know the time when the GPT model development is going on, that is, the last two months.

Those first-line investment managers who boasted that “SenseTime and Megvii are selling security cameras in my eyes”, those first-line investment partners who proudly told entrepreneurs that “your model is not a scene”, not to mention those US dollar investment fund partners who have hardly invested in artificial intelligence in history and have been tinkering with Chinese entrepreneurs to “go overseas” to engage in cryptocurrency for so many years, suddenly changed their appearance today and declared that they would support entrepreneurs in “China’s ChatGPT”. Then you can think about it, their vows and smugness contain some understanding and sincerity of the general artificial intelligence model, and some are speculation and calculation.

You can even think about it. The training of a super natural semantic model may cost tens of millions or even hundreds of millions of RMB a day, not to mention the computing power module that provides large model training - the world’s top GPU, because of the unreasonable embargo of the United States. It is becoming more and more difficult to obtain. With the attitude and behavior style of those investors in the past so many years, they can persist for a few days, how much money are they willing to persuade the investment committee to invest in, or can they help these entrepreneurial teams solve the GPU problem? Maybe one day, maybe half a year later, they will start urging these general model teams to “realize commercialization in subdivided fields” as soon as possible.

With Baidu’s insistence on investing in the PaddlePaddle model, it is inevitable that it will practice this model industry from the very beginning, and pursue commercialization in different industries as soon as possible. To a large extent, the training of large models of general artificial intelligence has an “impossible triangle” of massive data, high-quality and creative content output, and industrial application.

To achieve massive data and high-quality creative content output, it is bound to not be able to be quickly applied to the specific implementation of a certain industry - such as ChatGPT.

If you want to create specific industrial landing scenarios in the largest mass data created by humans on the Internet, you will definitely not be able to provide the highest quality results, because there must be conflicts between content generation based on mass data and precise decision-making systems-this is actually a waste.

If you want to achieve high-quality content output to assist accurate decision-making in industrial landing scenarios, you must sacrifice the largest amount of data. However, the data in most accurate industrial scenarios cannot support real large-scale model training and research. This is the dilemma faced by most of China’s “industry segmentation” artificial intelligence solutions today, and it is also the reason why the so-called “industry ChatGPT” is a false proposition.

Those entrepreneurs and investors who are gearing up to enter the “China’s ChatGPT” today, not to mention how much money and how many GPUs you have in your pockets, since you are all on this boat, you feel that you are holding a ticket. Which corner of the “impossible triangle” of general artificial intelligence will you decide to discard? This is a question that needs to be figured out first.

In other words, which investment institution—whether it is a financial investment institution or an investment department of a large company—has the determination to invest in training large-scale natural semantic models for several years and extend the return cycle infinitely? After all, history tells us that this is a group of people who are the least determined and most anxious to find a successor.

China has never lacked outstanding entrepreneurs and scientists, and the field of artificial intelligence is no exception. The level and accumulation of Chinese and American technology companies in the field of artificial intelligence are the closest in the world. At least a few years ago, there was not a big gap between China and the United States in the construction and training of large natural semantic models. However, China does lack some investment institutions and investors who have a broader vision, do not follow others’ opinions, and are determined and far-sighted.

People like Shen Xiangyang, Li Di, Ma Weiying, Wang Xiaochuan, and Li Zhifei, who came out to start large-scale generic natural semantic models are quite reliable, but the problem is that they need to change a group of investment institutions and investors who support them. Some investment institutions that are too good at “playing games” and speculating, and are too immersed in cryptocurrencies and other tracks, should be blacklisted.

To be honest, although no serious investment institutions have been looking at general artificial intelligence models for so many years, there are still some institutions that have invested in many artificial intelligence companies with extremely long payback cycles. For example, those VCs who have invested in China’s local lidar and autonomous driving solutions have contributed to the establishment of China’s brand-new competitiveness in the global auto industry’s unprecedented changes in a century. For example, those VCs who invested in China’s local GPUs—this is destined to be a track full of dangers, facing the ban and suppression of the United States, and the return cycle is extremely long; but these newly emerging local GPU players—whether it is Hanbo, Biren or others, they may provide ammunition for China’s general natural semantic processing model in the future. The investors behind them, if one day they really make up their minds and make a move to support China’s natural semantic large-scale model project, I may have some different expectations and confidence in them.

It’s just that there are not too many investors and investment institutions who don’t brag, don’t hold back, and don’t rush for quick success, but too few. However, China’s natural semantic model construction and training need such investors and investment institutions—whether it is a financial investor, a strategic investor, or a capital institution supported by the will of the state.

China needs to have its own general-purpose natural semantic large-scale model. It needs to have the vision of providing Chinese wisdom, Chinese value system, and Chinese solutions for global general-purpose artificial intelligence. It needs to avoid risks and legal, moral, and ethical issues in the whole process of corpus selection, model construction and training, and parameter adjustment. More importantly, it needs determination and patience.

In any case, it cannot be speculated.