Mengchen originated from Aofei Temple.Quantum bit | WeChat official account QbitAI
There are two things recently, letsearch engineBack in the spotlight.
Baidu released "Wen Xin Bai Zhong", an industrial search system driven by AI big model technology. The labor cost of building an internal search engine is reduced by more than 90%, and only very low data is needed.
Almost at the same time,OpenAI recently released a chat bot.ChatGPTNetizen found that although it sometimes makes mistakes in finding answers to questions, it feels much more refreshing to get a complete answer directly than to choose from the search.
Search engine, a classic technology, is about to usher in a round of change?
Back then, the search engine was the traffic portal in the PC Internet era and the absolute king.
After entering the era of mobile internet, although it is no longer the most concerned focus, search is still the same.High frequency just neededGradually integrate into people’s daily lives.
When people look at the search engine again, they will notice that there is no stop setting in it these years.
Search has changed.
In the past, when it comes to search, it must be inseparable.keyword. In previous World Cups, people were more accustomed to searching for "World Cup Live Address", and keywords were separated by spaces.
It’s like treating all the web pages as a big document, using a way similar to CTRL+F.Match the keywords.
With the popularity of mobile phones and mobile Internet, people’s needs are also changing. Now they are more inclined to ask a question and expect search engines toGive a direct answer.
For example, search "Where can I watch the World Cup live broadcast?" It is more colloquial and complicated, and even sometimes it is voice conversion.
Search engines are also adapting to this change, and the results are no longer just page sorting.
In view of some problems, it will give a deep understanding of the content.Answer extraction.
Sometimes it is more intuitive and easier to follow.Video content.
Even on the basis of understanding the needs, there are matching ones.Service jump.
These changes don’t seem complicated, but there are many technologies behind them.
Here, I want to focus on two new technologies that Baidu unveiled this year."knowing one"and"Qianliu".
Let’s first look at the cross-modal big model, the representative of AI technology in the search scene.
To put it simply, Zhiyi model can continuously learn from various resources in the whole network, whether it is text, pictures, videos or structured information.
Breaking the boundaries of resource forms makes it easier to understand users’ search needs.
Technically speaking, Zhiyi used the technology of Baidu Wenxin Big Model. Large-scale pre-training technology improves the model performance, and the model miniaturization technology with distillation compression rate as high as 99% reduces the cost, which can be fully applied in search scenarios.
It is understood that at present, Zhiyi conducts trillions of inferences every day in various scenarios of Baidu search. Such a huge scale of use has brought new problems, how to present the results of meeting the needs to users efficiently.
This should be mentioned.A new generation of indexing technology, Qianliu, is responsible for organizing information of different dimensions intelligently and orderly.
Compared with the previous indexing technology, Qianliu focuses on multi-domain and multi-dimensional three-dimensional raster indexing.
How to understand rasterization?
In the past, in order to improve efficiency, search engines horizontally layered content according to quality. Search from high-quality content first, and you can return the results in time if you meet the requirements, and then go to the next level if you are not satisfied.
Nowadays, among thousands of streams, a batch of content with the highest quality is vertically layered by domain. Combining quality layering with content hit, the content is cut into grids horizontally and vertically for on-demand retrieval, which greatly reduces the calculation amount of each retrieval.
Baidu engineers revealed that the amount of calculation saved in this way is not idle, but the content is intensively cultivated and different algorithms are used to improve the quality of the index from multiple dimensions.
There is an added benefit that personalization algorithms can be applied between different grids. Just like "eat more than one fish", different parts use different cooking methods.
With the cooperation of Zhiyi and Qianliu, the whole system will make real-time dynamic adjustment according to the latest knowledge of the model to ensure the best retrieval effect. Avoid invalid calculation to the greatest extent, and finally present the results that meet the requirements to users efficiently.
Are these new technologies effective in practical business?
The answer may be found in the data.
Feedback-driven innovation
At the Vientiane Baidu Mobile Ecology Conference held in September, Baidu pointed out that in the past year, Baidu search scaleIncrease by 17% against the trend.
The latest Baidu’s third-quarter financial report also shows that the number of mobile search queries has achieved double-digit growth year-on-year.
He Junjie, senior vice president of Baidu Group and general manager of Baidu Mobile Ecology Group (MEG), pointed out that the key is"Feedback-driven innovation".
On the one hand, the feedback comes from intelligent search. Baidu search responds to billions of search requests from more than 100 countries every day.
On the other hand, it comes from smart recommendation. In the third quarter of 2022, the content distribution of Baidu App information stream increased by 23% year-on-year, and users’ likes, comments and sharing were also the most direct feedback from users.
The high-frequency demand of users drives the transformation of AI technology. New technology can stimulate the expression of new users’ needs, and the two constitute."two-wheel drive", continue to promote the evolution of search.
For example, with the help of virtual human technology, interactive dialogue can be realized, and there are chat and companionship needs in addition to simple search.
As predicted by Baidu Research Institute in the top ten science and technology trends at the beginning of the year, AIGC(AI Generated Content) shines brilliantly this year.
In the future, AIGC will continue to combine with search depth, which will also bring "search is generation" or even "search is creation".
There will be no more "Sorry, the relevant webpage was not found, please check whether your input is correct", but the content that does not exist will be created instantly by AI when the user clicks the search button.
Baidu CEO Li Yanhong also said some time ago:
With the breakthrough of technology, AI painting, AI video, and even AI building a virtual world may be as simple as taking photos with mobile phones.
And all technological breakthroughs and innovations are inseparable from talents.
In order to better promote technological and algorithmic innovation, promote exchanges between Industry-University-Research and cultivate talents, Baidu held the first search technology innovation challenge.
The contest provided a total prize pool of 300,000 yuan, provided NVIDIA A100 computing power resources, and even had access to desensitized data of Baidu’s massive search business.
This competition is divided into two tracks:
Search questions and answers,Designed to explore open domain search scenarios. Faced with the problems of uneven quality of web documents, different lengths, scattered distribution of answers to questions, long length and so on, it is hoped that participants can further improve the effect of in-depth intelligent question answering and provide users with a better search experience.
Reasoning optimization of search model,It is very important to ensure the smooth search experience of hundreds of millions of users and control the cost of computing power. It is hoped that the contestants will challenge the optimal model reasoning performance through various optimization techniques.
During the contest, Baidu also provided a wealth of related courses and learning materials, and experts in search technology and GPU accelerated computing will also provide full technical guidance.
This contest is open to the whole society. As soon as the news came out, it immediately attracted a large number of college students, corporate teams or individual developers. At present, 1,500+teams have signed up and become a grand event in the field of search technology.
Contest address:https://sti.baidu.com
Reporting/feedback
关于作者