tech

In May, it became a month of AI "explosion".

On the early morning of May 8th, Apple released the strongest tablet on the planet, the iPad Pro with the M4 chip. The M4 chip brought ultimate AI performance, in the official words: "Stronger than any neural engine of any AI PC today!"

Six days after Apple's release, Open AI announced the arrival of the latest flagship large model, GPT-4o. Not only is it free to use, but it also spans listening, watching, and speaking.

Following Open AI, Google couldn't sit still. The day after, at the I/O 2024 developer conference, Google announced the update of the Gemini series of large models. Gemini 1.5 Pro is not only open to everyone, but the context window also takes a big step from 1 million tokens directly to 2 million, and can read 1500 pages of PDF at one breath.

Six days after the battle between Open AI and Google, Microsoft quietly threw a huge stone into the lake of AI again, proposing the concept of "Copilot+PCs".

After this series of operations, netizens started to get excited: "Things are getting interesting."

01

Microsoft fires at Apple

The emergence of the "Copilot+PCs" concept directly overturned the table of AI PCs.

Why do you say that?Compared to the previously vague concept of "AI PC" frequently demonstrated by the original equipment manufacturers, Microsoft has provided a clearer positioning. At the system level of Windows, it has integrated the newly released GPT-4o; at the hardware level, the built-in Snapdragon X Elite chip can process generative AI applications locally without relying on cloud computing power.

Advertisement

Microsoft's clear positioning has punctured the ambiguous concepts that many manufacturers are still hiding, bringing two interrogative features: What kind of hardware does an AI PC really need? And what kind of intelligence does it need?

At the press conference, Microsoft repeatedly targeted Apple.

For example, to demonstrate the performance of the first Surface Pro, Microsoft compared it with Apple's MacBook Air. According to Microsoft's official test, the multi-thread performance of the new Surface Pro is 58% higher than that of Apple's MacBook Air.

Microsoft also emphasized that to be called Copilot+PC, at least 40 TOPs of performance is required. The M4 chip released by Apple this month has an NPU computing power of exactly 38 TOPs.

In order to process generative AI Copilot locally, Microsoft not only requires an NPU but also at least 256GB SSD and 16GB RAM. This requirement is twice that of Apple's MacBook Air.

It can be said that the conditions set by Microsoft directly hit Apple's soft spot.

Since AI PC gradually became a hot term in the field of personal computers this year, Apple has been emphasizing that Mac is the best AI PC that consumers can buy. According to Apple, since the launch of the first M-series chip M1 in 2020, the M1 with NPU was born for AI.

Apple CEO Tim Cook mentioned earlier this year, "There is no better computer for artificial intelligence on the market than Mac."

When it comes to the biggest drawback of Mac, it must be the "ancestral" 8GB of memory. It has been 7 years since Cook last upgraded the memory, and in these 7 years, the Mac series has never improved the starting configuration of memory.Apple claims that the 8GB Unified Memory used by Mac is equivalent to the 16GB Memory used by competitors. The official response is that 8GB of memory is sufficient for many tasks such as browsing the internet, playing videos, and light editing.

However, the question remains: Can a starting memory of 8GB support Mac to become the best AI PC?

It is well known that running large AI models requires a large amount of video memory. A research team from the University of California, Berkeley, found that in the future, the memory wall may become a bigger bottleneck than computing power. The video memory capacity of the GPU severely restricts the scale of the trainable model and the speed of computing power enhancement, which may become an important bottleneck hindering the development and implementation of AI technology.

If the video memory of the GPU is only at the 8GB level, no matter how developers optimize, it is impossible to accommodate a large model with a hundred billion parameters.

If you want to configure a computer that can play AI smoothly, most people's opinions would be to start with 32GB of memory. In fact, at the 2024 China Flash Memory Market Summit, Intel even more aggressively stated that the entry-level standard for future AI PCs must be 32G memory, and the current 16G memory will definitely be eliminated.

Therefore, it is not surprising that Microsoft has raised the threshold configuration for future AI PCs to 16GB.

02

How much computing power is enough for AI PC chips?

In fact, although there is still debate on the software and system level for AI PCs, on the hardware level, manufacturers have reached a consensus: it is necessary to equip with heterogeneous computing platforms that include NPU, CPU, and GPU.

There are currently four manufacturers that have launched AI PC chips: Intel, AMD, Qualcomm, and Apple. However, Apple is self-produced and sold, so we will not discuss it much. Intel maintains a leading position in the progress of AI PC chips, AMD's related products have a faster iteration speed, and Qualcomm's AI PC chips have better edge AI inference capabilities than Intel and AMD. It is expected to gradually capture more market share in the future.The AI PCs that have been announced in the market are mostly equipped with Intel's Meteor Lake, such as Lenovo Xiaoxin Pro 16 2024 Core Ultra Edition, Lenovo's high-end YOGA series, Microsoft's two PC Surface Pro 10 Commercial Edition and Surface Laptop 6 Commercial Edition, ASUS Lingyao 14 2024 AI Ultra-Thin Notebook, etc.

This is the Meteor Lake released by Intel at the end of 2023, which uses the Intel 4 (7nm) process technology for the computing module, up to 6 performance cores, 8 energy-efficient cores, and 2 low-power island energy-efficient cores, totaling 22 threads supported.

Currently, Meteor Lake AI PCs have won the support of more than 100 ISV independent software vendors, with AI acceleration features exceeding 300 items, and AI large model acceleration optimization exceeding 500 items.

AMD also provides chips for AI PCs. AMD launched the Ryzen 7040 last year, also equipped with an NPU. Subsequently, AMD released the Ryzen 8040 processor. The Ryzen 8040 series, with the development code name Hawk Point, is still based on the Zen4 CPU architecture, RDNA3 GPU architecture, and XDNA NPU architecture. At present, AI PCs equipped with AMD Ryzen 8040 processors include ASUS ROG Huan 14 Air Notebook, ASUS Tianxuan, ASUS a Dou, ASUS Fearless, etc.

In addition, AMD's related products have a faster update and iteration speed than competitors. For example, on April 16, Eastern Time, AMD announced the launch of two new products, the Ryzen (Ryzen) Pro 8040 series and the Ryzen Pro 8000 series. Manufacturers such as HP and Lenovo will launch AI PCs equipped with Ryzen Pro 8040 series chips before the end of this year.

As mentioned earlier, Microsoft believes that a true AI PC needs to have a computing power of 40 TOPs. This puts forward a new requirement for the chip level of AI PCs.

Let's take a look at the current mainstream computing power of Meteor Lake, Ryzen 8040, and Snapdragon X Elite/X Plus.

The comprehensive computing power of Intel Meteor Lake is about 34 TOPS, and the NPU computing power is about 10 TOPS, officially announced to support the operation of 20 billion large models on the terminal side; AMD's Ryzen 8040 has a total computing power of 39 TOPS, and the AI computing power of the NPU is 16 TOPS; the computing power of the Snapdragon X Elite NPU has reached 45 TOPS, officially announced to support the local operation of large models with 13 billion parameters.Overall, in terms of chip computing power, Qualcomm has the most prominent advantage. In terms of computing speed (frequency), AMD's chips have a more outstanding ability to process instructions. In terms of power consumption, AMD's chips have a prominent advantage. In terms of core count, Intel's chips have stronger parallel data processing capabilities.

It is worth noting that Intel has announced its next-generation notebook chip, Lunar Lake, which will be able to provide more than 100 TOPS of performance, with the neural processing unit (NPU) capable of providing 45 TOPS. This performance also meets the 45 TOPS NPU performance threshold required for the next-generation AI PC that Intel proposed at the previous Taipei Artificial Intelligence Summit.

In fact, according to the "AI PC Industry (China) White Paper," when the edge-side hybrid AI computing power reaches 10 TOPS, it can complete AI model inference tasks for specific scenarios such as device intelligent management, image enhancement, and game optimization locally; when the AI computing power reaches 40 TOPS, it can meet most of the AI creation needs in scenarios such as work, study, and entertainment in conjunction with the GPU or cloud.

In this regard, the computing power of a truly mature AI PC chip will at least be one level higher than the current one.

03

What is AI PC looking forward to?

In early January, Microsoft added a brand new Copilot key to the Windows keyboard, marking the first major revision in nearly 30 years. With just one click of the button, users can interact seamlessly and intimately with Copilot. Now, with Microsoft's Copilot key, users can directly access the latest models, including OpenAI's GPT-4o.

Microsoft is the first company to quickly integrate GPT-4o into the terminal. Of course, considering the relationship between Microsoft and Open AI, the cooperation between the two parties should have started quite some time ago.

In fact, no matter what the hardware requirements are, a PC that can run and carry a large model locally can be considered a qualified AI PC. In the view of analysis institutions, future AI PC products will be more like personal AI assistants for users, which requires embedding personal large models, personal knowledge bases, and personal agents into the device to achieve multimodal natural language interaction of AI PCs and greatly improve the ability to understand intentions.The AI PC is the first step in the transfer of large models to the terminal.

Looking at the current large models on the market, a reasonable rule is that the larger the device (the more functions it has), the larger the parameter size of its large model on the terminal. For example, the recently unveiled SenseTime "Ririxin 5.0" uses a Mixture of Experts (MOE) architecture and is the first large model in China to fully benchmark and even surpass GPT-4 Turbo, with a parameter scale of 600 billion.

Large models with billions of parameters are obviously not suitable for PCs.

We can take a look at the PCs currently released that can carry large models. Huawei's first AI PC product, the MateBook X Pro, supports Huawei's own Pangu large model and third-party cooperative large models such as Wenxin Yiyan, iFlytek Xinghuo, and Zhi Pu Qingyan. In terms of personal intelligence, the computer manager of Huawei AI PC has an AI space with more than 100 intelligent bodies, covering various capabilities such as copywriting and programming, bringing users a new AI experience.

Lenovo has compressed Alibaba's Tongyi Qianwen large model through large model compression technology, and the compressed large model is called the Lenovo AI Now model. In terms of personal intelligence, Lenovo has released the industry's first AI PC personal intelligent body - Lenovo Xiaotian.

Honor's AI PC product, the MagicBook Pro 16, is equipped with the Honor voice assistant "YOYO Assistant" to help users complete semantic search, document summarization, service recommendation, and auxiliary creation.

200 billion parameters have become the higher level that AI PCs can currently support. For large model manufacturers, how to provide lightweight large models is a problem.

The current mainstream method is to use large model compression technology, which can save storage space, improve computing efficiency, and accelerate the inference process without significantly reducing model performance. For example, knowledge distillation, model quantization, and weight pruning, etc., can transform data types into int8 or even int4, thereby further reducing the computing power required during the inference process.

If more complex large models can be efficiently implemented on the PC side, then in the future, in addition to OpenAI's GPT series, Alibaba Cloud's Tongyi Qianwen, Tencent's Hunyuan, Baidu's Wenxin series, and so on, may all become important components of the terminal AI PC. For example, in terms of large models and AIGC applications, Honor has reached a cooperation with Baidu.Are those who buy AI PCs now pioneers or just being taken advantage of?

In the early days when large models were not so popular, it was quite good. Many enthusiasts and researchers would assemble their own, choosing to tinker with these open-source large models or open-source projects on their local computers.

So, how much does it cost to assemble a desktop computer that can train personal large models on the market? A rough calculation of the cost of each component is as follows:

It can be seen that the price of assembling a desktop computer to run large models locally is roughly between 11,000 and 40,000 yuan. Generally speaking, desktop computers do not need to consider portability and size, and the price is cheaper than that of laptops.

IDC is more conservative in terms of pricing. According to IDC's statistics, the average selling price of AI laptops will be in the range of 5,500 to 6,500 yuan, and the average price of AI desktop computers is about 4,000 yuan.

However, looking at the currently announced prices of AI PCs, taking the high-end positioned ThinkPad X1 Carbon AI 2024 as an example, the top configuration of 32GB+2TB version is priced at 16,999 yuan. The newly released YOGA Book 9i has the highest price of 17,999 yuan. And the high-end version of Huawei's newly released MateBook X PRO, Ultra9 32GB 2TB, is 14,999 yuan. If you want to buy Microsoft's Surface Pro with an X Elite chip and an OLED screen, you need to spend at least 11,088 yuan.

Although research institutions expect that demand growth and performance improvement will drive the average selling price of AI PCs to continue to rise after 2024, the overall trend will be "high initial pricing, price decline in the middle and late stages". The extent of the decline depends on the cost reduction space of hardware equipment such as chips. However, at the initial release of AI PCs, this price and performance are not enough to make the general public spend money to buy new machines.Observing the Global AI PC Landscape

With the release of AI CPUs and Windows 12, the year 2024 is set to mark the beginning of large-scale shipments of AI PCs, and by 2027, AI PCs are expected to become the mainstream type of PC products. Over the next five years, the global PC industry will deeply enter the era of AI PCs. IDC forecasts that the compound annual growth rate of global AI PCs over the next five years will reach 126%, and the penetration rate of China's AI PC market is expected to increase to about 85% by 2027. AI PCs will become the main driving force for the development of the PC market in China and globally.

From a hardware perspective, processor chips, memory, cooling, and interaction will be the main areas of revenue generation; in terms of model layer, vertical end-side models tailored for various industries will become the main development trend in the future, thereby supporting the AI transformation of traditional software and the development of AI-native applications; at the terminal layer, mainstream PC manufacturers are accelerating the layout of AI PC-related products.

In terms of chip ecosystem capabilities, Intel's Ultra chip ecosystem has a more comprehensive strength, with more top-tier large models and AI PC brand merchants as ecosystem partners.

Starting from the second quarter of 2023, leading manufacturers have successively launched early AI PC products such as ThinkPad 14 and EliteBook 805, and this trend has further accelerated in the second half of 2023, with major manufacturers launching more AI PCs with stronger performance. According to the statements and product progress of major manufacturers, in 2024, each leading manufacturer will launch a wave of new models with AI acceleration, providing users with differentiated experiences in a timely manner.

Hewlett-Packard, Dell, Lenovo, Acer, and ASUS have all stated that they plan to launch new AI PCs in sync with Intel and AMD's new CPU product roadmap, advancing products to the market during the Windows update period in 2024-2025, providing opportunities to accelerate equipment upgrades.Application Layer - Application Scenarios and Application Software

The application scenarios of AI PCs mainly include vertical industry-specific scenarios and general application scenarios. Vertical industry-specific scenarios mainly encompass educational vertical scenarios, medical vertical scenarios, legal scenarios, and financial scenarios, among others. General application scenarios refer to the AI PC's ability to provide personalized creative services, personal secretary services, and device manager services for scenarios such as work, study, and life.

The AI application software ecosystem is in its infancy, and international mainstream personal AI application software is restricted domestically, with the domestic software application ecosystem awaiting development. Application scenarios in various vertical segments still require further development. In the era of AIPC, deploying local large models on the terminal side is only the foundation; it is crucial to build an AIOS interactive interface and ecosystem, whose number and quality of applications and models directly determine the user's AI experience. It is currently still in the development stage.

06

Future Development Trends of the AI PC Industry

From the perspective of the industrial chain development trend, the heterogeneous scheme of "CPU+NPU+GPU" for upstream AI PC chips will become mainstream, and support secondary development of AI PC chips by users, while terminal-side models will show a trend of lightweight, industry-specific, and personalized development. In the midstream, mainstream PC brand players continue to accelerate the layout of the AI PC track. In the short term, Lenovo's AI PC has stronger comprehensive strength, but in the long run, computer manufacturers with mobile phone brands, based on their ecological advantages of integrating mobile phones and computers, will have greater potential for AI PC development. In the downstream, the government, medical, and educational industries will become the main industries to empower AI PC landing scenarios.

From the perspective of product development trends, future AI PC products will mainly include two types: one with high AI computing power and the other with low AI computing power. AI PC products with high AI computing power can run a large number of terminal-side large models in various vertical industry segments, while AI PC products with low AI computing power mainly focus on terminal-side large models for voice, text, and image processing.

From the perspective of business model development trends, as AI software and related technologies continue to develop and land in the future, AI assistants in AI PCs will adopt a business model similar to WPS, setting different prices based on different levels of service.

From the perspective of the development trend of the ecosystem system, PC terminal manufacturers will take on the mission of organizing the industry ecosystem, integrating industrial resources for users based on scenario needs, and becoming the core hub of the PC industry ecosystem. AI model technology manufacturers will focus on developing lightweight and industry-specific terminal-side large models, providing personalized model fine-tuning services, and decoupling and adapting their own models to AI PC personal intelligent entities. Traditional application manufacturers need to cooperate with model manufacturers to upgrade traditional applications to large model-enabled applications; in the long term, they need to undergo a more thorough reconstruction to transform themselves into AI-native applications. AI application stores will aggregate AI-native applications and applications empowered by AI, and provide convenient search and download support. Chip manufacturers will focus on establishing a universal, compatible AI development framework, reducing the threshold for large model and application development adaptation, and providing chips with efficient and inclusive intelligent computing power.From a holistic perspective, smart devices, serving as the medium through which artificial intelligence (AI) reaches users, will profoundly transform the PC industry. The leapfrog development of generative AI and Large Language Models (LLMs) has deeply changed personal life and work patterns, accelerating the intelligent transformation across various industries. The development of AI is shifting from being software-led to a parallel drive of hardware and software, and smart devices, as the ultimate medium for AI to reach users, are becoming an important breakthrough for the future development and implementation of AI. The integration of AI models with PCs will bring innovations in architectural design, interaction methods, content, and application ecosystems, which will deeply transform the PC industry.