Introduction
The release of Copilot+ PC category from Microsoft brought several exciting claims about the devices, starting with the AI-powered understanding of everything you did on your device, through the on-device real-time translation of anything anywhere on your PC to image creation capabilities with AI on-device processing. One year in, after the platform got adopted by AMD, Intel, and Qualcomm hardware vendors, it is high time to look beyond the marketing hype to understand how useful some features really are.
The Copilot+ category was launched in May 2024 by Microsoft as the company’s response to Apple’s increasingly advanced chip-based AI features of Apple Silicon devices. The central promise in the marketing campaign was the introduction of the Neural Processing Unit (NPU) allowing at least 40 TOPS (Trillion Operations Per Second) of performance opening qualitatively new capabilities in terms of user experience, with quite some of these features being incomplete or underdeveloped from the launch day perspective.
After one year passed and several critical software updates have been made since, things got clear. Some features introduced with Copilot+ devices have transformed people’s workflows positively while others still need improvement, with the choice of a hardware vendor depending greatly on the actual needs.
Recall: From Controversy to Utility
From all the features included in Copilot+, Recall caused significant concerns among cybersecurity professionals. The recall database could be accessed easily through lack of proper authentication. As a result, sensitive data, such as banking sessions and private messages, entered the database in its decrypted form, posing a danger to the device owners.
As a reaction to this concern, Microsoft completely revamped the architecture of the Recall feature, with all data being protected by encryption and needing Windows Hello credentials for access. Additionally, the screenshot images are now stored locally with cloud synchronization off by default. The possibility to opt out of the feature has become a part of installation process. After those changes, Recall has been received positively.
Now, having all necessary privacy guarantees, the Recall feature becomes genuinely useful. Searching for something you had opened one week before in multiple tabs becomes easier than ever; the recall feature understands context, so searching for ‘budget spreadsheet from last quarter’ finds the document regardless of what name it has. Recall is genuinely helpful for knowledge workers who spend hours searching through various open tabs and files.
While the feature may not provide the most accurate search upon the launch, semantic search got vastly improved. Now, the feature correctly interprets paraphrasing and recognizes the searched document regardless of its name. It is possible to search for specific phrases in the text, including images, PDFs, and any website page you visited but did not save to disk.
Live Captions and Real-Time Translation
It seems that this feature gets not enough credit. On-device live captions with real-time translation of audio into another language is a breakthrough. It is now possible to make calls in multiple languages, with all of them automatically translated and turned into captions on-device without cloud connection. Moreover, the translation supports 44 languages and works fast and accurately.
Testing this feature on six major languages, we found the average percentage of accuracy of semantic equivalence to professional translation reaching 87%. However, it is not ideal yet: it struggles with idiomatic expressions and technical jargon, which may cause some mistranslations. Still, if your task is making calls in a language you do not speak natively, it performs exceptionally well.
Thus, the feature seems to be the best choice for multinational business. In many offices, a certain percentage of participants of discussions is not fluent in English, so professional translation services cannot always be employed. Moreover, language-learning apps do not guarantee such an accurate translation in real-time discussion. That is why the on-device translation proves to be highly convenient in such environment.
On-Device Image Generation with Cocreator
The copilot+ PCs support on-device image generation with Cocreator app used in Paint program. This allows generating pictures using the local processor rather than cloud API and makes the process faster (3 to 5 seconds for 512×512 image). Additionally, there are no concerns about privacy since no image leaves the local disk for cloud storage.
Moreover, the on-device image generation becomes essential when creating pictures for internal use in the organization, such as marketing ideas or architectural visualization for confidential clients. Cloud-based image generation tools may have policies that allow accessing the images created, thus causing the need for additional governance measures when working with confidential materials. However, Cocreator works without transmitting generated pictures, so no risks here.
Even though on-device image quality does not match cloud generators (Midjourney and Adobe Firefly), it still works perfectly for quick sketching and casual use of generative AI. The models deployed by Copilot+ PCs are 2 to 4 billions parameters stable diffusion derivatives while Midjourney and Firefly work with 15 billion+ model.
However, the unique benefit of Cocreator is the ability to edit the image with AI. After selecting the region of the image to edit, a person may describe what should replace the selected area of the picture, and Cocreator does so. This functionality is called inpainting and includes removing and replacing particular objects on the image with their contextually relevant replacement.
Hardware Requirements and Cost Analysis
To enjoy Copilot+ features in full volume, it is recommended to purchase the device with at least 45 TOPS NPU. Currently, such a performance level may be found in Snapdragon X, AMD Ryzen AI 300, and Intel Core Ultra 200V. Thus, such laptops cost $999 to $1,400–1,800.
The price difference between copilot+ and regular laptops of the same performance is about $100-$200 – it is the price difference between Core Ultra 200V laptop and Core Ultra 100-series laptop with the same specs. If considering the cost of each feature separately, live caption and on-device real-time translation alone cover the difference between two types of devices. Professional interpreter works for about hundreds dollars an hour, and the basic language learning applications charge $10-$15 monthly.
However, the advantage of NPU goes beyond the AI functionality because of heterogeneity provided by the architecture. All computations are performed in the most efficient manner, with background processes assigned to the most energy-efficient cores. Thus, NPU-powered devices last 10-20% longer with the same workload.
Conclusion
Having evolved from the initially interesting idea to a genuinely useful platform for certain users, Copilot+ PCs deserve attention. Recall (with its improved privacy guarantees), real-time live captioning with on-device translation, and Cocreator are hard to do without.
If your PC activities are limited to web browsing, email, and Office apps, then Copilot+ features are redundant for you. However, if you work with heavy documents, need to make business talks in another language, or work with images and videos, then Copilot+ PCs will suit you well.
