Microsoft has started testing a new feature coming soon to Copilot on Windows 11 that allows the user to share their entire desktop with Copilot, enabling the ability for Microsoft's AI assistant to view and chat about what's on your screen, right alongside you.
The company first announced Copilot Vision for Windows 11 earlier this year, and first rolled out the ability to share individual apps or windows with Copilot. Now, the company is testing the ability to share your entire desktop, letting Copilot see multiple apps all at once if you choose.
Of course, the feature is entirely optional and requires you to enter into a special mode in the Copilot app first before Copilot Vision becomes active. Copilot is never automatically looking at your screen, requiring you to explicitly share your desktop with the AI assistant first before it can see what you see.
"When you share your desktop (or any specific browser or app window), Copilot can see what you see and talk to you about it in real time," says a Microsoft blog post announcing the share desktop feature. "It can help analyze content, provide insights, and answer your questions, coaching you through it aloud. Get tips on making improvements to your creative project, help with improving your resume, or guidance while navigating a new game."
Copilot Vision turns the Copilot app into a genuinely useful tool that lets it work alongside you, like a real copilot. You can use natural language to chat with Copilot as it views your screen with you, meaning you can talk on the fly about different topics as you context switch between apps and windows.
Copilot isn't able to directly manipulate your desktop to help out with tasks, at least not yet. Instead, it can highlight areas of the screen it wants you to click whenever you ask it for help in completing a task. If you don't know how to use a particular app, Copilot Vision can guide you through whatever task it is you need to complete.
Microsoft has announced a number of new features coming to Copilot this year, including Copilot Memory, which lets Copilot build a profile about you and carry that information across devices, learning your habits, likes, and dislikes.
The company has also teased that Copilot will eventually have a customizable persona, which will let users create a character that they can chat with using the Copilot app. Microsoft teased that you might even be able to make Copilot look like Clippy, which is a very fun idea.
Last year, Microsoft released a major update to the Copilot app that redesigned the user interface while delivering a native Windows 11 app, one that isn't web based. The company has slowly been adding new features to it ever since, including Copilot Pages, deep research, and more. The Copilot key on Copilot+ PCs is also set to open this app by default, offering fast access to features like Copilot Vision with the tap of a button.
Sharing your desktop with Copilot on Windows 11 is now in testing across all Windows 11 Insider Preview channels, including Release Preview, Beta, Dev and Canary Channels. The update can be acquired via the Microsoft Store, and will likely rollout in waves meaning you might not see it right away in the app.
Of course, Copilot Vision is only available in the United States currently, and sharing your desktop with Copilot is gated behind that same region lock. That means if you're in a market that doesn't have access to Copilot Vision yet, you won't be able to test sharing your desktop with Copilot today.
English (US) ·