{"id":335473,"date":"2025-04-16T12:56:17","date_gmt":"2025-04-16T19:56:17","guid":{"rendered":"https:\/\/msftnewsnow.com\/?p=335473"},"modified":"2025-05-07T05:12:26","modified_gmt":"2025-05-07T12:12:26","slug":"computer-use-feature-microsoft-copilot-studio-ui","status":"publish","type":"post","link":"https:\/\/msftnewsnow.com\/computer-use-feature-microsoft-copilot-studio-ui\/","title":{"rendered":"Microsoft Copilot Studio Reveals New “Computer Use” Feature to help pave the way for Next-Generation UI Automation with AI"},"content":{"rendered":"

Microsoft<\/a> has announced a major leap forward in business automation with the early access release of computer use<\/strong> in Microsoft Copilot Studio<\/a>, its low-code AI agent platform. This new feature allows Copilot agents to interact directly with any website or desktop application, using simulated clicks, keystrokes, and menu selections\u2014just like a human user. By bridging the gap between traditional API-based automation and the real-world interfaces businesses rely on, Microsoft is redefining what\u2019s possible for enterprise AI and robotic process automation (RPA).<\/p>\n

What Is “Computer Use” in Copilot Studio?<\/strong><\/h2>\n

\"Microsoft<\/p>\n

Computer use<\/strong> is a powerful new capability that enables Copilot agents to treat any system with a graphical user interface (GUI) as a tool. Agents can now automate workflows by clicking buttons, selecting menus, and entering data into fields on both web and desktop applications\u2014even when no API integration exists. As Charles Lamanna, Microsoft\u2019s Corporate Vice President for Business & Industry Copilot, puts it<\/a>: \u201cIf a person can use the app, the agent can too.”<\/p>\n

This breakthrough means Copilot Studio agents can:<\/p>\n

    \n
  1. Automate data entry across legacy and modern systems.<\/li>\n
  2. Conduct market research by navigating and extracting data from online sources.<\/li>\n
  3. Process invoices by extracting and inputting data into accounting platforms.<\/li>\n
  4. Interact with applications in real time, adapting instantly to UI changes without breaking workflows.<\/li>\n<\/ol>\n

    How Computer Use Transforms Automation<\/strong><\/h2>\n

    \"Microsoft<\/p>\n

    Unlike traditional RPA solutions, which often break when UI elements change, Copilot Studio\u2019s computer use feature leverages advanced AI reasoning. Agents automatically adapt to changes in app layouts or website designs, ensuring uninterrupted automation. This resilience is coupled with robust security and compliance frameworks, keeping enterprise data safe within Microsoft Cloud boundaries and ensuring it is not used to train external AI models.<\/p>\n

    The feature is fully hosted on Microsoft\u2019s infrastructure, eliminating the need for organizations to manage their own RPA servers. This not only accelerates deployment but also reduces maintenance and infrastructure costs, making enterprise automation more accessible and cost-effective.<\/p>\n

    Key Use Cases for Businesses<\/strong><\/h2>\n

    Microsoft highlights several high-value scenarios where computer use delivers immediate benefits:<\/p>\n

      \n
    1. Automated Data Entry<\/strong>: Agents can handle large-scale data transfers between disparate systems, reducing manual effort and errors.<\/li>\n
    2. Market Research<\/strong>: Marketing teams can automate the collection and analysis of online data, gaining insights without manual browsing.<\/li>\n
    3. Invoice Processing<\/strong>: Finance departments can streamline invoice data extraction and entry, accelerating the entire accounting workflow.<\/li>\n<\/ol>\n

      Making RPA Smarter and More Intuitive<\/strong><\/h2>\n

      With computer use, Microsoft Copilot Studio is democratizing automation. Users can describe automation tasks in natural language\u2014no coding required\u2014and test them using real-time video previews of the agent\u2019s reasoning and UI actions. This makes sophisticated automation accessible to non-developers and business users, not just IT specialists.<\/p>\n

      Other standout features include:<\/p>\n

        \n
      1. Real-time adaptation<\/strong>: Agents keep working even as buttons or screens change.<\/li>\n
      2. Intelligent decision-making<\/strong>: Agents \u201csee\u201d what\u2019s on the screen and make smart choices in complex environments.<\/li>\n
      3. Full transparency<\/strong>: Users can review a detailed history of agent activity, including screenshots and reasoning steps.<\/li>\n<\/ol>\n

        The Road Ahead: Early Access and Microsoft Build 2025<\/strong><\/h2>\n

        \"Microsoft<\/p>\n

        Microsoft is inviting organizations to apply for early access to the new feature in Copilot Studio<\/a>. The company will showcase more about this innovation at Microsoft Build in May 2025<\/a>, signaling a future where AI-powered agents can automate virtually any digital task, regardless of platform or integration limitations.<\/p>\n

        Microsoft Copilot Studio\u2019s new capability marks a new era in business automation, making UI automation smarter, more resilient, and accessible to all. As AI continues to transform enterprise workflows, Copilot Studio can help organizations streamline operations, boost productivity, and drive innovation across every industry.<\/p>\n","protected":false},"excerpt":{"rendered":"

        Microsoft has announced a major leap forward in business automation with the early access release of computer use in Microsoft Copilot Studio, its low-code AI agent platform. This new feature allows Copilot agents to interact directly with any website or desktop application, using simulated clicks, keystrokes, and menu selections\u2014just like a human user. By bridging … Read more<\/a><\/p>\n","protected":false},"author":208461344,"featured_media":334575,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"enabled":false},"version":2}},"categories":[24,2458],"tags":[829,761,778,946,1083],"class_list":["post-335473","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","category-ai-and-copilot","tag-copilot","tag-developer","tag-microsoft","tag-microsoft-copilot","tag-security"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/Copilot-hero-1024x576-1.jpg","jetpack_shortlink":"https:\/\/wp.me\/pfgCZY-1pgR","jetpack-related-posts":[{"id":336011,"url":"https:\/\/msftnewsnow.com\/microsoft-365-copilot-app-intros-search-image-note\/","url_meta":{"origin":335473,"position":0},"title":"Microsoft 365 Copilot App Introduces New Advanced Search, AI-Powered Image Features, and Collaborative Notebooks","author":"Dave W. Shanahan","date":"April 23, 2025","format":false,"excerpt":"Microsoft has announced a transformative Spring 2025 update for the Microsoft 365 Copilot app, unveiling a suite of advanced AI agents and features designed to redefine how people and organizations work. This release, dubbed the \"Wave 2 Spring release,\" marks a pivotal shift from AI as a simple productivity tool\u2026","rel":"","context":"In "AI and Copilot"","block_context":{"text":"AI and Copilot","link":"https:\/\/msftnewsnow.com\/ai-and-copilot\/"},"img":{"alt_text":"Microsoft 365 Copilot App Introduces New Advanced Search, AI-Powered Image Features, and Collaborative Notebooks","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/m365copilotnew.webp?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/m365copilotnew.webp?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/m365copilotnew.webp?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/m365copilotnew.webp?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/04\/m365copilotnew.webp?resize=1050%2C600&ssl=1 3x"},"classes":[]},{"id":10391,"url":"https:\/\/msftnewsnow.com\/copilot-for-sales-copilot-for-service\/","url_meta":{"origin":335473,"position":1},"title":"Microsoft unveils Copilot for Sales and Copilot for Service, 2 useful tools in AI-driven productivity and customer service","author":"Dave W. Shanahan","date":"February 2, 2024","format":false,"excerpt":"Microsoft has announced the general availability of two new tools designed to enhance productivity and customer service: Microsoft Copilot for Sales and Copilot for Service. These tools are part of Microsoft's commitment to helping organizations transform the way people work using secure, enterprise-grade AI capabilities.","rel":"","context":"In "AI and Copilot"","block_context":{"text":"AI and Copilot","link":"https:\/\/msftnewsnow.com\/ai-and-copilot\/"},"img":{"alt_text":"copilot for sales, copilot for service","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/MSFT_Dynamics365_Jan_30_Blog_240130_V01_1260.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/MSFT_Dynamics365_Jan_30_Blog_240130_V01_1260.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/MSFT_Dynamics365_Jan_30_Blog_240130_V01_1260.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/MSFT_Dynamics365_Jan_30_Blog_240130_V01_1260.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":13103,"url":"https:\/\/msftnewsnow.com\/microsoft-copilot-for-finance-now-avail\/","url_meta":{"origin":335473,"position":2},"title":"Copilot for Finance: Leveraging AI to transform tedious financial operations in Microsoft 365","author":"Dave W. Shanahan","date":"February 29, 2024","format":false,"excerpt":"Microsoft introduces Copilot for Finance, the latest AI-driven component in the Microsoft 365 Copilot series, crafted to elevate finance team performance. This cutting-edge tool harnesses artificial intelligence to automate and refine financial processes, offering finance professionals streamlined tasks and valuable insights within their everyday workflow.","rel":"","context":"In "AI and Copilot"","block_context":{"text":"AI and Copilot","link":"https:\/\/msftnewsnow.com\/ai-and-copilot\/"},"img":{"alt_text":"copilot for finance","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/copilot.webp?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/copilot.webp?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/copilot.webp?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2024\/02\/copilot.webp?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":332280,"url":"https:\/\/msftnewsnow.com\/sales-agent-and-sales-chat-microsoft-365-copilot\/","url_meta":{"origin":335473,"position":3},"title":"New Sales Agent and Sales Chat in Microsoft 365 Copilot: AI-powered sales assistants that make closing deals faster and smarter","author":"Dave W. Shanahan","date":"March 5, 2025","format":false,"excerpt":"Microsoft has once again pushed the boundaries of artificial intelligence in the business world with its latest announcement of two revolutionary AI agents designed specifically for sales teams. These cutting-edge tools, integrated into Microsoft 365 Copilot, promise to transform the way sales professionals work, enabling them to close more deals\u2026","rel":"","context":"In "AI and Copilot"","block_context":{"text":"AI and Copilot","link":"https:\/\/msftnewsnow.com\/ai-and-copilot\/"},"img":{"alt_text":"New Sales Agent and Sales Chat in Microsoft 365 Copilot: AI-powered sales assistants that make closing deals faster and smarter","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/03\/hl9un7fkvg6eumbangsj.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/03\/hl9un7fkvg6eumbangsj.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/03\/hl9un7fkvg6eumbangsj.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/03\/hl9un7fkvg6eumbangsj.jpg?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":348403,"url":"https:\/\/msftnewsnow.com\/microsoft-365-copilot-vs-microsoft-copilot-what\/","url_meta":{"origin":335473,"position":4},"title":"Microsoft 365 Copilot vs. Microsoft Copilot: The Ultimate Comparison Guide for Better Productivity and Security","author":"Dave W. Shanahan","date":"June 23, 2025","format":false,"excerpt":"Microsoft\u2019s Copilot brand has rapidly become synonymous with AI-powered productivity across its ecosystem. Yet, the terms \u201cMicrosoft Copilot\u201d and \u201cMicrosoft 365 Copilot\u201d are often used interchangeably, leading to confusion about what each actually offers, where they overlap, and what sets them apart. This comprehensive guide demystifies the differences, drawing on\u2026","rel":"","context":"In "How To Guides"","block_context":{"text":"How To Guides","link":"https:\/\/msftnewsnow.com\/how-to-guides\/"},"img":{"alt_text":"Microsoft 365 Copilot vs. Microsoft Copilot: The Ultimate Comparison Guide for Better Productivity and Security","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/06\/dmk4o7rorneuzmactjms-scaled.jpg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/06\/dmk4o7rorneuzmactjms-scaled.jpg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/06\/dmk4o7rorneuzmactjms-scaled.jpg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/06\/dmk4o7rorneuzmactjms-scaled.jpg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/06\/dmk4o7rorneuzmactjms-scaled.jpg?resize=1050%2C600&ssl=1 3x"},"classes":[]},{"id":341818,"url":"https:\/\/msftnewsnow.com\/microsoft-build-2025-preview-new-ai-models-copilot\/","url_meta":{"origin":335473,"position":5},"title":"Riveting Microsoft Build 2025 Preview: New AI Models, Copilot Upgrades, and JP Morgan Tech Conference Insights","author":"Dave W. Shanahan","date":"May 13, 2025","format":false,"excerpt":"Microsoft is gearing up for its annual Build developer conference, taking place May 19th\u201322nd, 2025, at the Seattle Convention Center and online. Microsoft Build 2025 is set to be a pivotal moment for the company, with industry watchers and developers anticipating a wave of announcements centered on artificial intelligence, cloud\u2026","rel":"","context":"In "News"","block_context":{"text":"News","link":"https:\/\/msftnewsnow.com\/news\/"},"img":{"alt_text":"Riveting Microsoft Build 2025 Preview: New AI Models, Copilot Upgrades, and JP Morgan Tech Conference Insights","src":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/05\/download-9.webp?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/05\/download-9.webp?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/05\/download-9.webp?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/msftnewsnow.com\/wp-content\/uploads\/2025\/05\/download-9.webp?resize=700%2C400&ssl=1 2x"},"classes":[]}],"jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/posts\/335473","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/users\/208461344"}],"replies":[{"embeddable":true,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/comments?post=335473"}],"version-history":[{"count":0,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/posts\/335473\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/media\/334575"}],"wp:attachment":[{"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/media?parent=335473"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/categories?post=335473"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/msftnewsnow.com\/wp-json\/wp\/v2\/tags?post=335473"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}