Microsoft’s annual developer conference, Build, which took place from May 23-25 in Seattle, Washington, gave us an inside look at the exciting updates for developers. This time, they put AI in the spotlight, unveiling their ambitious AI initiatives and what they have planned for the year ahead.

Day 1 was filled with major announcements, including collaboration between Microsoft and OpenAI to nurture and expand the AI plugins ecosystem. The company adopted an open standard for plugins that seamlessly integrated with OpenAI’s ChatGPT and Microsoft’s Copilot solutions, such as Bing Chat, Dynamics 365 Copilot, Windows Copilot, and Microsoft 365 Copilot. It’s an exciting time for AI and plugin development!

Key takeaways from Executive Vice President – Experiences and Devices, Rajesh Jha’s session. 

They spilled the beans on how developers can be the masters of the future of work with Microsoft 365 Copilot, and how Windows 11 is going to take AI and productivity to a whole new level.

We are looking at the new frontier of computing.

Microsoft 365 Copilot – the future of natural language

There are numerous plugins available for Microsoft 365 Copilot users to select from. You have Microsoft Power Platform connectors, Teams messaging extensions, and ChatGPT plugins. By enabling you to take advantage of your current software and tool investments, as well as your knowledge and abilities, these plugins aim to make your life as a developer easier. In order to boost up your development skills, there is a plugin waiting for you to investigate whether you’re into ChatGPT, Teams, or the Power Platform.

Creating Plugins Made Easy with Teams Toolkit

To build Teams messaging extensions that work as Microsoft 365 Copilot plugins, developers can use the Teams Toolkit for Visual Studio, Visual Studio Code, and CLI. To make the process of developing, testing, and debugging plugins simpler, Microsoft has added additional features to the Teams Toolkit. Developers can simply add any API provided by the OpenAPI specification to Microsoft 365 Copilot using the Teams Toolkit’s plugin-building experience. The private preview version of this functionality is presently accessible.

Customize User Experience with Adaptive Cards

When a developer’s plugin is used in conjunction with Adaptive Cards, they have complete control over how the user interface is customized. The Teams Toolkit constructs a plugin that comprises manifest and declarative Adaptive Cards, defining the Copilot user experience, based on the metadata in the OpenAPI specification. Before testing their plugin in Copilot, developers can further customize the manifest and cards to meet their unique scenario, including specifying login settings.

Supercharge Your Data with Semantic Index for Copilot

The doorway to clients’ compliance, security, and privacy policies, as well as their productivity and collaboration data, is provided by Microsoft Graph. By utilizing the power of Microsoft Graph data, Copilot makes sure that user prompts and responses are based on this information. Additionally, it inherits permissions and security during runtime, giving users control over security and governance. Importantly, the information does not go toward building Microsoft’s foundation models and instead stays in the tenants of the consumers.

Introducing the Semantic Index for Copilot

The Semantic Index for Copilot is an intricate user and business data map that enables Copilot to surface Microsoft Graph data. The Semantic Index captures semantics and similarities surrounding content and users by constructing vector embeddings from Microsoft Graph data. As a result, Copilot can quickly search through the billions of objects in a vector index using semantic search. In order to use the Semantic Index for Copilot, developers can use Graph connectors to upload their data to Microsoft Graph and receive personalized and useful results.

Access Business Data with Microsoft Dynamics 365 and Power Platform

Microsoft 365 Copilot now allows accessing structured data from Microsoft Dynamics 365 and Microsoft Power Platform stored in Microsoft Dataverse in addition to productivity and collaboration data in Microsoft Graph. This means that Copilot replies can be supported by both business and user data from Microsoft Graph. As we roll out Microsoft 365 Copilot in the program, customers in the Microsoft 365 Copilot Early Access Program will have the chance to test out this feature. Through Microsoft Power Platform connectors, developers can add data to Dataverse, increasing Copilot’s potential.

Developers Shaping the Future of Work with Microsoft 365 Copilot

As customers use Microsoft 365 Copilot with connected apps and services, developers are key in establishing new work patterns. In a simulated scenario, we see a user at Dentsu Inc., a provider of integrated marketing services and solutions, utilizing the capabilities of Microsoft 365 Copilot in conjunction with Atlassian’s Jira plugins, a line-of-business Dentsu app plugin for sourcing media assets, and data from Atlassian’s Confluence that has been integrated using a Microsoft Graph connector. Without the need for new code, this integration demonstrates Microsoft 365 Copilot’s potential.

Accelerating Success for Every Developer

Developers may take advantage of the AI opportunity and connect with millions of Microsoft 365 users thanks to the new extension architecture for Microsoft 365 Copilot. To create plugins for Copilot, developers can make use of their current knowledge, code, and resources. Through the Teams and Microsoft 365 platforms, Microsoft offers thorough support for the whole app life cycle, including tooling, distribution, management, commerce, and corporate readiness. Without writing additional code, developers who already have Teams messaging extensions can expand Microsoft 365 Copilot, and the Teams Toolkit for Visual Studio makes it simple to create new plugins. There are monetization possibilities in the commercial market.

More Innovations to Enhance App Building

Teams meetings can now feature real-time collaboration thanks to the Live Share SDK. Microsoft Teams’ customized avatars and reactions, which are now generally available, are designed to increase participation. Microsoft Teams’ immersive rooms, now in private preview, transform virtual meetings by fostering shared experiences that transcend geographical borders. With Microsoft Mesh, which is currently in private preview, developers can create unique, immersive experiences for the workplace that promote connection and belonging no matter where users are physically located.

Explore the Future of Work with Microsoft 365 Copilot

Customers can soon test out the extensibility features introduced at Microsoft Build as Microsoft 365 Copilot enters the Early Access Program. Building a Teams messaging extension for their plugin, developing a Microsoft Graph connection to import data into Copilot’s Semantic Index, and enrolling in the developer early access program are just a few of the preparations that developers and partners may make in advance of the release of Copilot. Utilize all the available tools and resources to get the most out of Microsoft 365 Copilot.

Exciting Innovations to Enhance App Building

We have several additional exciting news that will assist developers in creating apps for the changing workplace, in addition to Copilot extensibility. Teams meetings can now feature real-time collaboration thanks to the Live Share SDK. Meetings are made more enjoyable and interesting with the help of Microsoft Teams’ configurable avatars and reactions, which are being released in phases. Microsoft Teams’ immersive rooms, which are now under private preview, enable members to join virtually, virtually, or physically in the immersive space itself, creating shared experiences that transcend physical borders. Furthermore, Microsoft Mesh, which is now in private preview, gives creators and developers the tools to design unique, immersive experiences for the workplace that promote connection and belonging.

Highlights from Executive Vice President & Chief Product Officer Panos Panay’s session.

Introducing the Windows AI Library

To help you get started with your AI work, we’re launching the Windows AI Library, which includes a collection of ready-to-use machine learning models and APIs. Soon, further information on availability and preview dates will be available.

Hybrid AI Scenarios with ONNX Runtime and Olive

The development pattern Hybrid Loop, which we released last year, enables hybrid AI scenarios spanning Azure and client devices. Today, we’re happy to declare that this vision has come true thanks to Olive, a toolchain that makes it easier to optimize models for different Windows and other devices, and ONNX Runtime, which serves as the gateway to Windows AI. You can use ONNX Runtime to execute AI models on Windows or other devices utilizing CPU, GPU, NPU, or a combination with Azure. These are the same tools we use internally.

Hybrid Inferencing with ONNX Runtime and Azure EP Preview

The same API is now supported by ONNX Runtime for executing models locally or in the cloud. This makes it feasible to use local resources when they are available and cloud resources when they are needed in hybrid inferencing scenarios for your app. You can connect to models set up in AzureML or the Azure OpenAI service using the new Azure EP preview. You can have more control over prices and user experience by defining the cloud endpoint and the requirements for when to use the cloud.

Optimizing AI Models with Olive and ONNX Runtime

Olive is an extendable toolchain that combines cutting-edge methods for model compression, optimization, and compilation to help you optimize your models for various hardware targets. By utilizing ONNX Runtime, you can extend your Windows AI investment to all of your app platforms, including Windows, iOS, Android, and Linux. Olive and ONNX Runtime both speed up the deployment of your AI models into applications, making it simpler to build incredible AI experiences with less engineering work and higher performance.

Partner Success Stories with NPUs on Windows

By utilizing NPUs to run their AI models, partners including WhatsApp, Luminar Neo, and Camo are realizing significant performance improvements and opening up new user experiences. Deliver AI experiences like Camo with your apps by integrating the ONNX Runtime into the Windows platform. With these new AI toolchains and hardware upgrades for the NPU, we can’t wait to see what the Windows developer community will come up with.

Productivity Enhancements with Dev Home

Let’s discuss the new features and enhancements that Windows 11 has to offer in order to increase productivity for all developers. We are aware that the life of a developer entails several manual activities and interruptions that may lower productivity. Dev Home, a new productivity ally for Windows 11 developers, is being introduced as a result. You can experience Dev Home to get back in the flow and organize your work.

WinGet configuration is one of the Dev Home’s features, which streamlines and expedites the machine setup procedure. By using a WinGet configuration file, you can save the laborious manual process of configuring a new system or launching a project. You only need to execute one command to get started coding!

Dev Drive: Tailored Storage Volume for Developers

Dev Drive, a storage volume created especially for developers, is also being introduced. The Resilient File System serves as its foundation, and it provides enhanced performance and security. To make managing repositories with thousands of files and directories easier, Dev Drive is enhanced to store project source code, working folders, and package caches.

Customizable Dashboard and GitHub Integration in Dev Home

Dev Home comes with a dashboard that you can customize so you can keep track of all your workflows and tasks in one spot. To effectively manage coding assignments, pull requests, and projects, you may even add GitHub widgets. To make it simple to start producing games, we’re working with Team Xbox to bring the GDK (Game Development Kit) to Dev Home. To further customize your experience, you may even add your own specially created extensions.

Smarter Windows Terminal with GitHub Copilot Integration

With the integration of GitHub Copilot, your preferred tool, Windows Terminal, is becoming more intelligent. The Terminal application will make use of natural language AI to suggest commands, describe mistakes, and aid you in performing tasks. In other developer tools, such as WinDBG, we are also testing with GitHub Copilot.

Improvements to the Taskbar for Enhanced Workflow

We’ve made changes to the Taskbar based on your suggestions. With just one click, you can now easily recognize and open any instance of an app that is stored in the Taskbar. The app’s instances are all individually labeled. In addition, you can conceal the Time and Date in the Taskbar so that you can record your screen without having to remove the time and date in order to stay focused. Additionally, you can terminate a program without opening the Task Manager by performing a simple right-click on the application from the Taskbar.

These are just a few of the fascinating new functions and enhancements that Windows 11 will provide developers. We’re dedicated to easing the burden and bringing enjoyment and fun back to development. Keep checking back for further updates, and we can’t wait to see what you do with these new features!

Updates and Enhancements in the Microsoft Store on Windows

The Microsoft Store for Windows was also the subject of some interesting news from Panos! We’re updating the Microsoft Store platform because we want to give you the best resources and chances to create outstanding apps. The following is new:

Microsoft Store AI Hub: The best AI-powered experiences produced by Microsoft and the developer community will be featured in a unique section of the Microsoft Store that will be unveiled soon. It will be a carefully chosen selection of excellent AI apps and materials.

AI-Generated Keywords: We’re releasing a useful tool in Partner Center that makes search tags for your apps by utilizing AI. By doing this, you may improve app discovery and make it simpler for customers to find your app in the store.

AI-Generated Review Summary: We are aware that it can be time-consuming to sift through a large number of app evaluations. To give users a quick and simple method to browse app reviews, we’re harnessing the power of AI to gather and summarize reviews. Users will be able to find fresh information more quickly as a result.

Microsoft Store Ads: Beginning in June, we’ll make Microsoft Store Ads available in more than 150 regions worldwide, extending its reach outside the United States. A new premium spotlight placement that we’re introducing will provide you with more opportunities to reach your target audience with high traffic visibility in the Store and Bing.com search results.

Backing up and restoring apps: We are aware of how crucial it is for you to keep clients when they transfer devices. We are therefore enhancing the backup and restore functionality for apps. You may test out and give feedback on this update since it is currently in preview and is accessible to Windows Insiders.

Conclusion 

The most recent improvements and changes to the Microsoft Store for Windows are giving developers more control than ever. The potential for developing and exhibiting outstanding apps has never been stronger thanks to new AI-powered features, increased advertising opportunities, and improved tools for user reach.

The launch of the Microsoft Store AI Hub will curate the best AI experiences, and app discovery will be sped up by AI-generated review summaries and keywords. Additionally, a global audience is now available thanks to the expansion of Microsoft Store Ads to over 150 territories globally.

Developers can confidently offer excellent experiences and keep customers across devices when combined with improved backup and restoration capabilities. These updates open a fascinating new chapter in Windows development, and we eagerly anticipate seeing the amazing works of art and accomplishments they will foster.

[To share your insights with us, please write to sghosh@martechseries.com]