Microsoft 365 Copilot for Mobile

Empowering every person and every organization on the planet to achieve more like never before using the power of AI

Duration
March '23 - March'24

Team
Office AI - IDC Noida

My role and responsibilities
Worked as a product designer towards shaping the vision, execution and successfully shipping in the first Copilot experience in Word and PowerPoint on mobile in November '23.
Tap to read details

Collaborating during sprints, I initially explored document consumption scenarions on mobile and iPad, working closely with Office AI PMs to design and showcase the project's vision to leadership for funding the project and start development.

Post leadership alignment, my day-to-day responsibilites included participating in user research and come up with insights to define and deliver high-fedility design workflows for the document consumption experience with Copilot on mobile. I worked closely with developers while progressing the experience from multiple inner-ring testing stages to succefully shipping for General Availability by November 2023.

I also co-initiated and actively maintained the Copilot mobile design system library specific to office apps, aligning with the One Copilot branding guidelines, that was leveraged by multiple product teams working on mobile platform. In design discussions with the Fluent design system team, I shared mobile designs, contributing to the evoling Fluent Copilot design library. Also designed end-to-end workflows for iOS and Android, ensuring clear communication and cross platform consistancy.

Staying updated on guidelines, I collaborated with engineering, tested builds, and participated in sprints and bug bashes. Sharing insights with fellow designers, I drove clarity in designs and workflows. Partnering with the Global MSAI development team, I supported development for General Availability and initiated plans for the mobile V-next, exploring scalability in the One Copilot design language.

Copilot in Word

Summarise document • Ask suggested questions • Chat with Copilot

Quickly review documents on your mobile device with Copilot.
Copilot can help generate summaries, suggest questions,
and chat with you about the document.

View summary

Open a Word document shared with you.
Tap Get a Summary from Copilot on the nudge at the bottom of the app to open the Copilot pane and generate a summary of the document.
For more information about the document, choose one of Copilot's suggested questions, type a question, or tap the microphone to ask Copilot with your voice. Copilot will scan the document and give you an answer.

Chat with Copilot

Use Copilot icon to chat about the document.
Tap the microphone to ask Copilot a question with your voice or type in prompt box.
Tap the Send arrow for Copilot to generate a response.
Copilot will include references to show you where it pulled its information from.

Ask Copilot's suggested questions

To view Copilot's suggested questions and answers, tap Suggest some Q&As in the Copilot pane, or tap the Sparkle icon in the compose box and select the prompt.
Browse Copilot's suggested questions then tap on the down arrow to see Copilot's answers.

Copilot in PowerPoint

Summarise deck • Ask suggested questions • Identify Key Slides • Chat with Copilot

Need to review a presentation on the go?
With Copilot in PowerPoint, you can ask questions and get a quick
summary of the presentation or important key slides, all on your mobile phone.

View summary

Open the PowerPoint app and select a presentation that has been shared with you.
Tap Get Summary from Copilot nudge to generate a summary of the presentation.
Ask follow-up questions about the summary in the prompt box or use Copilot's suggested questions.

View key slides

In the prompt box, tap the Sparkle icon to open the prompt guide, and select Show key slides.
Copilot generates a summary of the key slides in the pane.

Ask Copilot questions

In the Copilot pane, tap Show Suggested Q&A to view suggested questions and answers.
Tap the down arrow to see Copilot's answers.
Use the microphone in the prompt box to ask Copilot another question with your voice or type your prompt.

Behind the Design

Emerging technologies like large language models (LLMs) are redefining human-computer interaction, shifting AI from "autopilot" to "copilot" requiring a fundamental shift in interaction design. At Microsoft IDC, we developed the Copilot experience on mobile—a revolutionary conversational UX for Microsoft 365 apps that not only unlocks the potential of LLMs but does so ethically, empowering users and amplifying human agency.

By rethinking everything from visual identity to interaction patterns in the current system, we crafted the system focusing on fostering "appropriate trust"—helping users understand the technology’s capabilities and limitations. Copilot pioneers a new UX paradigm balancing innovation and responsibility.

AI for All Altitudes

Productivity spans multiple levels—immersed in details, strategizing broadly, or synthesizing complex information. Copilot empowers users at every altitude by aligning LLM capabilities with human cognitive needs through a three-part framework: Immersive, Assistive, and Embedded experiences.

Immersive experiences

These provide deep, contextual understanding across tools and tasks, enabling creation, collaboration, and comprehension. With LLMs and business data, Copilot helps you gain insights, prepare for presentations, and adapt to your needs seamlessly.

Assistive & Embedded experiences

Focused on accelerating work within specific apps, these experiences support tasks like summarising in Word or showing key slides in PowerPoint. They adapt to your context, scaling from embedded modules to full conversations with Copilot, ensuring consistency and flow in every interaction.

Enhancing Document Comprehension with Microsoft 365 Copilot

Copilot excels in breaking down complex content, summarizing key points, and uncovering insights, making information more accessible and actionable.
By automating repetitive tasks and integrating seamlessly across the Microsoft 365 ecosystem, Copilot keeps knowledge flowing and supports users in staying focused, creative, and efficient.

Designed to empower

Copilot design is centered on an ethical framework that prioritizes human agency. From UI patterns to visual identity, we balanced user control and guidance—ensuring the AI’s capabilities are transparent and accessible. Instead of hiding functionality behind buttons, we opted for natural language interactions that encourage exploration while providing safeguards against misuse. Recognizing LLMs as probabilistic rather than deterministic, we designed turn-by-turn conversational UX to help users harness the Copilot's power while staying aligned with their goals.

Visualizing AI Moments with Clarity

Color and iconography to clearly distinguish AI interactions across Microsoft products. Vibrant accents signal when Copilot is active or producing outputs, building trust by making AI moments visually transparent. This design reinforces user awareness and confidence, enabling informed critique of AI-generated content.

Discoverability

Introducing the intelligence of Copilot into existing systems in order to ensure users easily discover and engage with Copilot.

M365 Copilot app chat gets invoked from the universal top navigation in order to have a consistent entry point across multiple office apps.  Branding is mono-line icon as the other icons in the menu.

Copilot nudge to suggest relevant actions based on the content which also acts as a learning moment for the user discovering Copilot's potential

Creating intentional friction

To maintain human agency, we introduced intentional friction to avoid overreliance on the model. Copilot isn’t autopilot—it requires human oversight and collaboration, as the best results come from back-and-forth interactions where users refine responses with each turn.

Education is paramount

Zero-state design, fallibility notices, sharing intended use cases, prompt suggestions — optimized to prioritize and promote education.

Interacting with an LLM is brand new, fairly novel, and potentially intimidating, so we want you to have a clear overview of its capabilities and limitations from the very first time you enter the experience

AI badge to go alongside all Copilot outputs that would allow you to learn more about the underlying tech being used anytime you tap or click on it.

That’s a vital part of building responsibly with AI; ethical principles are moot if they’re not baked into the actual UI.

We use zero-state design (what a person sees on screen before interacting) to teach people about mistakes the model might make and the need to fact-check outputs.

Sharing intended use cases and prompt suggestions to prioritize and promote education through the design.

Similarly, what the model puts out is only as good as the prompt that you type in, and prompt writing is a new skill that will take time to master.

Longer, more detailed prompts tend to give you better results, which is why we created a prompt capabilities menu and include prompt suggestions as hints.

Over time, as people get more familiar and comfortable with this technology, the designs will likely shift to emphasize that less.

Making it worth the wait

Large language models (LLMs) sometimes require extra processing time to generate responses, but those seconds can save hours. In designing for Microsoft 365 Copilot, we embraced latency as an opportunity to create transparency and user engagement.

By leveraging moments of waiting—through features like zero states to educate users, reminders to fact-check, and insights into how the model generates responses—we turned potential frustration into eager anticipation. These moments not only manage expectations but also reinforce trust and understanding of the AI’s capabilities and limitations.

Flair appears right after output is finished generating. This is a quick highlight animation to draw attention to the component after a refresh or completion and is designed to play once, without a transition to any other state.

Citations

When tapping directly on a citation, a sheet comes up and the relevant reference gets highlighted for a brief period of time.

This helps the user learn about the references and by tapping on it they can see where the content is present in the document.

Error handling

Different experiences have different error cases, making it a challenge to author a single set of error strings to cover every occurrence. The common goal in addressing errors is to give users a plain-spoken explanation and guide them towards a successful outcome whenever possible.

In chat, we use conversational responses to offer guidance on the next steps, gently steering the user away from the error situation.

Collecting feedback is crucial

There are still many unknowns with GPT. As we bring this technology to customers, we might encounter problematic, offensive, and challenging outputs.

Therefore, it is crucial to have a clear and distinct pathway for users to report problematic outputs. This framework is crucial for us to learn where the risks are and make changes to safeguard our users. It will also empower our users in moments where trust could be lost.

Positive Feedback: