vrPACE - Germany
WINNERS OF THE NAKATHON 2023!
BUSINESS IMPACT CATEGORY WINNER
USER EXPERIENCE CATEGORY WINNER
BEST PITCH PEOPLE CHOICE CATEGORY WINNER
*** You can watch vrPACE experience at the bottom of this page
Team members
Alexander Kroll
Head of Cloud Native & Backend Development (Virtual Reality Visionary)
Dennis Herzner
Cloud Solution Architect
(Chief Collaboration Catalyst)
Julian Haimerl
Senior Technical Consultant
(Design Director Extraordinaire)
Stefan Sauer
Cloud Infrastructure Engineer
(Lead Voice Virtuoso)
Dimitri Hainatski
Managing Technical Consultant
(Senior Transcription Titan)
The proposal
We developed AIcebreaker, an AI-powered virtual agent that accompanies employees on their onboarding and carrier journey. In the virtual world the user can conduct a real and interactive dialogue with the agent using the voice as input.
The agent is trained to answer questions about the company, the core values and carrier opportunities in a very friendly and engaging way.
The phases of the project
Ideation phase
We wanted to figure out how we can make use of emerging technologies like artificial intelligence - especially the well-known and prominent candidate ChatGPT - in virtual experiences.
There were two main challenges to tackle:
- How can ChatGPT provide real benefit in an onboarding scenario?
- How can we make interaction as easy as possible? (Writing text into a form in 3D space has the worst possible user experience … seriously)
After some experiments with ChatGPT we found that the bot can give very good answers to questions that new joiners typically have. The idea of an AI-powered onboarding assistant was born. Regarding the interaction we found that “talking” will be the best and most natural way.
We had to implement audio recording using the microphone, speech-to-text, the conversation with ChatGPT and text-to-speech. Challenge accepted!
Implementation
After some research we found that we can easily use several APIs to achieve our goals and implement a simple Q&A loop. For the main functionality three calls to different services needed to be implemented and orchestrated:
- Speech-to-text -> OpenAI Transcription API
- ChatGPT -> OpenAI ChatCompletion API
- Text-to-Speech -> Azure Cognitive Services
For sake of simplicity we implemented our solution without a custom backend (keep it simple…you know). That means that the Unity Game orchestrates all calls to the APIs. For productive use we (as experienced Cloud architects and engineers) would not recommend this. However, at the end of the first day of the Hackathon we were able to talk to a simple white cube that replied to questions about the company and gave us advices for our future and bright career at NTT DATA.
Tweaking
The second day of the Hackathon was about making our solution perfect. We replaced the cube by a real avatar and added animations and icons.
Also we improved the answers of ChatGPT. We did this by adding more context into the initial configuration message (aka system message).
This message is hidden to the user and gives ChatGPT instructions for the conversation, example:
- You answer with few sentences. (this is very important because ChatGPT otherwise answers very verbose…)
- You are a helpful assistant that welcomes the chat partner very friendly.
- You are representing the company NTT DATA and want to make the onboarding process as comfortable and engaging as possible.
Pitch preparation
For the final pitch we prepared a storyline and exercised the presentation multiple times upfront. We even recorded the dry-runs in order to deal with Murphy’s law in case of technical issues (you never know…). We even trained the assistant to answer questions about its implementation details and tell a joke about the company.
Our efforts into this preparation paid off and convinced the jury!