Company: Le Polish Bureau
World's first Apple Vision Pro project to receive the FWA award.
In today's increasingly digitised world, the quality of our relationships suffers. Especially across generations. Our parents and grandparents feel left out from our heavily rushed and often geographically remote lives.
I wanted to imagine a completely new way of communicating, one that would utilise the spatial capabilities of Apple Vision Pro. One that takes 20 seconds for the sender to produce 5 minutes of high quality, deeply personal content for the receiver. Received messages, while fundamentally recorded audio, undergo analysis by AI and are transcribed into text. Subsequently, the AI generates a spherical image that encapsulates the message's content. This visual representation is then transformed into a three-dimensional bubble, enabling the recipient to gain an immediate 'sense' of the message's essence. When a message is played back, we immerse the user in an AI-generated representation of the message's content.
I developed the application using Xcode Beta, Swift, and SwiftUI.
I designed and animated 3D components, such as the bubble representation of the message, using custom shaders and shader graphs in Reality Composer Pro.
On the backend, I established an auto-scaling infrastructure on the Google Cloud Platform, incorporating a Google Cloud Run service to host our backend API. Whenever a voice message was sent, it was routed through this backend to undergo transcription into text using Google Cloud Speech to Text. Subsequently, I utilized OpenAI's ChatGPT API to analyze the message's semantic content and distill key concepts articulated by the user. I then tasked ChatGPT with transforming these concepts into descriptive scenes. Lastly, I fed these descriptions into Blockade Labs' AI-powered Skybox generator to produce immersive 360-degree scenes.
