XR, MR, VR, AR…. In a world where digital and physical spaces increasingly converge ('phygital'), the role of the audio-visual motion designer is more critical than ever.
Those creatives (like us at AY-PE) bring life to immersive audio-visual experiences, software and cross-reality digital interactivity, captivating audiences in evocative, inspiring and often jaw-dropping moments. By marrying traditional design methods with evolving technology, we can craft compelling experiences - tailored to unique environments, technology and human psychology to maximize the end impact.
AY-PE real-life phygital immersive environments, in VR
From Concept to Canvas: The Designer’s Journey
When a client approaches us with an idea for an immersive project - whether it’s a film projected onto irregular building or sculptural surfaces, mixed reality holographic experiences or largescale, multi-user, interactive digital installations - they’re essentially seeking a translator. We convert written or oral messaging and abstract visual concepts into tangible, breathtaking experiences.
This process requires a balance of creativity, technical expertise, and spatial understanding, developed from analogue design through to meta-tech.
At the heart of any awesome experiential digital design though, is a thorough understanding of context, location and canvas - designing in your mind’s eye with the ‘twin’ setting of the end reality.
The process iterates in three ways:
1. Traditional Foundations: Sketching the Vision
The journey begins with traditional design tools. Storyboards or base animatics serve as the initial blueprint, providing a visual script that outlines the narrative flow of the experience. These are often augmented by hand-drawn or digital sketches of the physical space—a three-dimensional representation of the intended projection surfaces, be they walls, sculptures, or entire rooms.
For example, a dome projection might start with sketches or animatics that explore the curvature of the surface and how visual elements might flow across it. These early stages allow us to experiment with ideas and align with the client’s vision before committing to more resource-intensive processes.
2. Bringing Ideas to Life with CGI 3D Modelling
As the storyboards and scripts solidify, computer-generated imagery (CGI) of the space grows alongside it, with our designers working in digital 3D modelled environments. These tools allow for precise simulation of the physical space, complete with detailed geometry and textural understanding of the projection surfaces.
This stage is where the interplay between content and space is fine-tuned. We experiment with how light, motion, and texture interact within the confines of the room or object and how the message or story unfolds from a visitor’s interpretational perspective.
Our designers might use this for:
Sculptures as Canvases: we simulate how a story might play out across the angled shards of a large jagged sculpture. Careful iterative CGI designing ensures the beauty or power of the physical piece is maximised with audio-visual flow and the interplay of content across it, whilst also delivering a moving, memorable and enthralling story.
Room Dynamics: we explore how graphic colouring, motion and animation might impact audiences in the space, working alongside the exhibition or events branding and the space’s wider content. Balancing these impacts can include anything from effecting the right mood, to guiding foot traffic and audience pulsing within a space.
Walls as Dynamic Storyboards: we test how a narrative unfolds across multiple walls and sound positioning, ensuring the AV content guides the viewer’s gaze effectively, to land the most impactful experience.
AY-PE 3D model built from photogrammetry to test lighting, wall textures and plan projection show CGI with the church's real-world conditions.
3. Digital Twins and Virtual Reality: Elevating Design
To push the boundaries of immersion, designers now incorporate digital twins into production - virtual replicas of physical spaces that can be explored in real-time using virtual reality (VR). Tools like Unreal Engine or Unity are employed to create these high-fidelity models, allowing clients and teams to step into the design before it’s executed.
With a VR headset, stakeholders can:
Experience the installation as if they are physically present.
Provide feedback on the scale, pacing, sound placement and aesthetics of a project.
Explore interactivity in real-time, testing how users might engage with content.
For largescale mixed / extended reality / phygital experiences, digital twins offer an exceptional ability to ease the ‘leap of faith’ that goes between approving a video file of the experience, to the physical install itself. The client can see the audio-visual software in operation in their space, removing uncertainties and speeding up the process to the ‘wow that’s amazing’ stage.
Digital twinning seven of AY-PE's real-life phygital / XR / MR productions
Broaden the appeal
But that’s not all. Digital twinning in experiential design is a key tool for more: access, outreach and promotion.
Got an exhibit, area or behind the scenes experience that some or all people cannot access? Digital twin it.
About to launch or showcase a new largescale item and want to engage audiences with it?
Digital twin it.
Or if you, like us, have created amazing immersive experiences which are based in set locations around the world and you would love more audiences to see them at scale and context.
Digital twin it.
We recently showcased seven of our in-situ AV experiences at the UNESCO Media Arts Expo. In reality these experiences are based at fixed locations across the world (such as London’s British Museum, the Science Museum Hong Kong, or Silverstone Museum/ racetrack) - missing the engagement of many.
Ques of eager users of all ages formed, keen to dive into our immersive experiences and explore the virtual reality twin and feel what the evocative, real-life experience is. Users, quite literally, nearly fell of the VR chair, trying to launch further into the hybrid world.
Connecting Audiences to Reality
From traditional sketches to cutting-edge digital twins, the experiential AV design process is an iterative creative journey of transforming ideas into living, breathing experiences.
As the boundaries between the physical and digital worlds continue to blur, the role of the motion designer becomes ever more vital - crafting experiences that are not only visually stunning but also contextually connective and deeply human, resonating across physical spaces and realities.
Make sure your design processes, outreach or promotions are making the most of this phygital world.
Reach out to discuss how our AV digital artistry can help achieve your vision, messages and goals: info@ay-pe.com