AI-integrated immersive / interactive design is the future of the experience design industry. This might sound far-fetched, but by "integration," I mean top-down, full-stack integration. Amazon recently announced the integration of “AI” into several of its flagship products. While this represents a surface-level, single-point integration, it's newsworthy not because it’s groundbreaking, but for raising questions about privacy (Source: Wired).
By integrating closed-loop AI into key aspects of designing interactive and immersive experiences, we open up a wilderness of imagination for every user (Example: Medium). Dave Vondle used a combination of LLMs and physical computing to retrofit an old payphone into an access point for an AI-driven alternate reality game (ARG). By controlling what gets sent to external LLMs and when, he created a nearly closed-loop system without breaking the bank. Customizing individual narratives automatically; reimagining surroundings in projected or screened environments on a whim; and creating truly customized experiences will be more possible than ever with top-down AI integration, whether generative (Midjourney, Stable Diffusion, KreaAI), language models (ChatGPT, Alpaca), or applied AI (computer vision and sensor fusion).
It's important to note the distinction between these AI types, as they're often discussed interchangeably. When I say "full-stack AI integration," I’m referring to the combined and coherent use of each AI type. Many interactive experiences already implement AI in various ways, whether using generative AI visuals or computer vision for tracking. By utilizing these different forms of AI in conjunction with existing experience design techniques and technologies, we can create stunning, imaginative, and most importantly, legitimately immersive, and interactive experiences. AI enables scalability once thought impossible for boutique agencies, accelerates workflows in the design phase, integrates into production for real-time location services (RTLS) and other tracking systems, and its deployment at launch creates memorable Instagram moments.
Yet, AI alone does not ensure good experiential design; proper use of experiential technology is paramount. As AI is just another tool, it should be added to your experiential design palette. Adding new tools to the experience design workflow is one of the most rewarding steps in research and development, enabling the creation of new and even more joyful experiences. Correct use of any experiential technology is what makes or breaks an experience.
Many industries have been exploring the power of immersive and interactive experiences for years, including retail, museums and the arts, advertising, and even corporate headquarters looking to enhance their billion-dollar offices. With all these opportunities to create amazing, joyful, or educational experiences, audiences and customers are inevitably disappointed by low-effort, non-immersive or non-interactive experiences. These experiences are often limited by budgets, client demands, and executive expectations, leading to an underwhelming experience that is often only good for that one Instagram post.
We dream of having closed-loop, networks of AI trained on specific computer vision tasks, narrative language models, and IP-trained AI-voiced characters. Even more so, I dream of an open-source solution that makes this easy. Such a system would enable creative, boutique studios, like Future Colossal, to create tangible, immersive experiences despite constraints like budget and timelines. The content creation industry is already benefiting greatly from a multifaceted approach to AI use. YouTube and TikTok are filled with AI-generated content, music, and even AI VTubers. These channels create scripts with ChatGPT, animate in Blender using AI-generated assets, and use AI music models for background tracks. If this same technique is applied to the immersive/interactive experience design industry, the impact will be enormous, and hopefully the results will be unforgettably spectacular.
