As live events and presentations become more sophisticated, content creators will need sets or studios equipped to produce
immersive content that merges the real world
with the virtual. The demand for studios that
can accommodate Mixed Reality (MR) video
capture and live streaming, which combines
Virtual Reality (VR) and Augmented Reality
(AR), will grow rapidly in the near future.
These advanced studios will be either independent off-site locations that are available for
rent by the day or hour, or custom system integrations designed to be installed on-site at the
office. MR studios are proliferating at a steady
pace across the world. These studios are geared
for gaming or eSports, corporate communications, presentations, product launches, demos,
training, education, architecture, science, medical, government, and virtual commerce.
Both MR and VR blend the real and virtual
worlds, but each allows for a different level of interactivity. Currently, VR-based MR shows the
presenter in a fully computer-generated virtual
environment. Microsoft HoloLens MR is based
on holographic/3D objects being displayed in
real rooms and environments.
Today, live and recorded MR content is viewed
on 2D screens such as flat screens, smartphones,
and desktops to help viewers better visualize
how the presenter or player is merged in a virtual world. Viewing streams on devices such as
Oculus’ Gear VR or Android’s Daydream, VR
headsets, and HoloLens-like headsets will become more common over time.
VR/MR studios will need pre-lit greenscreens
that support room-scale VR. Solutions for chro-makeying/compositing live video with Unity 3D
and VR headsets will allow subjects to be composited with graphics, overlays, virtual sets, and
These studios will also be equipped with cam-
eras that are synced and calibrated to allow
multiple live, real-world camera feeds that are
mounted with trackers that match the virtual
camera Field of View (FOV) in 3D space while
switching between first-person Point of View
(POV) and third person when needed. Floating
cameras will be configured for motion tracking
on gimbals, allowing for fluid camera moves to
get different perspectives of the subject or pre-
senter composited consistently within the vir-
tual space and interacting with virtual objects.
When using the Microsoft HoloLens, Spatial
Mapping is required to allow the developer or
interface designer to identify where to place ob-
jects and reference points within the stage, or
on surfaces such as walls, ceilings, or floors, with
accurate scale and physics. This involves scan-
ning the set for real objects and converting them
into meshes to determine how presenters can
interact with them. Real cameras will be config-
ured to capture directly from a POV that shows
viewers what the HoloLens view looks like.
Unity is the current standard for VR, AR, and
MR development. It is a powerful 3D engine and
game development platform that actively facil-
itates development through partnerships with
all major VR companies and Microsoft for the
HoloLens. Unity 3D will be a critical component
in the studio since it allows for the creation of 3D
graphics, animations, and customized physics.
Unity 3D can trigger events with physical-location or object hotspots. A presenter will be
able touch an object that activates another sequence of events or animations, or even walk to
another location on the stage and be transported to another virtual set.
Widely used in the gaming community, Open
Broadcaster Software (OBS) is an open source
encoding and streaming solution optimized for
MR video. Other software encoders such as Telestream Wirecast can do the job if you composite
everything before it hits the encoder.
MR will enhance user experiences by providing a new degree of visualization and simulation, incorporating video, audio, data, graphics, holographs, and environments. It will deeply
impact viewers by drawing them deeper into
engaging, unforgettable experiences.
Mark Alamares ( firstname.lastname@example.org) is a tech entrepreneur and
media strategist for Fortune 500 companies. He is the CEO of
Mobeon, an advanced media studio and consultancy for enterprise,
specializing in immersive content production and distribution.
Comments? Email us at email@example.com, or check
the masthead for other ways to contact us.
Mixed Reality Studios
Are the Future of Live Streaming