Spatial Mapping for Magic Leap 1
Last Revised: June 13, 2022
This Learn More is intended to supplement the information included within Magic Leap's Privacy Policy regarding what data is collected by your Magic Leap 1 to enable spatial mapping, how that data is handled, as well as the spatial mapping features and the functionality available for your Magic Leap 1. For information regarding Spatial Mapping for Magic Leap 2, please refer to the “Learn Mores” for Magic Leap 2, available here.
In this Learn More, we cover:
For information regarding Magic Leap 1 devices or Magic Leap's first generation ecosystem, please refer to the “Learn Mores” for Magic Leap 1, available here.
What is Spatial Mapping?
Spatial maps are a critical part of the Magic Leap ecosystem as they help Magic Leap enable interactions between digital content and your actual physical surroundings. Magic Leap 1 device sensors continuously scan your environment, process that information, and use it to create three dimensional representations of your area (which we refer to as spatial maps). The end result is a mapped environment that enables apps to render digital media in your field of view as if it were really in front of you. Spatial maps can include three different levels of detail, based on the choices you make: World Features, World Models, and World Understanding.
What About People in My Surroundings?
Spatial maps that include World Features or World Models work similarly when someone is in your surrounding area. Spatial maps only reflect information from the parts of your environment that are stationary. If someone walks through the room while your Magic Leap 1 device is spatially mapping the space, they will be excluded from the spatial maps. If someone is sitting in the room while your Magic Leap 1 device is spatially mapping the space, that individual may be included in the spatial map created, but the spatial map is not designed to reflect a level of detail sufficient to identify that individual.
When using World Understanding, prior to running computer vision algorithms to identify objects, Magic Leap redacts the color images from your device using computer vision and data loss prevention algorithms, which are designed to obfuscate images that include faces or nudity. In addition, when World Understanding data capture is occurring, your Magic Leap 1 device's status indicator will display an orange LED pattern to let people in your environment know that color images are being transmitted to Magic Leap.
World Features
World Features are 3D representations (also called “point clouds”) of the world around you, and serve as the basic building blocks of a spatial map. World Feature points can be used as references to place and persist content (so you can come back to that content later, at the same place).
How World Features Work
Computer vision algorithms use images collected from the peripheral cameras on your Magic Leap 1 device to extract features from those images. These new features are then compared to previously stored features and merged to create one set of features that represent your surroundings. The final set of World Features is a spatial map at its most basic level. The processing to extract World Features from peripheral camera images is done locally on your Magic Leap 1 device. If you choose to store your World Features in a Personal World or contribute them to Shared World, only the resulting World Features spatial map will be sent to Magic Leap's cloud.
Your Choices
Magic Leap offers three different storage options for your World Features data. You can choose to store your World Features On Device or in a Personal World, or to contribute them to Shared World.
- If you choose to store your World Features On Device, World Features will still be available to you and your applications. However, there is a limit to the amount of spatial maps you can store on your Magic Leap 1 device.
- If you elect to store your World Features in the cloud (either a Personal World or contributed to Shared World), you will reduce the mapping time required by some applications and will be able to store more spatial maps.
- Using Shared World means your Magic Leap 1 device may not need to spatially map new areas you enter if other Shared World users have mapped them before you. In addition, if you contribute your World Features to Shared World, it will improve shared and other multi-user experiences by allowing both you and other users to utilize the same World Features - enabling you to see digital content in the same physical location.
You can change your selection around where your World Features are stored at any time in Settings. However, if you decide to contribute World Features to Shared World and then later decide to change your settings, the World Features you have already contributed to the collective spatial maps ordinarily will not be deleted. This is because once you contribute World Features to Shared World, they become part of the combined Shared World spatial maps, and are not directly associated with a particular device or account. If there are areas you do not want to be part of the collective spatial maps available to Shared World users, we encourage you to change your settings before entering those areas.
Also, please note that apps will be able to access the spatial maps of your surroundings, including World Features, while such apps are running on your Magic Leap 1 device, even while in the background. Apps need this information to display within and interact with your environment.
World Models
World Models add more detail to your spatial maps, including dense mesh data and planes. The result is spatial maps that provide much more accurate representations of the world around you, enabling digital content to respond to the environment in a more natural way.
How World Models Work
World Models add additional components to your spatial maps, including:
Dense Mesh Data. Dense mesh data is a triangular geometry that closely approximates the structure of the visible walls and objects around you, such as furniture.
Planes. From dense mesh data, large flat surfaces or planes can be derived. This helps identify walls, floors, and optimal surfaces for placing content. Dense mesh data and planes additionally enable occlusion and collision consistent with your environment.
On Device Storage
World Models are designed to be processed and stored locally on your Magic Leap 1 device.
Also, please note that apps will be able to access the spatial maps of your surroundings, including World Models, while such apps are running on your Magic Leap 1 device, even while in the background. Apps need this information to display within and interact with your environment.
World Understanding
World Understanding is an optional component of spatial mapping which allows Magic Leap 1 devices to recognize some types or classes of objects in your environment, such as the ability to identify chairs and posters, and include data about those objects in your spatial maps. Applications can then access object data in your spatial maps (with your permission), and use it in a variety of ways, including to make your digital content fit even more seamlessly into the world around you.
Magic Leap will continue to improve and develop new classes of objects for detection in upcoming releases. You can find details about new classes of objects in the release notes for each Lumin OS release.
World Understanding requires cloud processing using Shared World and is not currently available when you save spatial maps only on device or store your World Features in a Personal World.
How World Understanding Works
When you elect to use World Understanding, your Magic Leap 1 device will send picture camera color images and depth sensor images to our World Understanding cloud service. The World Understanding service runs computer vision algorithms to identify objects in your space. Once objects are identified, the World Understanding service generates unique labels and other data about those objects, which are placed into your spatial maps (whether in a Personal World or contributed to Shared World).
Magic Leap retains the color images and depth sensor images collected by the World Understanding service for limited periods of time in order to train Magic Leap's computer vision models and improve performance of this functionality.
Your Choices
World Understanding requires cloud processing using Shared World and is not currently available when you save spatial maps only on device or store your World Features in a Personal World. If you have elected to contribute your World Features to Shared World, you can then decide if you would also like to enable World Understanding. If you enable World Understanding, you agree to contribute your World Understanding results to Shared World and other users who are contributing to Shared World may have access to the object data in the spatial maps you have contributed and you may have access to object data in the spatial maps contributed by other users.
You can change your selection around whether to enable World Understanding at any time in Settings. However, if you decide to enable World Understanding and contribute your spatial maps to Shared World and then later decide to change your settings, the World Understanding object data that you have already contributed to the collective spatial maps ordinarily will not be deleted. This is because once you contribute object data to Shared World, they become part of the combined Shared World spatial maps, and are not directly associated with a particular device or account. If there are areas you do not want to be part of the collective spatial maps available to Shared World users, we encourage you to change your settings before entering those areas.
Third-party apps require your permission to access object data that has been included in your spatial maps. You will be asked to allow a third-party app to access object data when such app tries to access object data for the first time. Note that you may be asked for this permission if you are contributing your spatial maps to Shared World, even if you have World Understanding disabled, as there may be object data on your spatial maps that was contributed by other users. If you wish to change which permissions you have granted a third-party app, you can do so through Settings at any time.
Spatial Mapping Storage Options
Magic Leap offers three different storage options for different types of spatial map data. You can choose to store your World Features “On Device” or in a “Personal World,” or to contribute them to “Shared World.” Each of these options is described in more detail below. If you decide to contribute World Features to Shared World, you can then decide if you would also like to enable World Understanding, and your World Understanding results will be contributed to Shared World as well. World Models are designed to be processed locally on your Device.
On Device
On Device storage means that your spatial maps will be stored locally on your Magic Leap 1 device and will not be sent to Magic Leap's cloud. When you store your spatial maps locally on your Magic Leap 1 device, map generation and localization is handled by your device's operating system. Spatial maps will still be available to you and your applications. However, there is a limit to the amount of spatial maps you can store on your Magic Leap 1 device and once your Magic Leap 1 device's memory is full, previous spatial maps will be overwritten. As a result, one impact of this setting is that your Magic Leap 1 device may spend more time spatially mapping areas you have visited previously.
Personal World
If you choose to store your World Features in a Personal World, Magic Leap will store your World Features data in its cloud, but those World Features will not be shared with other Magic Leap users. Using a Personal World allows you to save more World Features than you can on your Magic Leap 1 device and your Magic Leap 1 device will spend less time spatially mapping areas you have visited previously. Storing your World Features in a Personal World also enables apps to persist content across device sessions so that content appears in the same place the next time you access it.
To identify the specific spatial map you need when you enter an area, spatial maps that are stored in Magic Leap's cloud are associated with information to identify where the spatial map was created, which includes information about the closest WiFi networks (BSSIDs (aka MAC address), signal strengths, and SSIDs (aka WiFi network names)), as well as latitude and longitude for the Magic Leap 1 device's location. Sections of spatial maps are then also assigned an area tag number, which is used to identify which spatial maps should be sent to your Magic Leap 1 device when you enter an area and what spatial maps are thought to be nearby to queue up if you move.
Shared World is Magic Leap's platform to maximize the power of spatial computing. If you elect to use Shared World, the World Features (and if you enable World Understanding, your World Understanding results) in each spatial map created from your Magic Leap 1 device about a particular area will be saved to a collective spatial map of that area stored in Magic Leap's cloud ecosystem. This means the World Features and World Understanding results in a spatially mapped area will be available to all Shared World users who contribute to the collective spatial map of that area. Using Shared World means your Magic Leap 1 device may not need to map new areas you enter if other Shared World users have mapped them before you. If you contribute your spatial maps to Shared World, it will improve shared and other multi-user experiences by allowing both you and other users to utilize the same World Features - enabling you to see digital content in the same physical location.
Every time a spatial map is generated for an area in Shared World, the collective version of the spatial map is updated to reflect new information, though previous versions of spatial maps are retained to improve ongoing spatial map quality, even if they are not used in the current collective map. Certain areas may have already been spatially mapped by alternate processes, such as LIDAR. In these areas, spatial maps originating from Magic Leap 1 devices may not be merged into the collective spatial map.
Similar to when you store your World Features in a Personal World, spatial maps that are contributed to Shared World are associated with information to identify where the spatial map was created to identify the specific spatial map you need when you enter an area, which includes information about the closest WiFi networks (BSSIDs (aka MAC address), signal strengths, and SSIDs (aka WiFi network names)), as well as latitude and longitude for the location. Sections of spatial maps are then also assigned an area tag number, which is used to identify which spatial maps should be sent to your Magic Leap 1 device when you enter an area and what spatial maps are thought to be nearby to queue up if you move.
You can change your selection around where your World Features are stored at any time in Settings. However, if you decide to contribute World Features to Shared World and then later decide to change your settings, the World Features you have already contributed to the collective spatial maps ordinarily will not be deleted. This is because once you contribute World Features to Shared World, they become part of the combined Shared World spatial maps, and are not directly associated with a particular device or account. If there are areas you do not want to be part of the collective spatial maps available to Shared World users, we encourage you to change your settings before entering those areas.