Set Up AR Foundation and ARKit

Code and workflows to implement AR Foundation and ARKit in your project to create an AR build

What you'll develop on this page

Launching the project on an AR enabled iPhone

We will update our build to trigger an AR session if the platform is AR-enabled.

Github branch link: https://github.com/moetsi/Unity-DOTS-Multiplayer-XR-Sample/tree/Setting-Up-AR-Foundation

Join our Discord for more info

Setting up packages and subsystems

AR Foundation requires a bit more love than simply adding packages. We also need to specify "subsystems."

• Let's start by adding our packages to manifest.json

• Paste these two lines into your manifest.json file in the Project folder

"com.unity.xr.arfoundation": "4.2.3",
"com.unity.xr.arkit": "4.2.3",

Adding packages to manifest.json

• Next we need to enable our Plug-in

• In Unity, navigate to "Project Settings" (Edit > Project Settings) then click XR Plug-in Management and

• Click on the iOS tab and check the box next to "ARKit"

Adding ARKit as a plug-in in XR Plug-in Management

• Next we need to provide our app end-users a reason for why we're asking to use their camera (because we need it for ARKit) when we ask for permission during start-up of the app

  • Navigate to "Build Settings" (under "File"), then "Player Settings", and when "Player" is selected on the left panel, click the iOS tab

    • Scroll down to "Camera Usage Description" under "Other Settings" and enter the following text into the text field: "Camera will be used for AR capabilities"

    • Scroll down to "Requires ARKit support" and set it to true

Adding Camera Usage Description and enabling "Requires ARKit support"

We've enabled AR development on iOS devices in Unity

• We updated our manifest.json

• We enabled the ARKit subsystem in XR Plug-in Management

• We updated our Player Settings

Updating MainScene to integrate with AR platforms

AR Session and AR Session Origin

Now we are going to set up AR Session and AR Session Origin in MainScene. AR Session and AR Session Origin are GameObjects that are core to building Unity AR Foundation experiences.

Unity's explanation of ARSession

An AR scene should include an ARSession component. The AR Session controls the lifecycle of an AR experience by enabling or disabling AR on the target platform. The ARSession can be on any GameObject.

When you disable the ARSession, the system no longer tracks features in its environment, but if you enable it at a later time, the system attempts to recover and maintain previously-detected features.

If you enable the Attempt Update option, the device tries to install AR software if possible. Support for this feature is platform-dependent.

NOTE

An AR session is a global construct. An ARSession component manages this global session, so multiple ARSession components will all try to manage the same global session.

Checking for device support

Some platforms might support a limited subset of devices. On these platforms, your application needs to be able to detect support for AR Foundation so it can provide an alternative experience when AR is not supported.

The ARSession component has a static coroutine that you can use to determine whether AR is supported at runtime:

public class MyComponent {
    [SerializeField] ARSession m_Session;

    IEnumerator Start() {
        if ((ARSession.state == ARSessionState.None) ||
            (ARSession.state == ARSessionState.CheckingAvailability))
        {
            yield return ARSession.CheckAvailability();
        }

        if (ARSession.state == ARSessionState.Unsupported)
        {
            // Start some fallback experience for unsupported devices
        }
        else
        {
            // Start the AR session
            m_Session.enabled = true;
        }
    }
}

Session state

To determine the current state of the session (for example, whether the device is supported, if AR software is being installed, and whether the session is working), use ARSession.state. You can also subscribe to an event when the session state changes: ARSession.stateChanged.

ARSessionState	Description
None	The AR System has not been initialized and availability is unknown.
Unsupported	The current device doesn't support AR.
CheckingAvailability	The system is checking the availability of AR on the current device.
NeedsInstall	The current device supports AR, but AR support requires additional software to be installed.
Installing	AR software is being installed.
Ready	AR is supported and ready.
SessionInitialized	An AR session is initializing (that is, starting up). This usually means AR is working, but hasn't gathered enough information about the environment.
SessionTracking	An AR session is running and is tracking (that is, the device is able to determine its position and orientation in the world).

AR Session Origin

The purpose of the ARSessionOrigin is to transform trackable features, such as planar surfaces and feature points, into their final position, orientation, and scale in the Unity scene. Because AR devices provide their data in "session space", which is an unscaled space relative to the beginning of the AR session, the ARSessionOrigin performs the appropriate transformation into Unity space.

This concept is similar to the difference between "model" or "local" space and world space when working with other Assets in Unity. For instance, if you import a house asset from a DCC tool, the door's position is relative to the modeler's origin. This is commonly called "model space" or "local space". When Unity instantiates it, it also has a world space that's relative to Unity's origin.

Likewise, trackables that an AR device produces, such as planes, are provided in "session space", relative to the device's coordinate system. When instantiated in Unity as GameObjects, they also have a world space. In order to instantiate them in the correct place, AR Foundation needs to know where the session origin should be in the Unity scene.

ARSessionOrigin also allows you to scale virtual content and apply an offset to the AR Camera. If you're scaling or offsetting the ARSessionOrigin, then its AR Camera should be a child of the ARSessionOrigin. Because the AR Camera is session-driven, this setup allows the AR Camera and detected trackables to move together.

From AR Foundation's About AR Foundation documentation

• Back in Unity, navigate to MainScene in your Scenes folder and open up MainScene

  • Right-click on the Hierarchy, select "XR" then AR Session

  • Right-click on the Hierarchy, select "XR" then AR Session Origin

Creating AR Session and AR Session Origin

• Now let's create the script that will check for device support

  • If there is no AR support on a particular device attempting to use your app, then the AR Session component and the AR Session Origin GameObject will be disabled

  • Select the "AR Session" GameObject in Hierarchy

  • When AR Session is selected, go to the Inspector and click "Add Component" and create a new script called "ARPlatformInitializer" and click "Create and add"

  • Once created and added, drag the file into the Scripts and Prefabs/Mixed folder

  • Paste the code snippet below into ARPlatformInitializer.cs:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using Unity.Entities;
using Unity.NetCode;

public class ARPlatformInitializer : MonoBehaviour
{
    [SerializeField] GameObject m_Session;
    [SerializeField] GameObject m_SessionOrigin;

    IEnumerator Start() {
        if ((ARSession.state == ARSessionState.None) ||
            (ARSession.state == ARSessionState.CheckingAvailability))
        {
            yield return ARSession.CheckAvailability();
        }

        if (ARSession.state == ARSessionState.Unsupported)
        {
            //If we AR is unsupported we disable both GameObjects
            m_SessionOrigin.SetActive(false);
            m_Session.SetActive(false);
        }
        else
        {
            //If AR is supported we create our IsARPlayerComponent singleton in ClientWorld
            foreach (var world in World.All)
            {
                if (world.GetExistingSystem<ClientSimulationSystemGroup>() != null)
                {
                    world.EntityManager.CreateEntity(typeof(IsARPlayerComponent));
                }
            }

        }
    }
}

Creating ARPlatformInitializer

• Now let's create a new component (the component that is referenced in the script above, which is why you might have an error right now in Unity-- the component it references does not exist yet!)

• Create a C# script named "IsARPlayerComponent" in the Client/Components folder

• Paste the code snippet below into IsARPlayerComponent.cs:

using Unity.Entities;
using UnityEngine;

public struct IsARPlayerComponent : IComponentData
{
}

Create IsARPlayerComponet

• Now select the "AR Session" GameObject in Hierarchy again and drag the ARSession and ARSessionOrigin GameObjects from the Hierarchy into the appropriate fields in the ARPlatformInitializer component in Inspector

Setting the fields of ARPlatformInitializer

• Let's go to NavigationScene, hit play, and host a game to make sure that are GameObjects are indeed disabled (by expanding MainScene in Hierarchy while in playmode)

  

We have set up our two core AR GameObjects

• We added AR GameObjects to MainScene

  • AR Session

  • AR Session Origin

• We created ARPlatformInitializer

• We created IsARPlayerComponent

Configuring the AR Camera background using a Scriptable Render Pipeline

This is a necessary step when using AR Foundation with the Universal Render Pipeline.

We are going to be following the instructions here: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/ar-camera-background-with-scriptable-render-pipeline.html

• In Assets/ navigate to UniversalRenderPipelineAsset_Renderer and in the Inspector Add Renderer Feature "AR Background Renderer Feature"

Adding AR Background rendering to our URP Pipeline Asset

We will now be able to render the AR Background

• We updated our URP Pipeline Asset by adding a Renderer Feature for AR Backgrounds

iOS build configuration

We are going to create an iOS build configuration in Assets/BuildSettings.

• Navigate to BuildSettings, right-click, and select "Create", then, "Build", then, "Empty Build Configuration" and name it "iOS-Build"

Creating iOS-Build

• Expand "Shared Configurations" at the top then click "+ Add Configuration"

• Drag "BaseBuildConfiguration" into the field and then hit "Apply"

Adding BaseBuildConfiguration as a Shared Configuration

• Next on "Classic Scripting Settings" component

  • choose "IL2CPP" as Scripting Backend

  • choose "Debug" as IL2CPP Compiler Configuration

  • under add "Classic Build Profile" component, choose "iOS" from the Platform dropdown, hit "Apply", and then at the top of Inspector hit "Build and Run"

    • Sometimes you need to hit "Build and Run" twice as it fails the first time (sometimes even 3 times...)

      • "Editor's active Build Target needed to be switched to iOS..."

Setting IL2CPP as Scripting Backend

• Next, in Xcode, make sure you set up your signing capabilities

  • Click on the "Unity-iPhone" project at on the left

  • Click on "Signing & Capabilities"

  • Make sure you are able to sign the application

• Hit "Play" in XCode to deploy the application

  • you can either connect an iOS device to run your application or choose a simulated device as the destination

Setting up Signing & Capabilities in Xcode and hitting Play to deploy the application to a connected device

• Host a game

Launching Asteroids project (now AR-enabled) on an iPhone

The new iPhones eat up some top and side margins in the UI. We will update this in the final "UI Updates" section.

We created an AR session on iOS devices

• We created our iOS build configuration

• Compiled our code to Xcode

• Built our app in Xcode

Github branch link:‌

git clone https://github.com/moetsi/Unity-DOTS-Multiplayer-XR-Sample/ git checkout 'Setting-Up-AR-Foundation'‌

Join our Discord for more info