# Set Up AR Foundation and ARKit ## What you'll develop on this page ![Launching the project on an AR enabled iPhone](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFnZ-A1r0ksjtgBMIL%2F-MSFoIqbBiw0P_oAzYSx%2FARFoundation%20-%20Setting%20up%20-%20Deployed%20to%20iPhone.gif?alt=media\&token=559a69fe-2ead-4cdd-9178-59ef6c57e2d6) We will update our build to trigger an AR session if the platform is AR-enabled. Github branch link: {% hint style="info" %} [Join our Discord for more info](https://discord.com/invite/88j758eUvs) {% endhint %} ## Setting up packages and subsystems AR Foundation requires a bit more love than simply adding packages. We also need to specify "subsystems." * Let's start by adding our packages to manifest.json * Paste these two lines into your manifest.json file in the Project folder ``` "com.unity.xr.arfoundation": "4.2.3", "com.unity.xr.arkit": "4.2.3", ``` ![Adding packages to manifest.json](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFXLIOCXdtj-yjWvc5%2F-MSFYIg7CmgljczxTl8I%2FARFoundation%20-%20Setting%20up%20-%20Packages.gif?alt=media\&token=04055652-0ecc-47ac-8a0f-d11ea678ec53) * Next we need to enable our Plug-in * In Unity, navigate to "Project Settings" (Edit > Project Settings) then click XR Plug-in Management and * Click on the iOS tab and check the box next to "ARKit" ![Adding ARKit as a plug-in in XR Plug-in Management](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFXLIOCXdtj-yjWvc5%2F-MSFYMlovyXc1e-Peauq%2FARFoundation%20-%20Setting%20up%20-%20Adding%20arkit%20subsystem.gif?alt=media\&token=6eb1a8bb-cde7-4643-bda8-492ff5f70d29) * Next we need to provide our app end-users a reason for why we're asking to use their camera (because we need it for ARKit) when we ask for permission during start-up of the app * Navigate to "Build Settings" (under "File"), then "Player Settings", and when "Player" is selected on the left panel, click the iOS tab * Scroll down to "Camera Usage Description" under "Other Settings" and enter the following text into the text field: "Camera will be used for AR capabilities" * Scroll down to "Requires ARKit support" and set it to true ![Adding Camera Usage Description and enabling "Requires ARKit support"](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFYeDXcGzJK-kUBtoa%2F-MSFZ0ehgwFNAkJv3Z4L%2FScreen%20Recording%202021-01-29%20at%2006.51.44%20PM.gif?alt=media\&token=e0bd6baf-0e4a-4e0e-b239-c1089e1dc2af) {% hint style="success" %} We've enabled AR development on iOS devices in Unity * We updated our manifest.json * We enabled the ARKit subsystem in XR Plug-in Management * We updated our Player Settings {% endhint %} ## Updating MainScene to integrate with AR platforms ### AR Session and AR Session Origin Now we are going to set up AR Session and AR Session Origin in MainScene. AR Session and AR Session Origin are GameObjects that are core to building Unity AR Foundation experiences. > #### Unity's explanation of ARSession > > An AR scene should include an `ARSession` component. The AR Session controls the lifecycle of an AR experience by enabling or disabling AR on the target platform. The `ARSession` can be on any `GameObject`. > > ARSession component > > When you disable the `ARSession`, the system no longer tracks features in its environment, but if you enable it at a later time, the system attempts to recover and maintain previously-detected features. > > If you enable the **Attempt Update** option, the device tries to install AR software if possible. Support for this feature is platform-dependent. > > **NOTE** > > An AR session is a global construct. An `ARSession` component manages this global session, so multiple `ARSession` components will all try to manage the same global session. > > **Checking for device support** > > Some platforms might support a limited subset of devices. On these platforms, your application needs to be able to detect support for AR Foundation so it can provide an alternative experience when AR is not supported. > > The `ARSession` component has a static coroutine that you can use to determine whether AR is supported at runtime: > > ```csharp > public class MyComponent { > [SerializeField] ARSession m_Session; > > IEnumerator Start() { > if ((ARSession.state == ARSessionState.None) || > (ARSession.state == ARSessionState.CheckingAvailability)) > { > yield return ARSession.CheckAvailability(); > } > > if (ARSession.state == ARSessionState.Unsupported) > { > // Start some fallback experience for unsupported devices > } > else > { > // Start the AR session > m_Session.enabled = true; > } > } > } > ``` > > **Session state** > > To determine the current state of the session (for example, whether the device is supported, if AR software is being installed, and whether the session is working), use `ARSession.state`. You can also subscribe to an event when the session state changes: `ARSession.stateChanged`. | `ARSessionState` | **Description** | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `None` | The AR System has not been initialized and availability is unknown. | | `Unsupported` | The current device doesn't support AR. | | `CheckingAvailability` | The system is checking the availability of AR on the current device. | | `NeedsInstall` | The current device supports AR, but AR support requires additional software to be installed. | | `Installing` | AR software is being installed. | | `Ready` | AR is supported and ready. | | `SessionInitialized` | An AR session is initializing (that is, starting up). This usually means AR is working, but hasn't gathered enough information about the environment. | | `SessionTracking` | An AR session is running and is tracking (that is, the device is able to determine its position and orientation in the world). | > #### AR Session Origin > > AR session origin > > The purpose of the `ARSessionOrigin` is to transform trackable features, such as planar surfaces and feature points, into their final position, orientation, and scale in the Unity scene. Because AR devices provide their data in "session space", which is an unscaled space relative to the beginning of the AR session, the `ARSessionOrigin` performs the appropriate transformation into Unity space. > > This concept is similar to the difference between "model" or "local" space and world space when working with other Assets in Unity. For instance, if you import a house asset from a DCC tool, the door's position is relative to the modeler's origin. This is commonly called "model space" or "local space". When Unity instantiates it, it also has a world space that's relative to Unity's origin. > > Likewise, trackables that an AR device produces, such as planes, are provided in "session space", relative to the device's coordinate system. When instantiated in Unity as `GameObject`s, they also have a world space. In order to instantiate them in the correct place, AR Foundation needs to know where the session origin should be in the Unity scene. > > `ARSessionOrigin` also allows you to scale virtual content and apply an offset to the AR Camera. If you're scaling or offsetting the `ARSessionOrigin`, then its AR Camera should be a child of the `ARSessionOrigin`. Because the AR Camera is session-driven, this setup allows the AR Camera and detected trackables to move together. > > \ > From [AR Foundation's About AR Foundation documentation](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/index.html) * Back in Unity, navigate to MainScene in your Scenes folder and open up MainScene * Right-click on the Hierarchy, select "XR" then AR Session * Right-click on the Hierarchy, select "XR" then AR Session Origin ![Creating AR Session and AR Session Origin](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSF_YYUwtDrmMkhiFK_%2F-MSFa0OOBBxLDuDuTGPl%2FScreen%20Recording%202021-01-29%20at%2006.53.42%20PM.gif?alt=media\&token=69445439-862e-4ae6-991f-28e35b073568) * Now let's create the script that will check for device support * If there is no AR support on a particular device attempting to use your app, then the AR Session component and the AR Session Origin GameObject will be disabled * Select the "AR Session" GameObject in Hierarchy * When AR Session is selected, go to the Inspector and click "Add Component" and create a new script called "ARPlatformInitializer" and click "Create and add" * Once created and added, drag the file into the Scripts and Prefabs/Mixed folder * Paste the code snippet below into ARPlatformInitializer.cs: ``` using System.Collections; using System.Collections.Generic; using UnityEngine; using UnityEngine.XR.ARFoundation; using Unity.Entities; using Unity.NetCode; public class ARPlatformInitializer : MonoBehaviour { [SerializeField] GameObject m_Session; [SerializeField] GameObject m_SessionOrigin; IEnumerator Start() { if ((ARSession.state == ARSessionState.None) || (ARSession.state == ARSessionState.CheckingAvailability)) { yield return ARSession.CheckAvailability(); } if (ARSession.state == ARSessionState.Unsupported) { //If we AR is unsupported we disable both GameObjects m_SessionOrigin.SetActive(false); m_Session.SetActive(false); } else { //If AR is supported we create our IsARPlayerComponent singleton in ClientWorld foreach (var world in World.All) { if (world.GetExistingSystem() != null) { world.EntityManager.CreateEntity(typeof(IsARPlayerComponent)); } } } } } ``` ![Creating ARPlatformInitializer ](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSF_YYUwtDrmMkhiFK_%2F-MSF_vy5U7guIlgX1xdi%2FScreen%20Recording%202021-01-29%20at%2006.59.21%20PM.gif?alt=media\&token=4ff10c0d-7baf-4a79-bfcc-fe5f8fc7292b) * Now let's create a new component (the component that is referenced in the script above, which is why you might have an error right now in Unity-- the component it references does not exist yet!) * Create a C# script named "IsARPlayerComponent" in the Client/Components folder * Paste the code snippet below into IsARPlayerComponent.cs: ``` using Unity.Entities; using UnityEngine; public struct IsARPlayerComponent : IComponentData { } ``` ![Create IsARPlayerComponet](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFaAUHLfzoqSRHUKj8%2F-MSFaY9kZNCl3rbfcnnl%2FARFoundation%20-%20Setting%20up%20-%20create%20IsARPlayerComponent.gif?alt=media\&token=14d98df3-f0e3-4f7a-a856-7d5e66915091) * Now select the "AR Session" GameObject in Hierarchy again and drag the ARSession and ARSessionOrigin GameObjects from the Hierarchy into the appropriate fields in the ARPlatformInitializer component in Inspector ![Setting the fields of ARPlatformInitializer](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFaaqv5jpTwrMtPVLH%2F-MSFasBr1-DymvmojeiU%2FARFoundation%20-%20Setting%20up%20-%20setting%20ARPlatformInitializer%20component%20fields.gif?alt=media\&token=f95f6df9-1a4c-49d6-a470-bc350ef58a00) * Let's go to NavigationScene, hit play, and host a game to make sure that are GameObjects are indeed disabled (by expanding MainScene in Hierarchy while in playmode) ![AR GameObjects are inactive in MainScene](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFaaqv5jpTwrMtPVLH%2F-MSFbQvF9s1zlq3Fxr1x%2FARFoundation%20-%20Setting%20up%20-%20hitting%20play%20to%20see%20ar%20GOs%20inactive.gif?alt=media\&token=ced04e3d-defc-4dc5-b5de-7798fa3c6670) {% hint style="success" %} We have set up our two core AR GameObjects * We added AR GameObjects to MainScene * AR Session * AR Session Origin * We created ARPlatformInitializer * We created IsARPlayerComponent {% endhint %} ## Configuring the AR Camera background using a Scriptable Render Pipeline This is a necessary step when using AR Foundation with the Universal Render Pipeline. We are going to be following the instructions here: * In Assets/ navigate to UniversalRenderPipelineAsset\_Renderer and in the Inspector Add Renderer Feature "AR Background Renderer Feature" ![Adding AR Background rendering to our URP Pipeline Asset](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fyn3LB1bSaeu6ffZnfKxk%2Fuploads%2FuTfybgf8cB9uUkzUfFHT%2Fimage.png?alt=media\&token=2a28945d-8915-4ef0-ad56-c140ca03f48c) {% hint style="success" %} We will now be able to render the AR Background * We updated our URP Pipeline Asset by adding a Renderer Feature for AR Backgrounds {% endhint %} ## iOS build configuration We are going to create an iOS build configuration in Assets/BuildSettings. * Navigate to BuildSettings, right-click, and select "Create", then, "Build", then, "Empty Build Configuration" and name it "iOS-Build" ![Creating iOS-Build](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFc09zYHaQkJhtNi1s%2F-MSFc7_d7Ml9JdDlZs5G%2FScreen%20Recording%202021-01-29%20at%2007.07.57%20PM.gif?alt=media\&token=381b38c3-9183-4b96-b3c3-002a5d74ef01) * Expand "Shared Configurations" at the top then click "+ Add Configuration" * Drag "BaseBuildConfiguration" into the field and then hit "Apply" ![Adding BaseBuildConfiguration as a Shared Configuration](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFc09zYHaQkJhtNi1s%2F-MSFcAhOB95nl8eHX5aw%2FScreen%20Recording%202021-01-29%20at%2007.08.29%20PM.gif?alt=media\&token=22199897-3612-4c32-8445-ce4f81b6b2a4) * Next on "Classic Scripting Settings" component * choose "IL2CPP" as Scripting Backend * choose "Debug" as IL2CPP Compiler Configuration * under add "Classic Build Profile" component, choose "iOS" from the Platform dropdown, hit "Apply", and then at the top of Inspector hit "Build and Run" * Sometimes you need to hit "Build and Run" twice as it fails the first time (sometimes even 3 times...) * "Editor's active Build Target needed to be switched to iOS..." ![Setting IL2CPP as Scripting Backend](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFcGfe10My96fkKUki%2F-MSFctu1WbrkuF2M9yLX%2FScreen%20Recording%202021-01-29%20at%2007.12.42%20PM.gif?alt=media\&token=40422936-2b46-4303-9966-79c8fb6598d6) * Next, in Xcode, make sure you set up your signing capabilities * Click on the "Unity-iPhone" project at on the left * Click on "Signing & Capabilities" * Make sure you are able to sign the application * Hit "Play" in XCode to deploy the application * you can either connect an iOS device to run your application or choose a simulated device as the destination ![Setting up Signing & Capabilities in Xcode and hitting Play to deploy the application to a connected device](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFdV1NEjgVBfbuz70x%2F-MSFie0eWxt4Kd2m1xWI%2FARFoundation%20-%20Setting%20up%20-%20Hitting%20Build%20in%20Xcode.gif?alt=media\&token=2361510c-f919-4384-b84b-13e7963e0aab) * Host a game ![Launching Asteroids project (now AR-enabled) on an iPhone](https://1184371945-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MPeyID8jdArWRKTskW0%2F-MSFnZ-A1r0ksjtgBMIL%2F-MSFo6Udm1IkHq0OA5Hb%2FARFoundation%20-%20Setting%20up%20-%20Deployed%20to%20iPhone.gif?alt=media\&token=29e28874-8d07-4324-88c0-306431b55220) The new iPhones eat up some top and side margins in the UI. We will update this in the final "UI Updates" section. {% hint style="success" %} We created an AR session on iOS devices * We created our iOS build configuration * Compiled our code to Xcode * Built our app in Xcode {% endhint %} **Github branch link:**‌ `git clone https://github.com/moetsi/Unity-DOTS-Multiplayer-XR-Sample/`\ `git checkout 'Setting-Up-AR-Foundation'`‌ {% hint style="info" %} [Join our Discord for more info](https://discord.com/invite/88j758eUvs) {% endhint %}