Update UI using AR Foundation

Code and workflows to optimize and update the UI in our AR enabled project

What you'll develop on this page

We will update the UI to both fit better and to dynamically update the instructions shown to AR players.

Github branch link: https://github.com/moetsi/Unity-DOTS-Multiplayer-XR-Sample/tree/UI-Updates

Updating UI

Updating the instructions

We are going to update the instructions on the bottom-left of the screen in ARPlatformInitializer.

We are going to grab each of the instructions and update it with new text once we know that the project is running on an AR platform.

  • Paste the code snippet below into ARPlatformInitializer.cs:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using Unity.Entities;
using Unity.NetCode;
using UnityEngine.UIElements;

public class ARPlatformInitializer : MonoBehaviour
{
    [SerializeField] GameObject m_Session;
    [SerializeField] GameObject m_SessionOrigin;

    //This is how we will grab access to the UI elements we need to update
    public UIDocument m_GameUIDocument;
    private VisualElement m_GameManagerUIVE;
    private VisualElement m_BottomLeft;
    private Label m_1stInstruction;
    private Label m_2ndInstruction;
    private Label m_3rdInstruction;
    private Label m_4thInstruction;

    void OnEnable()
    {
        //We set the labels that we will need to update
        m_GameManagerUIVE = m_GameUIDocument.rootVisualElement;
        m_BottomLeft = m_GameManagerUIVE.Q<VisualElement>("bottom-left");
        m_4thInstruction = m_GameManagerUIVE.Q<Label>("instructions-4");
        m_3rdInstruction = m_GameManagerUIVE.Q<Label>("instructions-3");
        m_2ndInstruction = m_GameManagerUIVE.Q<Label>("instructions-2");
        m_1stInstruction = m_GameManagerUIVE.Q<Label>("instructions-1");
    }

    IEnumerator Start() {
        if ((ARSession.state == ARSessionState.None) ||
            (ARSession.state == ARSessionState.CheckingAvailability))
        {
            yield return ARSession.CheckAvailability();
        }

        if (ARSession.state == ARSessionState.Unsupported)
        {
            //If we AR is unsupported we disable both GameObjects
            m_SessionOrigin.SetActive(false);
            m_Session.SetActive(false);
        }
        else
        {
            //If AR is supported we create our IsARPlayerComponent singleton in ClientWorld
            foreach (var world in World.All)
            {
                if (world.GetExistingSystem<ClientSimulationSystemGroup>() != null)
                {
                    world.EntityManager.CreateEntity(typeof(IsARPlayerComponent));
                }
            }

            //Due to a UI Toolkit bug we cannot currently update existing labels without getting wacky behavior
            // https://forum.unity.com/threads/updating-labels-causes-a-gap-on-first-ui-view.1049081/
            //So we will remove all labels and attach new ones to our bottom left container
            m_BottomLeft.Remove(m_4thInstruction);
            m_BottomLeft.Remove(m_3rdInstruction);
            m_BottomLeft.Remove(m_2ndInstruction);
            m_BottomLeft.Remove(m_1stInstruction);

            //Now that our container is empty we make 3 new labels for our 3 new instructions
            Label instruction1 = new Label();
            Label instruction2 = new Label();
            Label instruction3 = new Label();

            //Now we add our instruction-text class to our labels so they have the same styling as before
            instruction1.AddToClassList("instruction-text");
            instruction2.AddToClassList("instruction-text");
            instruction3.AddToClassList("instruction-text");

            //Now we update the instruction text in the labels
            instruction3.text = "Tap with 1 finger to spawn and shoot";
            instruction2.text = "Move device to move player";
            instruction1.text = "Tap with 3 fingers to self-destruct";

            //Because we flex grow upwards we start with the bottom instruction (instruction 1) and then add the rest
            m_BottomLeft.Add(instruction1);
            m_BottomLeft.Add(instruction2);
            m_BottomLeft.Add(instruction3);
        }
    }
}
  • In MainScene drag the GameUI GameObject from Hierarchy into the Game UI Document field within the AR Platform Initializer component in Inspector when AR Session is selected in Hierarchy

  • Go to the BuildSettings folder, select iOS-Build, then click "Build and Run" in Inspector

  • Then launch the app

We now have dynamic Game UI based on platform type

  • We updated ARPlatformInitializer

Updating the styling

We are going to update the styling to update the padding in the footer and header. Currently, the curve and notch in newer iPhones cover up quite a bit at the top and bottom of the screen. Although adjusting the styling to account for newer iPhones is not required (well, none of this is), we are explaining this just to make your project a little easier to navigate. We will also update our logo to use an SVG.

  • Add the SVG file to the UI folder

    • With your SVG selected go to the Inspector and select

      • "UI Toolkit Vector Image" in the drop-down list for Generated Asset Type

      • Target Resolution 2160p

      • Click "Apply"

  • Paste the code snippet below into TitleScreenUI USS:

.screen {
    flex-grow: 1;
    font-size: 20px;
    align-items: stretch;
    background-color: rgb(255, 255, 255);
}
.quit-button {
    margin-right: 10px;
    margin-left: 0;
    width: 120px;
    height: 68px;
    font-size: 24px;
    color: rgb(0, 0, 0);
    background-color: rgba(0, 0, 0, 0);
    border-left-color: rgb(0, 0, 0);
    border-right-color: rgb(0, 0, 0);
    border-top-color: rgb(0, 0, 0);
    border-bottom-color: rgb(0, 0, 0);
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    border-left-width: 3px;
    border-right-width: 3px;
    border-top-width: 3px;
    border-bottom-width: 3px;
}
.quit-button:hover {
    background-color: rgba(0, 0, 0, 0);
    border-top-left-radius: 9px;
    border-bottom-left-radius: 9px;
    border-top-right-radius: 9px;
    border-bottom-right-radius: 9px;
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
}
.quit-button:active {
    background-color: rgb(0, 0, 0);
    color: rgb(255, 255, 255);
}
.main-menu-button {
    width: 219px;
    margin-left: 10px;
    margin-right: 0;
    margin-top: 0;
    margin-bottom: 0;
}
.header {
    position: absolute;
    flex-direction: row;
    justify-content: space-between;
    flex-grow: 1;
    top: 0;
    width: 100%;
    align-items: flex-start;
    padding-top: 85px;
}
.main-content {
    position: absolute;
    top: 108px;
    left: auto;
    right: auto;
    bottom: auto;
    max-width: 550px;
    width: 100%;
    height: auto;
    align-items: center;
    padding-left: 20px;
    padding-right: 20px;
    -unity-font: url('/Assets/UI/Fonts/HV.ttf');
    color: rgb(0, 0, 0);
}
.title {
    font-size: 64px;
    margin-top: 100px;
    -unity-font: url('/Assets/UI/Fonts/Cervo.otf');
    color: rgb(0, 0, 0);
}
.section-title-container {
    width: 100%;
    margin-top: 100px;
}
.section-title {
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    font-size: 36px;
    color: rgb(0, 0, 0);
}
.blue-button {
    width: 100%;
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    margin-left: 0;
    margin-right: 0;
    margin-top: 21px;
    margin-bottom: 0;
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    height: 68px;
    border-left-color: rgb(150, 191, 208);
    border-right-color: rgb(150, 191, 208);
    border-top-color: rgb(150, 191, 208);
    border-bottom-color: rgb(150, 191, 208);
    background-color: rgba(0, 0, 0, 0);
    font-size: 24px;
    color: rgb(150, 191, 208);
    flex-wrap: wrap;
    white-space: normal;
}
.blue-button:hover {
    border-left-width: 7px;
    border-right-width: 7px;
    border-top-width: 7px;
    border-bottom-width: 7px;
    background-color: rgba(0, 0, 0, 0);
    border-top-left-radius: 9px;
    border-bottom-left-radius: 9px;
    border-top-right-radius: 9px;
    border-bottom-right-radius: 9px;
}
.blue-button:active {
    background-color: rgb(150, 191, 208);
    color: rgb(255, 255, 255);
}
.green-button {
    left: auto;
    right: auto;
    margin-left: 0;
    margin-right: 0;
    margin-top: 36px;
    margin-bottom: 0;
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    width: 100%;
    height: 120px;
    color: rgb(160, 194, 114);
    font-size: 36px;
    background-color: rgba(0, 0, 0, 0);
    border-left-color: rgb(160, 194, 114);
    border-right-color: rgb(160, 194, 114);
    border-top-color: rgb(160, 194, 114);
    border-bottom-color: rgb(160, 194, 114);
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    flex-wrap: wrap;
    white-space: normal;
}
.green-button:hover {
    border-top-left-radius: 9px;
    border-bottom-left-radius: 9px;
    border-top-right-radius: 9px;
    border-bottom-right-radius: 9px;
    border-left-width: 7px;
    border-right-width: 7px;
    border-top-width: 7px;
    border-bottom-width: 7px;
    background-color: rgba(0, 0, 0, 0);
}
.green-button:active {
    background-color: rgb(160, 194, 114);
    color: rgb(255, 255, 255);
}
.data-section {
    width: 100%;
    background-color: rgb(255, 255, 255);
}
.data-section-input {
    flex-direction: column-reverse;
    height: 49px;
    margin-left: 0;
    margin-right: 0;
    margin-top: 36px;
    margin-bottom: 0;
    font-size: 36px;
    color: rgb(160, 194, 114);
    border-bottom-width: 4px;
    border-bottom-color: rgb(160, 194, 114);
    border-top-color: rgb(160, 194, 114);
    border-left-color: rgb(160, 194, 114);
    border-right-color: rgb(160, 194, 114);
    width: auto;
    background-color: rgba(0, 0, 0, 0);
}
.data-section-label {
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    color: rgb(160, 194, 114);
}
.unity-base-field {
}
.screen-scroll-container {
    flex-grow: 1;
    background-color: rgb(255, 255, 255);
}
.quit-game-button {
    flex-direction: column-reverse;
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    margin-left: 10px;
    margin-right: 0;
    margin-top: 0;
    margin-bottom: 0;
    background-color: rgba(0, 0, 0, 0);
    border-left-width: 3px;
    border-right-width: 3px;
    border-top-width: 3px;
    border-bottom-width: 3px;
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    width: 219px;
    height: 68px;
    border-left-color: rgb(255, 255, 255);
    border-right-color: rgb(255, 255, 255);
    border-top-color: rgb(255, 255, 255);
    border-bottom-color: rgb(255, 255, 255);
    font-size: 24px;
    color: rgb(255, 255, 255);
    white-space: normal;
}
.quit-game-button:hover {
    border-top-left-radius: 9px;
    border-bottom-left-radius: 9px;
    border-top-right-radius: 9px;
    border-bottom-right-radius: 9px;
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
}
.quit-game-button:active {
    background-color: rgb(255, 255, 255);
    color: rgb(0, 0, 0);
}
.logo {
    flex-grow: 1;
    max-width: 300px;
    height: 68px;
    margin-left: 10px;
    background-image: url('/Assets/UI/Moetsi Logo SVG.svg');
    -unity-background-scale-mode: scale-to-fit;
    padding-top: 10px;
}
.local-games-list-container {
    align-items: center;
    justify-content: center;
    width: 100%;
    height: 200px;
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    border-left-color: rgb(0, 0, 0);
    border-right-color: rgb(0, 0, 0);
    border-top-color: rgb(0, 0, 0);
    border-bottom-color: rgb(0, 0, 0);
}
.local-games-list {
    width: 100%;
    height: 100%;
}
.or {
    color: rgb(0, 0, 0);
    margin-top: 20px;
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    font-size: 36px;
}
.HostGameScreen {
    align-items: center;
}
.JoinGameScreen {
    align-items: center;
}
.ManualConnectScreen {
    align-items: center;
}
.row {
    flex-direction: row;
    justify-content: space-between;
    align-items: center;
    height: 74px;
}
.game-name-data {
    margin-left: 17px;
}
.list-item-game-name {
    margin-left: 0;
    font-size: 24px;
    margin-right: 0;
    margin-top: 0;
    margin-bottom: 0;
    color: rgb(150, 191, 208);
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
}
.list-item-game-name-label {
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    font-size: 10px;
}
.list-item-button {
    width: 141px;
    height: 46px;
    margin-left: 0;
    margin-right: 12px;
    margin-top: 0;
    margin-bottom: 0;
    border-left-color: rgba(0, 0, 0, 0);
    border-right-color: rgba(0, 0, 0, 0);
    border-top-color: rgba(0, 0, 0, 0);
    border-bottom-color: rgba(0, 0, 0, 0);
}

If you want to use a different SVG (not the Moetsi logo SVG), you will need to update the name of the SVG in the .logo class.

  • Paste the code snippet below into GameUI USS:

.quit-game-button:hover {
    border-top-left-radius: 9px;
    border-bottom-left-radius: 9px;
    border-top-right-radius: 9px;
    border-bottom-right-radius: 9px;
    border-left-width: 5px;
    border-right-width: 5px;
    border-top-width: 5px;
    border-bottom-width: 5px;
}
.quit-game-button:active {
    background-color: rgb(255, 255, 255);
    color: rgb(0, 0, 0);
}
.quit-game-button {
    flex-direction: column-reverse;
    padding-left: 0;
    padding-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    margin-left: 10px;
    margin-right: 0;
    margin-top: 0;
    margin-bottom: 0;
    background-color: rgba(0, 0, 0, 0);
    border-left-width: 3px;
    border-right-width: 3px;
    border-top-width: 3px;
    border-bottom-width: 3px;
    border-top-left-radius: 10px;
    border-bottom-left-radius: 10px;
    border-top-right-radius: 10px;
    border-bottom-right-radius: 10px;
    height: 68px;
    border-left-color: rgb(255, 255, 255);
    border-right-color: rgb(255, 255, 255);
    border-top-color: rgb(255, 255, 255);
    border-bottom-color: rgb(255, 255, 255);
    font-size: 24px;
    color: rgb(255, 255, 255);
    white-space: normal;
    max-width: 140px;
    width: 100%;
}
.game-ui-screen {
    background-color: rgba(0, 0, 0, 0);
    justify-content: space-between;
    color: rgb(255, 255, 255);
}
.game-ui-header {
    height: auto;
    flex-direction: row;
    align-items: flex-start;
}
.top-right-container {
    margin-right: 10px;
}
.top-right-values {
    color: rgb(255, 255, 255);
    -unity-text-align: upper-right;
    font-size: 18px;
}
.top-right-labels {
    -unity-text-align: upper-right;
    color: rgb(255, 255, 255);
    font-size: 10px;
}
.spacers {
    height: 3px;
}
.footer {
    bottom: 0;
    flex-direction: row;
    justify-content: space-between;
    position: absolute;
    flex-grow: 1;
    width: 100%;
    height: auto;
    padding-bottom: 45px;
}
.bottom-left {
    flex-direction: column-reverse;
    margin-left: 10px;
}
.instruction-text {
    color: rgb(255, 255, 255);
    white-space: normal;
    font-size: 18px;
}

Let's also update our PanelSettings (in the UI folder) to better handle mobile and desktop.

  • In Inspector, make the following updates to PanelSettings:

    • Scale Mode = Scale With Screen Size

    • Screen Match Mode = Match Width Or Height

    • Reference Resolution

      • X = 1000

      • Y = 1200

    • Screen Match Mode Parameters

      • Height = 1

Our UI dynamically updates when we change our Scale Mode to "scale with screen size." We choose "height" as the main driver because we have a more vertical UI in our app. If it is wider, it should not affect the scale of our UI, but if it is taller or shorter we would want our UI to scale.

  • Now build for iOS (clicking "Build and Run" in Inspector when iOS-Build is selected from the BuildSettings folder) and take a look at our new UI updates

We now have a more desktop/mobile-friendly UI

  • We imported an SVG

  • We updated TitleScreen USS

    • And updated the logo class to use the imported SVG name if necessary

  • We updated GameUI USS

  • We updated PanelSettings

Github branch link:

git clone https://github.com/moetsi/Unity-DOTS-Multiplayer-XR-Sample/ git checkout 'UI-Updates'

That's all folks!

Hopefully this gitbook was helpful to other XR developers. We plan on keeping this gitbook updates as packages and Editors are updated.

Please reach out on Discord if you have any questions or if you would like some additional sections to cover other topics.

Last updated