2D GUI

Babylon.js 2D GUI allows you to create and manage graphical user interfaces (GUIs) on top of your 3D scene. It provides a flexible system for adding interactive elements such as buttons, sliders, text, images, and other UI components to your scene, all rendered in 2D space. Unlike 3D objects in the scene, the 2D GUI elements remain fixed in the screen’s 2D plane, making them ideal for HUDs, menus, and other user interface elements that should overlay the 3D world. The GUI system in Babylon.js is built using dynamic textures and controls that you can customize to match your design needs. It allows you to add both simple static interfaces and complex interactive UI elements, with full support for animations, events, and styling.

There are two modes in Babylon 2D:

Fullscreen Mode

It refers to a type of UI layout where the GUI covers the entire screen, regardless of the 3D content in the scene. This mode is typically used when you want to create menus, HUDs (Heads-Up Displays), scoreboards, or other interfaces that should span the full screen, such as in game menus or overlays.

In Babylon.js, AdvancedDynamicTexture.CreateFullscreenUI is used to create a fullscreen user interface within a 3D scene. To simplify and make this process more flexible in a React-like context, Reactylon uses advancedDynamicTexture component and defines two additional props:

kind - it acts as identifier. When kind='createFullscreenUI' is provided, Reactylon will initialize the fullscreen UI creation logic.
createFullscreenUI - it abstracts the parameters of AdvancedDynamicTexture.CreateFullscreenUI into a more manageable object format.

createFullscreenUI: {
    name: Parameters<typeof AdvancedDynamicTexture.CreateFullscreenUI>[0];
    foreground?: Parameters<typeof AdvancedDynamicTexture.CreateFullscreenUI>[1];
    scene?: Parameters<typeof AdvancedDynamicTexture.CreateFullscreenUI>[2];
    sampling?: Parameters<typeof AdvancedDynamicTexture.CreateFullscreenUI>[3];
    adaptiveScaling?: Parameters<typeof AdvancedDynamicTexture.CreateFullscreenUI>[4];
};

In the following example, the advancedDynamicTexture, kind and createFullscreenUI are used to create a fullscreen UI that displays a green button at the center of the screen.

<>
    <sphere name='sphere' positionY={1} options={{ diameter: 2 }} />
    <ground name='ground' options={{ width: 6, height: 6 }} />
    <advancedDynamicTexture
        kind='createFullscreenUI'
        createFullscreenUI={{ name: 'createFullscreenUI-name' }} >
        <button2D
            name='btn-fullscreen'
            width={'20%'}
            height={'40px'}
            color='white'
            cornerRadius={20}
            background='green'
            onPointerClick={() => { console.log('clicked on me') }}>
            <textBlock name='btn-fullscreen-text' text='CLICK ME' />
        </button2D>
    </advancedDynamicTexture>
</>

Texture Mode

It refers to the way the GUI is rendered. In texture mode, the 2D GUI is rendered onto a dynamic texture that is applied to a mesh or used as part of the scene. This is especially useful when you want the GUI to be part of the 3D world, such as displaying UI elements on a screen, a 3D object, or a specific plane in the scene.

In Babylon.js, AdvancedDynamicTexture.CreateForMesh is used to create a dynamic texture for a specific mesh, allowing you to apply a UI to that mesh. To make this process more flexible and usable in a React-like context, Reactylon uses the advancedDynamicTexture component and defines two additional props:

kind - it acts as an identifier. When kind='createForMesh' is provided, Reactylon will initialize the mesh UI creation logic.
createForMesh - it abstracts the parameters of AdvancedDynamicTexture.CreateForMesh into a more manageable object format.

createForMesh: {
    mesh: string;
    width?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[1];
    height?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[2];
    supportPointerMove?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[3];
    onlyAlphaTesting?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[4];
    invertY?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[5];
    materialSetupCallback?: Parameters<typeof AdvancedDynamicTexture.CreateForMesh>[6];
};

In the following example, the advancedDynamicTexture, kind and createForMesh are used to create a green button on a sphere.

<>
    <sphere name='sphere' options={{ segments: 16, diameter: 2 }}>
        <plane name='plane' options={{ size: 4 }} positionY={2} />
    </sphere>
    <advancedDynamicTexture kind='createForMesh' createForMesh={{ mesh: 'plane' }}>
        <button2D
            name='btn-for-mesh'
            width={1}
            height={0.4}
            color='white'
            background='green'
            onPointerClick={() => {
                console.log('clicked on me');
            }}
            fontSize={50}>
            <textBlock name='btn-for-mesh-text' text='CLICK ME' />
        </button2D>
    </advancedDynamicTexture>
</>

Controls

In Babylon.js, Controls are UI elements that allow you to interact with the user through a graphical interface. These controls are part of Babylon.js GUI, which provides a variety of pre-built UI elements that can be used in 3D scenes, including buttons, sliders, text boxes, images, and more. Controls can be grouped by:

Basic controls like buttons, text blocks, sliders, images, and input fields.
Container controls like the StackPanel, a layout container that arranges its child controls vertically or horizontally, one after another, like a stack.
Custom controls: you can also create custom controls by extending existing ones or implementing new ones.

These controls are typically added to an advancedDynamicTexture, which acts as the canvas where UI elements are drawn.

The following example demonstrates a few fundamental controls available in Babylon.js. For a comprehensive list of all controls and their usage, refer to the official Babylon.js documentation.

<advancedDynamicTexture
    kind='createFullscreenUI'
    createFullscreenUI={{ name: 'createFullscreenUI-name' }} >
    <stackPanel width={'200px'} horizontalAlignment={Control.HORIZONTAL_ALIGNMENT_RIGHT} verticalAlignment={Control.VERTICAL_ALIGNMENT_TOP} paddingTop={10} spacing={10}>
        <inputText height={'40px'} text='Input text...' color='white' horizontalAlignment={Control.HORIZONTAL_ALIGNMENT_RIGHT} />
        <inputTextArea
            name='text-area'
            height={'200px'}
            width={'200px'}
            text={`This is a text area. \nIt's quite long, so will need to be displayed across multiple lines. \n\nIt also has several line breaks.`}
            color='white'
        />
        <stackPanel height={'20px'} isVertical={false}>
            <checkbox width={'20px'} height={'20px'} color='green' />
            <textBlock text='checkbox' width={'150px'} color='white' textHorizontalAlignment={Control.HORIZONTAL_ALIGNMENT_LEFT} />
        </stackPanel>
        <stackPanel height={'20px'} isVertical={false}>
            <radioButton width={'20px'} height={'20px'} color='white' background='green' />
            <textBlock text='radiobutton' width={'150px'} color='white' textHorizontalAlignment={Control.HORIZONTAL_ALIGNMENT_LEFT} />
        </stackPanel>
        <stackPanel height={'50px'}>
            <textBlock text='slider' height={'20px'} width={'150px'} color='white' textHorizontalAlignment={Control.HORIZONTAL_ALIGNMENT_LEFT} />
            <slider value={0} minimum={0} maximum={20} width={'200px'} height={'20px'} />
        </stackPanel>
    </stackPanel>
</advancedDynamicTexture>

Callout section for .addControl and .removeControl

Learn More

For other controls, properties and customizations, please refer to Babylon.js documentation: https://doc.babylonjs.com/features/featuresDeepDive/gui/gui/.

Reflection and Refractions 3D GUI