Babylon.js

Babylon.js and uno UI can draw to the same canvas by sharing a WebGPU device and context. The order of the draw calls determines whether the UI appears in front of or behind the scene.

The examples below assume you are familiar with the basics of uno UI covered in the First Layout section.

Shared setup

Use WebGPUEngine from @babylonjs/core. Initialize the engine first, then pass its WebGPU device and the canvas context to uno UI.

import ResourcesWebGPU from '@uno.build/ui/ResourcesWebGPU'
import UI from '@uno.build/ui/UI'
import { WebGPUEngine } from '@babylonjs/core/Engines/webgpuEngine.js'
import { Color4 } from '@babylonjs/core/Maths/math.color.js'
import { Scene } from '@babylonjs/core/scene.js'

const canvas = document.querySelector('canvas')
const context = canvas.getContext('webgpu')
const format = navigator.gpu.getPreferredCanvasFormat()
const engine = new WebGPUEngine(canvas, {
  antialias: false,
  audioEngine: false,
  premultipliedAlpha: true,
  doNotHandleTouchAction: true,
  swapChainFormat: format,
})
await engine.initAsync()

const resources = await ResourcesWebGPU.create({
  canvas,
  device: engine._device,
  context,
  format,
})
const { ui } = await UI.create({ resources })

const scene = new Scene(engine)
scene.preventDefaultOnPointerDown = false
scene.preventDefaultOnPointerUp = false
scene.clearColor = new Color4(1, 1, 1, 1)
scene.imageProcessingConfiguration.isEnabled = false

Keep antialias: false for this direct canvas composition so Babylon.js draws into the shared canvas texture without resolving a separate multisampled color buffer over the background UI.

Control the draw order

Clear the color at the start of each frame, then preserve it in subsequent draws.

uno UI: load_op

The load_op option of ui.draw() controls what happens to the existing pixels before drawing the UI:

load_op does not choose which layer is in front. The draw order does that. Using 'clear' after drawing the scene erases the scene; using 'load' keeps it visible wherever the UI is transparent.

Babylon.js: autoClear

scene.autoClear controls whether Babylon.js clears the color before rendering the scene:

Keep scene.autoClearDepthAndStencil at its default value of true. Changing only autoClear lets Babylon.js continue clearing depth and stencil for each frame.

Babylon.js normally submits its WebGPU commands at the end of each frame, after the callback passed to engine.runRenderLoop() returns. When drawing uno UI after scene.render(), call engine.flushFramebuffer() between them to submit the scene before the UI.

UI in front of the scene

Draw the scene first, submit its commands, then draw the UI:

scene.autoClear = true

engine.runRenderLoop(() => {
  const time = performance.now()
  cube.rotation.x = time / 2000
  cube.rotation.y = time / 1000

  scene.render()
  engine.flushFramebuffer()
  ui.draw()
})

Babylon.js clears to white and draws the cube. uno UI preserves that image and draws the panel over it.

UI behind the scene

Draw the UI first with load_op: 'clear', then render the scene with autoClear = false:

scene.autoClear = false

// uno UI now paints the background, since Babylon.js will not clear the color.
ui.root.style('backgroundColor', '#ffffff')
ui.update()

engine.runRenderLoop(() => {
  const time = performance.now()
  cube.rotation.x = time / 2000
  cube.rotation.y = time / 1000

  ui.draw({ load_op: 'clear' })
  scene.render()
})

uno UI clears the canvas and draws the white background and panel. Babylon.js keeps those pixels and draws the cube over them. Leaving autoClear = true would erase the UI before drawing the cube.

A skybox or fullscreen post-process drawn by Babylon.js can cover the UI even when color clearing is disabled.

UI behind and in front

For separate background and foreground layouts, create two UI instances using the same resources:

const { ui: background_ui } = await UI.create({ resources })
const { ui: foreground_ui } = await UI.create({ resources })

Apply the layout and viewport setup to each UI, then draw the background UI, the scene, and the foreground UI in that order:

scene.autoClear = false

engine.runRenderLoop(() => {
  const time = performance.now()
  cube.rotation.x = time / 2000
  cube.rotation.y = time / 1000

  background_ui.draw({ load_op: 'clear' })
  scene.render()
  engine.flushFramebuffer()
  foreground_ui.draw()
})

The background UI starts the frame, Babylon.js draws over it, and the foreground UI draws last. Keep the foreground layout transparent wherever the scene should remain visible.

Demo

This demo uses two UI instances: one for the background and one for the foreground. The foreground UI reuses the React component from the React demo.