Lots of new Layer features;

[bjornbytes]

• Add Layer:setCurve for curved layers. Curved. Layers.
• Add new sphere and cube layer types (existing layers are quad layers now). cube layers are cubemap textures, and sphere layers are equirectangular images. These are mainly intended to be used for backgrounds that have higher sampling quality, more stable tracking, and don't need to be re-rendered every frame. Currently sphere/cube layers always render underneath the main layer.
• Remove Layer:get/setViewMask. Instead, there's a stereo flag in newLayer, which will create an array texture and put each layer in the corresponding eye, just like how the main layer works. This works for all layer types, so you can have stereo cubemaps and stuff.
• lovr.headset.setLayers takes a main key that can be used to disable the main layer, which can be useful for overlay apps that don't need to render any 3D content.
• There is a new static flag in newLayer which can be used for static content. The layer can only be changed once, but it uses a lot less VRAM.
• There is a new transparent flag in newLayer. Disabling transparency may improve performance slightly.
• Add Layer:get/setColor, so you can change a layer's color or fade it in/out without rerendering it.
• You can initialize a layer from an Image/Texture/filename, or a table of images/filenames.
• New HeadsetFeatures: layerCube, layerSphere, layerCurve.
• Default errhand disables layers now.

[bjornbytes]

• Should cube/sphere layers be named cubemap and equirect instead? It's more clear but less cute.
• Should transparent default to true or false?
• Premultiplied alpha is not working right. Also, does it need to be configurable?

[bjornbytes]

• Could add Layer:get/setColor via XR_KHR_composition_layer_color_scale_bias.
• Could expose sampler settings for layers via XR_FB_swapchain_update_state_vulkan, but it's kinda niche, may want to wait until someone needs it. Same for mipmaps I guess.

[jmiskovic]

Why differentiate between cube and sphere layer type in API, is there anything user-facing that makes it worth to expose it? I imagine I would expect to pass in the equirect or cubemap texture and the layer would internally figure it out.

Would it be possible to fetch the underlying Layer texture after the init? Not really that important but nice to have.

[bjornbytes]

LayerType needs to be exposed because if you create a Layer from a 2D texture, lovr doesn't know if you want that to be a quad or a sphere. And if you create an empty layer with a width/height, it isn't clear what type of layer that should be. There's already some auto-detect logic for textures to make it easier, though. For example, you can do lovr.headset.newLayer(cubemapTexture) and it will make a cube layer.

Regarding the texture: Yes, you can call Layer:getTexture to get the texture, but note that it's backed by a swapchain (ringbuffer of textures) so it returns a new (blank) texture each frame, expecting you to render to it. Layer:getPass is also a convenience function that will return a Pass that renders to the Layer.

[bjornbytes]

I'm actually trying out a simpler API on the background-layers branch where there is just a lovr.headset.setBackground(Image|Texture) function used to set a single cube/sphere background layer, and the Layer object is just used for quad panels now. It means that background layers are a little bit less flexible, but it simplifies the Layer object immensely.