MANYFOLD_mesh_progressive extension proposal

[Floppy]

This is a proposed extension for creating streamable meshes, using a progressive mesh approach. It achieves this by defining a series of "vertex splits" in the buffer, which, when applied correctly to a base mesh stored in the normal way, allow the progressive refinement of the mesh. In this way, the base mesh can be rendered as soon as its buffers are complete, and the vertex split buffer can be applied as it is loaded, most likely over a network.

The proposal is ready for feedback, though I've got a few things to round out before it's completely finished. I'm opening it now because I've just finished the file writing half of a reference implementation, which is linked in the proposal itself, so I'm reasonably confident that the proposal makes sense.

All feedback appreciated!

[javagl]

Hopefully it's OK to tag @mlimper here, referring to "SRC" (which is literally impossible to sensibly search for 😆 - maybe you have comments or pointers)

[mlimper]

Thanks everyone : )

@javagl It's true, we produced an extension proposal and wrote about it in a paper 10 years ago :-D

The "SRC" paper and talk recording are still available here (and in the ACM digital library):
https://max-limper.de/publications.html

We had proposed this as a gltf extension too - knowing about the "classic" progressive meshes approach, but then still deciding against it. Back then, the following considerations were relevant:

• Progressive meshes only support manifold geometry - we wanted something that works with any 3D mesh data (triangle soup). We ended using our own POP buffers method, so the paper can also be seen as an attempt to show how this can be integrated into standards (both X3D and glTF) with the aim of minimal overhead.
• The other issue we saw with progressive meshes was significant decode times. We wanted to keep that elegance of glTF that allows the client to just push a binary buffer [chunk] to the GPU and render it (being "ready to render"). We also wanted it to be backwards compatible for implementations that just want to load it all in one go - and therefore all we changed was proposing the concept of "buffer chunks". They would just be marking parts of a buffer that make sense to load progressively, but they don't fundamentally change how data is loaded to the GPU etc.. When doing progressive streaming, chunks are simply appended to the existing data OR can be all loaded in one piece if the implementation wants to keep it simple. That, however, it a very specific property of the POP buffer method and doesn't work with any other method, at least to the best of my knowledge.

I also heard engineers implementing a gltf workflow at a big company say that they want to avoid Draco because they think it adds too much decode overhead. In that sense, it seems worth evaluating decode times for different devices for the progressive meshes approach.

So IMHO the most important considerations should be:

• What happens if parts of the data aren't guaranteed to be manifold meshes, can those ignore the extension and just be loaded in the old fashioned way (in one piece), without additional decoding?

• This extension will probably not be able to guarantee that implementations that don't support it can still visualize the asset - which is tricky (it'd be the same argument in case of Draco though).

• The asset won't be "ready to render", that has to be evaluated and decode times taken into account.

But that's really just my two cents here, just wanted to share that since I was mentioned and during the work we did 10 years ago we thought about similar topics, so just wanted to share some thoughts and the paper/videos we produced back then :-)

[Floppy]

Thanks for the comments! I will have a look at SRC, for sure. I first came across progressive meshes in about 2000, while doing my PhD, so this is also a blast from the past for me too! 😁

That's reminded me, I meant to ping @hhoppe for visibility on this as well.

It definitely won't be appropriate for all situations, I agree. The geometry must be manifold as you say, and it may take a long while to encode and decode, so certainly if a user is looking at realtime applications and just dropping into a render buffer, it's not for them. However, for progressive display of models over a network, I think it will work much better than the alternative of waiting for a 100M file to download in full!

BTW regarding manifold meshes: I think I've got an implementation that takes account of at least some non-manifold models, but that's all in the creation of the vertex split stream, which is intentionally left outside the scope of the proposal.

Regarding compatibility: the way I've written it, anything that doesn't support the extension should be able to display the base mesh, which is encoded in the normal way - the examples I've tried load up the low-res mesh in F3D, for instance, with no changes. In fact, I see that as a potential feature; one possible application of this is for previewing commercial models, where you perhaps get sent the base mesh but only part (or none) of the progressive stream that gives you full detail.

For context, my application is a web-based viewer for 3d print models (https://manyfold.app), which don't need to be realtime, but can be very large. Ideally, also, they're manifold (though definitely aren't always).