Add OpenGL texture interop

This means importing/exporting buffers, textures and (if necessary) sync objects.

Status:

• texture interop (done)
• buffer interop
• sync object interop? (probably not needed)

added enhancement good first issue help wanted opengl labels

There are different ways of doing this depending on the hardware and drivers in use (of course). For dma_bufs, there's an EGL based mechanism, and for nvidia drivers, there's a CUDA mechanism that is partially common with the Vulkan mechanism.

Due to the lack of standardisation in OpenGL, the primitives that need to be passed around are not abstracted. For the nvidia/cudu mechanism, we'd need to literally 'export' the opengl textures by passing the pointers/IDs/whatever you call them.

For the dma_buf method, you'd either import the dma_buf as an fd and then turn that into an EGL image or import the EGL image itself. There are pros and cons to each method.

In principle, much of this is already done in mpv, which can be used for guidance.

Neither API has explicit sync objects, so at least that part is simple.

changed milestone to %v2.66+

removed milestone

mentioned in commit 00ddb268

mentioned in commit vliaskov/libplacebo@55129b17

mentioned in commit vliaskov/libplacebo@406a8e92

Hi! I am interested in mpv/libplacebo. As a first issue, I have worked a bit on this:

• An egl+dmabuf prototype: An EGLImage is created from the texture to be exported, and than exported into a dmabuf fd. THe importer creates a new EGLImage from the dmabuf fd, and then a new texture from the EGLImage using glEGLIglEGLImageTargetTexture2DOES. PL_HANDLE_DMA_BUF is used.

• An egl only prototype: An EGLImage is created from the texture to be exported, and the importer uses glEGLIglEGLImageTargetTexture2DOES to create a new texture from the exported EGLImage. A new handle type PL_HANDLE_EGL_IMAGE is introduced (though it is not completely handled yet in all parts of the code)

Are either one of these preferred? Should both of these be made available?

The commits also include a new testcase for opengl testing this, but I am sure it can be improved or cover more cases.

I can make a cleaner WIP MR (with either one or both of these implementations) if that would simplify review and feedback.

That's cool, thanks for looking into this!

Are either one of these preferred? Should both of these be made available?

I'm not sure I fully understand the distinction, but re-using PL_HANDLE_DMA_BUF makes more sense to me because dmabufs are already the "cross-API" compatible universal interface for this sort of thing. There shouldn't be a need for user code to distinguish between GL and Vulkan when using the pl_gpu API.

I can make a cleaner WIP MR (with either one or both of these implementations) if that would simplify review and feedback.

Yes, please do. I'll provide comments in that MR.

mentioned in merge request !116 (closed)

mentioned in commit vliaskov/libplacebo@708274c0

changed the description

I think this can be closed. There's not a whole lot that still needs to be done, since textures are generally more useful in OpenGL than buffers.

closed