ES h264/h265 decoding

I don't expect anyone to just write out the code for me, just looking for pointers in the right direction.

I get elementary streams from the network and have them in memory. Typically I'm using VideoCodec and VTDecompressionSessions to decode frames, and I'd like to use a library that's a little higher level and easier to maintain in my overall project which the decoder is just a small part.

For example:

• H264 Nalus frame by frame over websockets. I also have a custom header with resolution, framecount, timestamp, few other things...
• usually a base or main H264, never any prediction frames, YUV420 planar.

In the documentation for LibVLC itself I see callbacks for when a frame is ready to display. Are there ways to copy data in and out in a low latency fashion?

If I can figure this out on iOS/Android I'd be more than willing to help document and provide what I can.