Feature/138

[kekyo]

No description provided.

[kekyo]

@RyanYANG52 I have made some changes, so please try it in your environment.

Buffer pooling is done in the BufferPool class. A simplified implementation of this class is the DefaultBufferPool class, which is used by default. If you want to use your own buffer pooling, implement it by inheriting from the BufferPool class and passing it to CameraDevices constructor as follows:

var cameraDevices = new CameraDevices(new YourOwnBufferPool());

// ...

• Or FrameProcessor constructor can receive custom BufferPool instance.

[kekyo]

@RyanYANG52 Hmm, the last commit contains a bug, I forgot calling pool.Return(). However, modifying this may cause the CLR to crash. So I would like you to evaluate e7f4c5e which does not include buffer pooling.

As a matter of fact, I think that in a situation where PixelBuffer is recycled correctly, the buffer will be mostly reused after a few hundred generation. How about you?

[RyanYANG52]

e7f4c5e works, in my case after 1-2 second it can be reused for most of frames, I will look into the BufferPool impl in weekend.

[RyanYANG52]

I look into the BufferPool feature, for my use case (no transcode) my camera jpeg size increase from 13k to 30k about 30 frame after initial start, DefaultBufferPool may not be suitable. However with a BufferPool abstraction, I could create my impl like you said. Maybe just one byte array as cache, if size is not enough then create a new array with double capacity , like List does inside.
I only use referImage and no transcode YUV, but if I understand correctly, for YUV image and it's decoded image, it will be the same size for every frame. with PixelBuffer being reused correctly now, maybe BufferPool will not be much of use for YUV.

[kekyo]

it will be the same size for every frame. with PixelBuffer being reused correctly now, maybe BufferPool will not be much of use for YUV.

You are correct. I will continue to modify the current DefaultBufferPool implementation and will try to merge it soon.

[kekyo]

@RyanYANG52 Implemented buffer returner, could you please test it?

[RyanYANG52]

1

I test in my PC, it works. But I'm not still sure about the DefaultBufferPool impl.
using my 1080p mjpeg web camera on windows, after 1 minute, debug log results:

DefaultBufferPool: Returned: Size=453084, Saved=76
DefaultBufferPool: Rend: Size=461498/461498, ExactSize=False, Saved=76
Allocated: Size=461498
DefaultBufferPool: Returned: Size=461498, Saved=77
DefaultBufferPool: Rend: Size=465660/465660, ExactSize=False, Saved=77
Allocated: Size=465660
DefaultBufferPool: Returned: Size=465660, Saved=78
DefaultBufferPool: Rend: Size=466170/466170, ExactSize=False, Saved=78
Allocated: Size=466170

for default impl, the max retained memory in pool would be around 16 * 13 * 400k = ~80M

for this simple buffer provider, I only get one or two Rent call per capture

public class NoneBufferPool : BufferPool
{
    public override byte[] Rent(int size, bool exactSize)
    {
        Debug.WriteLine($"NoneBufferPool: Rent: {size}, {exactSize}");
        return new byte[exactSize ? size: size * 2];
    }

    public override void Return(byte[] buffer)
    {
    }
}

I think mjpeg web camera is relatively main stream, if add current DefaultBufferPool as a silent default, may not be the best for existing user, can be a optin

2

Another thing is for YUV and transcode frames, they rent exactSize and only return when size is less then current, which should never happen (not sure) because size does not change per PixelBuffer.
for ForceCopy because it never return, I think a simplenew byte[] is ok.

So I think for BufferPool interface maybe only rent for not exact size, like ArrayPool, or add a exactSize param on Return as well

3

Rent and Return is not matched;

PixelBuffer currently has no dispose or clean method, FrameProcessor has one, when it disposes reserver.Clear(), does not Return the imageContainer.
It is not very important

[kekyo]

How about this changes?

I think mjpeg web camera is relatively main stream, if add current DefaultBufferPool as a silent default, may not be the best for existing user, can be a optin

In practice, I believe that in MJPEG, the sizes rarely match, and as a result, Rent often fails. I changed the implementation:

1. Since the size is distributed by 13, if the size differs even a little, different Holders will be searched and consequently the stored buffer cannot be found.
   This left the original implementation almost intact (without thinking! ;)
   For this purpose, I changed it to a simple linear search, considering that there is little need to distribute it in the first place.
   Since it is a two or three digit elements, a linear search should be sufficient.
2. Arrays stored in the buffer pool are now held in WeakReference. Now arrays that do not match in size will be recovered by GC in due course.

Another thing is for YUV and transcode frames, they rent exactSize and only return when size is less then current, which should never happen (not sure) because size does not change per PixelBuffer. for ForceCopy because it never return, I think a simplenew byte[] is ok.

You are correct, I have fix it.

PixelBuffer currently has no dispose or clean method, FrameProcessor has one, when it disposes reserver.Clear(), does not Return the imageContainer.

I was wondering if I should implement PixelBuffer.Dispose().
But in the first place, the call to FrameProcessor.ReleasePixelBuffer() is optional, and implicitly the concrete implementation of FrameProcessor guarantees the call design.

Therefore, if I implement PixelBuffer.Dispose(), I will need to do some work it as well, but since there is little chance that this will improve the cost for users who use the default frame processor, I decided to leave it as it is now.

[RyanYANG52]

I love it, simple & powerful, and with WeakReference, PixelBuffer.Dispose() is not necessary

[kekyo]

Will write docs and merge, thanks!