shubendrak shubendrak - 10 months ago 169
Android Question

Android - Get byte[] from Surface / virtualDisplay

I am trying to screen cast my android device screen to a web browser using

projection API

Projection API
renders its output to a
and returns a virtualDisplay. I have done till this. I saw the webrtc library for android. they have made it to receive input only from device camera. I am trying to read and modify webrtc code to stream whatever is shown to the surface.

My question is How can i receive byte[] data from a
regularly like the
function. What other available options i have?

Answer Source

Here is how i solved my problem. I used ImageReader class like

imageReader = ImageReader.newInstance(displayWidth, displayHeight, PixelFormat.RGBA_8888, 2);
                    displayWidth, displayHeight, density,
                    flags, imageReader.getSurface(), null, handler);
imageReader.setOnImageAvailableListener(new ImageAvailableListener(), null);

private class ImageAvailableListener implements ImageReader.OnImageAvailableListener {
    public void onImageAvailable(ImageReader reader) {
        Image image = null;
        Bitmap bitmap = null;

        ByteArrayOutputStream stream = null;

        try {
            image = imageReader.acquireLatestImage();
            if (image != null) {
                Image.Plane[] planes = image.getPlanes();
                ByteBuffer buffer = planes[0].getBuffer();
                int pixelStride = planes[0].getPixelStride();
                int rowStride = planes[0].getRowStride();
                int rowPadding = rowStride - pixelStride * displayWidth;

                // create bitmap
                bitmap = Bitmap.createBitmap(displayWidth + rowPadding / pixelStride,
                        displayHeight, Bitmap.Config.ARGB_8888);
                    stream = new ByteArrayOutputStream();
                    bitmap.compress(Bitmap.CompressFormat.JPEG, 50, stream);
                    StringBuilder sb = new StringBuilder();
                    sb.append(StringUtils.newStringUtf8(Base64.encode(stream.toByteArray(), Base64.DEFAULT)));

        } catch (Exception e) {

I am converting byte[] to Base64 string and sending to through webrtc datachannel.