react-native-vision-camera-plugin-frame-to-file

A react-native-vision-camera frame processor to save a frame to a file. Can be used to send frames to a server performing some remote image processing.

Installation

npm install react-native-vision-camera-plugin-frame-to-file

Demo

This demo was recorded on an iPhone 11. Format is 3024x4032@30fps.

The bottom left square continuously displays the last saved frame.

• Without resizing the frame, it runs at around 1 fps.
• When resizing the frame, it runs at around 8 fps.
• When cropping the frame, it runs at around 23 fps.

Saved file format

The frame is saved as a PNG file. Support for JPEG might come later on.

Usage

[!WARNING]

• Only IOS is supported for now !
• Saving an entire 3024x4032 frame produces a heavy file ~10/20mb. Consider using runAtTargetFps.
• Saving resized/cropped frames in combination with react-native-resize-plugin can also be a good idea. However only the uint8 dataFormat & argb pixel format are supported.

Saving the full frame

import { useToFilePlugin } from 'react-native-vision-camera-plugin-frame-to-file';

// ...
const { toFile } = useToFilePlugin();

// ...
const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  const filePath = toFile(frame);
  // Do something with the saved file's path
  console.log(filePath);
}, []);

Saving a resized frame using react-native-resize-plugin

[!IMPORTANT]

• You still have to provide the frame as a first parameter, but it will be ignored. Idea came from this issue

import { useToFilePlugin } from 'react-native-vision-camera-plugin-frame-to-file';
import { useResizePlugin } from 'vision-camera-resize-plugin';

// ...
const { toFile } = useToFilePlugin();
const { resize } = useResizePlugin();

// ...
const RESIZE_FACTOR = 4;
const TARGET_TYPE = 'uint8' as const;
const TARGET_FORMAT = 'argb' as const;

// ...
const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  const resizedFrame = resize(frame, {
    dataType: TARGET_TYPE,
    pixelFormat: TARGET_FORMAT,
    scale: {
      width: frame.width / RESIZE_FACTOR,
      height: frame.height / RESIZE_FACTOR,
    },
  });
  const filePath = toFile(frame, {
    resizedFrameBuffer: resizedFrame.buffer,
    resizedFrameProperties: {
      width: frame.width / RESIZE_FACTOR,
      height: frame.height / RESIZE_FACTOR,
      dataType: TARGET_TYPE,
      pixelFormat: TARGET_FORMAT,
    },
  });
  // Do something with the saved file's path
  console.log(filePath);
}, []);

Saving a cropped frame using vision-camera-resize-plugin's cropping feature

[!IMPORTANT]

• You still have to provide the frame as a first parameter, but it will be ignored. Idea came from this issue

import { useToFilePlugin } from 'react-native-vision-camera-plugin-frame-to-file';
import { useResizePlugin } from 'vision-camera-resize-plugin';

// ...
const { toFile } = useToFilePlugin();
const { resize } = useResizePlugin();

// ...
const RESIZE_FACTOR = 4;
const TARGET_TYPE = 'uint8' as const;
const TARGET_FORMAT = 'argb' as const;

// This crop square is for you to adjust/determine
const cropSquare = {
    width: 0,
    height: 0,
    x: 100,
    y: 100
}

// ...
const frameProcessor = useFrameProcessor((frame) => {
  'worklet';
  const resizedFrame = resize(frame, {
    dataType: TARGET_TYPE,
    pixelFormat: TARGET_FORMAT,
    scale: {
        width: cropSquare.width,
        height: cropSquare.height,
    },
    crop: {
        x: cropSquare.x,
        y: cropSquare.y,
        width: cropSquare.width,
        height: cropSquare.height,
    },
  });
  const filePath = toFile(frame, {
    resizedFrameBuffer: resizedFrame.buffer,
    resizedFrameProperties: {
      width: cropSquare.width,
      height: cropSquare.height,
      dataType: TARGET_TYPE,
      pixelFormat: TARGET_FORMAT,
    },
  });
  // Do something with the saved file's path
  console.log(filePath);
}, []);

Performance

Saving the full frame

| iPhone 11 | Iphone 13 Pro | | ------------- | ------------- | | | |

Saving the resized frame

| iPhone 11 | Iphone 13 Pro | | ------------- | ------------- | | | |

Saving a resized/cropped frame

| iPhone 11 | Iphone 13 Pro | | ------------- | ------------- | | | |

Contributing

See the contributing guide to learn how to contribute to the repository and the development workflow.

License

MIT

Made with create-react-native-library