🐛 Exception in HostFunction, when using useSkiaFrameProcessor

[ishiharas]

What's happening?

I have setup a fresh basic expo development build project, created a default camera page and tried to use the useSkiaFrameProcessor hook. Nothing is inside it, except the frame.render() function.

When I try to open the camera preview, I get the following exception with a black preview screen:
Frame Processor Error: Exception in HostFunction: Failed to convert NativeBuffer to SkImage!, js engine: VisionCamera

After playing around I tried to change the "pixelFormat" prop to rgb. That fixes the exception from occuring, but shows a "rgb-preview-image"?

Using the normal useFrameProcessor hooks works without any issues.

I am testing on the following device: Samsung S24+ with Android 14

I have attached a small part of the logcat logs before the exception appears, the basic expo project as a zip and
also the full logcat logs from android studio as a text file for both pixel formats.

lorcana-scanner.zip

Logcat-pixelFormatRGB.txt
Logcat-pixelFormatYUV.txt

If there is anything else, that I can provide for debugging this issue, let me know. I am happy to help.

Reproduceable Code

export default function ScannerScreen() {
  const device = useCameraDevice("back");
  const { hasPermission, requestPermission } = useCameraPermission();

  // const devices = Camera.getAvailableCameraDevices();
  // console.log(JSON.stringify(devices, null, 2));

  const skiaFrameProcessor = useSkiaFrameProcessor((frame) => {
    "worklet";
    frame.render();
  }, []);

  const frameProcessor = useFrameProcessor((frame) => {
    "worklet";
  }, []);

  useEffect(() => {
    if (!hasPermission) {
      requestPermission();
    }
  }, []);

  if (!hasPermission)
    return <ThemedText type="title">No Permission</ThemedText>;
  if (device == null) return <ThemedText type="title">No Device</ThemedText>;

  return (
    <Camera
      style={StyleSheet.absoluteFill}
      device={device}
      isActive={true}
      pixelFormat="rgb"
      frameProcessor={skiaFrameProcessor}
    />
  );
}

package.json:
{
  "name": "lorcana-scanner",
  "main": "expo-router/entry",
  "version": "1.0.0",
  "scripts": {
    "start": "expo start",
    "reset-project": "node ./scripts/reset-project.js",
    "android": "expo run:android",
    "ios": "expo run:ios",
    "web": "expo start --web",
    "test": "jest --watchAll",
    "lint": "expo lint"
  },
  "jest": {
    "preset": "jest-expo"
  },
  "dependencies": {
    "@expo/vector-icons": "^14.0.2",
    "@react-navigation/bottom-tabs": "^7.2.0",
    "@react-navigation/native": "^7.0.14",
    "@shopify/react-native-skia": "^1.11.8",
    "expo": "~52.0.37",
    "expo-blur": "~14.0.3",
    "expo-build-properties": "~0.13.2",
    "expo-constants": "~17.0.7",
    "expo-dev-client": "~5.0.12",
    "expo-font": "~13.0.4",
    "expo-haptics": "~14.0.1",
    "expo-linking": "~7.0.5",
    "expo-router": "~4.0.17",
    "expo-splash-screen": "~0.29.22",
    "expo-status-bar": "~2.0.1",
    "expo-symbols": "~0.2.2",
    "expo-system-ui": "~4.0.8",
    "expo-web-browser": "~14.0.2",
    "react": "18.3.1",
    "react-dom": "18.3.1",
    "react-native": "0.76.7",
    "react-native-fast-opencv": "^0.4.1",
    "react-native-gesture-handler": "~2.20.2",
    "react-native-reanimated": "~3.16.1",
    "react-native-safe-area-context": "4.12.0",
    "react-native-screens": "~4.4.0",
    "react-native-vision-camera": "^4.6.4",
    "react-native-web": "~0.19.13",
    "react-native-webview": "13.12.5",
    "react-native-worklets-core": "^1.5.0"
  },
  "devDependencies": {
    "@babel/core": "^7.25.2",
    "@types/jest": "^29.5.12",
    "@types/react": "~18.3.12",
    "@types/react-test-renderer": "^18.3.0",
    "jest": "^29.2.1",
    "jest-expo": "~52.0.4",
    "react-test-renderer": "18.3.1",
    "typescript": "^5.3.3"
  },
  "private": true
}

Relevant log output

VideoCapture            com.anonymous.lorcanascanner         D  cameraCaptureResult timestampNs = 1721106362091877, current system uptimeMs = 816185653, current system realtimeMs = 1721106446
skia                    com.anonymous.lorcanascanner         D  Could not create EGL image, err = (0x300c)
RNSkia                  com.anonymous.lorcanascanner         I  Failed to convert HardwareBuffer to OpenGL Texture!
ReactNativeJS           com.anonymous.lorcanascanner         E  Frame Processor Error: Exception in HostFunction: Failed to convert NativeBuffer to SkImage!, js engine: VisionCamera

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 10,
  "minZoom": 0.6000000238418579,
  "maxExposure": 20,
  "supportsLowLightBoost": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera",
    "ultra-wide-angle-camera",
    "wide-angle-camera",
    "telephoto-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": true,
  "minFocusDistance": 10,
  "minExposure": -20,
  "name": "0 (BACK) androidx.camera.camera2",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

Samsung S24+ (Android 14)

VisionCamera Version

4.6.4

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here! 🍻

Thank you for providing detailed information about your issue. It looks like a valid bug with the use of useSkiaFrameProcessor.

A few things to consider: Have you tried reproducing this issue in the VisionCamera Example app? This will help us determine if it's specific to your project setup. Also, please ensure your environment matches the library requirements and test with different pixel formats.

If the problem persists, providing specific logs from adb logcat during the error occurrence will be useful for mrousavy.

Let’s keep this open while we gather more insights!

Note: If you think I made a mistake, please ping @mrousavy to take a look.

[RileyAdlington]

I can confirm that something similar is happening in the example project on my S24. Switching the useFrameProcessor hook to a useSkiaFrameProcessor hook and adding a frame.render() call before the runAtTargetFps() results in the following error.

Frame Processor Error: Cannot get Platform Buffer - getNativeBuffer() requires HardwareBuffers, which are only available on Android API 26 or above. Set your app's minSdk version to 26 and try again.

When I edit the build.gradle file's minSdkVersion as recommended to 26, the app returns the same multicoloured preview image for a few seconds and then crashes. In my case though, when I set the pixelFormat prop to 'rgb' then I get a few frames of normal camera function before it hangs then exits.

I have attached what I think are the relevant logs from Android Studio but I have never used it before now so apologies if they are some not very helpful ones.

ExampleProjectLogcat-YUV.txt