An realtime pose detection library for [Android](https://www.android.com/) and [Compose Multiplatform](https://www.jetbrains.com/lp/compose-multiplatform/). 
Android version uses CameraX and GoogleML Kit, while iOS version uses AVFoundation with VisionKit and CoreML.
We also support analysing pre-recorded video files.
We now also support adding custom object detection models to the library, allowing you to detect custom objects in your camera feed or video files along side body poses.

## Quick Start

Import the Compose library

```kotlin
implementation("com.performancecoachlab.posedetection:posedetection-compose:4.3.0")
```

Add camera use to your android manifest

```xml
<uses-feature
        android:name="android.hardware.camera"
        android:required="false" />
<uses-permission android:name="android.permission.CAMERA" />
```

Add camera use to you iOS info.plist

```xml
<key>NSCameraUsageDescription</key>
<string>We need access to your camera to analyse your performance.</string>
```

## Usage

Request camera permissions
```kotlin
var permissionGranted by remember { mutableStateOf(false) }
PermissionProvider().apply {
    if (!hasCameraPermission()) RequestCameraPermission(onGranted = {
        permissionGranted = true
    }, onDenied = { permissionGranted = false }) else permissionGranted = true
}
```

Create a Skeleton Repisitory
```kotlin
val skeletonRepository = remember { SkeletonRepository() }
val customObjectRepository = remember { CustomObjectRespository() }
```

Initialise the camera feed
```kotlin
if (permissionGranted) {
    CameraView(
        skeletonRepository = skeletonRepository,
        customObjectRepository = customObjectRespository,
    )
} else {
    Text("Camera permission not granted")
}
```

Create a Pose to detect
```kotlin
val upRightPose = Pose(
    leftShoulder = Pose.PoseRange(0.0, 40.0),
    rightShoulder = Pose.PoseRange(0.0, 40.0),
    leftHip = Pose.PoseRange(160.0, 180.0),
    rightHip = Pose.PoseRange(160.0, 180.0),
    leftKnee = Pose.PoseRange(160.0, 180.0),
    rightKnee = Pose.PoseRange(160.0, 180.0)
)
```

Listen for skeleton updates and detect specific poses
```kotlin
val skeleton by skeletonRepository.skeletonFlow.collectAsState()
val poseDetected = skeleton?.let {
    upRightPose.matches(it)
}
```

Analyse pre recorded video files.
Initialise the video extraction for android with your application context.
```kotlin
VideoExtractionContext.setUp(applicationContext)
```
extract frames from the video and request analysis
```kotlin
rememberCoroutineScope().launch {
    try {
        extractFrame(url, frame, VideoExtractionContext)?.let { frame ->
            frameAnalyser.analyseFrame(frame)?.let { skeleton ->
                bitmap = frame.drawSkeleton(skeleton)
            }
        }
    } catch (e: Exception) {
        e.printStackTrace()
    }
}
```

Add a custom object detection model
Initialse the custom models for ios and android respectively.
For android you need to add a .tflite model file to your assets folder, then set androidModelPath to the name of the model file, including the .tflite extension.
For iOS you need to add a .mlmodel model file to your Xcode project, then set iosModelPath to the name of the model file without the .mlmodel extension.
```kotlin
val generalModel = initialiseObjectModel(
    ModelPath(
        "lite-model_efficientdet_lite2_detection_metadata_1.tflite",
        "YOLOv3FP16"
    )
)
```
Once this is done, you can use the model to detect objects in the camera feed or video frames.


Check out the sample app for full example of how to use the library.

## License

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

       https://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
