This repository has no description
1An realtime pose detection library for [Android](https://www.android.com/) and [Compose Multiplatform](https://www.jetbrains.com/lp/compose-multiplatform/).
2Android version uses CameraX and GoogleML Kit, while iOS version uses AVFoundation with VisionKit and CoreML.
3We also support analysing pre-recorded video files.
4We now also support adding custom object detection models to the library, allowing you to detect custom objects in your camera feed or video files along side body poses.
5
6## Quick Start
7
8Import the Compose library
9
10```kotlin
11implementation("com.performancecoachlab.posedetection:posedetection-compose:4.3.0")
12```
13
14Add camera use to your android manifest
15
16```xml
17<uses-feature
18 android:name="android.hardware.camera"
19 android:required="false" />
20<uses-permission android:name="android.permission.CAMERA" />
21```
22
23Add camera use to you iOS info.plist
24
25```xml
26<key>NSCameraUsageDescription</key>
27<string>We need access to your camera to analyse your performance.</string>
28```
29
30## Usage
31
32Request camera permissions
33```kotlin
34var permissionGranted by remember { mutableStateOf(false) }
35PermissionProvider().apply {
36 if (!hasCameraPermission()) RequestCameraPermission(onGranted = {
37 permissionGranted = true
38 }, onDenied = { permissionGranted = false }) else permissionGranted = true
39}
40```
41
42Create a Skeleton Repisitory
43```kotlin
44val skeletonRepository = remember { SkeletonRepository() }
45val customObjectRepository = remember { CustomObjectRespository() }
46```
47
48Initialise the camera feed
49```kotlin
50if (permissionGranted) {
51 CameraView(
52 skeletonRepository = skeletonRepository,
53 customObjectRepository = customObjectRespository,
54 )
55} else {
56 Text("Camera permission not granted")
57}
58```
59
60Create a Pose to detect
61```kotlin
62val upRightPose = Pose(
63 leftShoulder = Pose.PoseRange(0.0, 40.0),
64 rightShoulder = Pose.PoseRange(0.0, 40.0),
65 leftHip = Pose.PoseRange(160.0, 180.0),
66 rightHip = Pose.PoseRange(160.0, 180.0),
67 leftKnee = Pose.PoseRange(160.0, 180.0),
68 rightKnee = Pose.PoseRange(160.0, 180.0)
69)
70```
71
72Listen for skeleton updates and detect specific poses
73```kotlin
74val skeleton by skeletonRepository.skeletonFlow.collectAsState()
75val poseDetected = skeleton?.let {
76 upRightPose.matches(it)
77}
78```
79
80Analyse pre recorded video files.
81Initialise the video extraction for android with your application context.
82```kotlin
83VideoExtractionContext.setUp(applicationContext)
84```
85extract frames from the video and request analysis
86```kotlin
87rememberCoroutineScope().launch {
88 try {
89 extractFrame(url, frame, VideoExtractionContext)?.let { frame ->
90 frameAnalyser.analyseFrame(frame)?.let { skeleton ->
91 bitmap = frame.drawSkeleton(skeleton)
92 }
93 }
94 } catch (e: Exception) {
95 e.printStackTrace()
96 }
97}
98```
99
100Add a custom object detection model
101Initialse the custom models for ios and android respectively.
102For android you need to add a .tflite model file to your assets folder, then set androidModelPath to the name of the model file, including the .tflite extension.
103For iOS you need to add a .mlmodel model file to your Xcode project, then set iosModelPath to the name of the model file without the .mlmodel extension.
104```kotlin
105val generalModel = initialiseObjectModel(
106 ModelPath(
107 "lite-model_efficientdet_lite2_detection_metadata_1.tflite",
108 "YOLOv3FP16"
109 )
110)
111```
112Once this is done, you can use the model to detect objects in the camera feed or video frames.
113
114
115Check out the sample app for full example of how to use the library.
116
117## License
118
119 Licensed under the Apache License, Version 2.0 (the "License");
120 you may not use this file except in compliance with the License.
121 You may obtain a copy of the License at
122
123 https://www.apache.org/licenses/LICENSE-2.0
124
125 Unless required by applicable law or agreed to in writing, software
126 distributed under the License is distributed on an "AS IS" BASIS,
127 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
128 See the License for the specific language governing permissions and
129 limitations under the License.