This repository has no description
1An realtime pose detection library for [Android](https://www.android.com/)
2and [Compose Multiplatform](https://www.jetbrains.com/lp/compose-multiplatform/).
3Android version uses CameraX and GoogleML Kit, while iOS version uses AVFoundation with VisionKit
4and CoreML.
5We also support analysing pre-recorded video files.
6We now also support adding custom object detection models to the library, allowing you to detect
7custom objects in your camera feed or video files along side body poses.
8
9## Quick Start
10
11Import the Compose library
12
13```kotlin
14implementation("com.performancecoachlab.posedetection:posedetection-compose:4.7.2")
15```
16
17Add camera use to your android manifest
18
19```xml
20<uses-feature
21 android:name="android.hardware.camera"
22 android:required="false" />
23<uses-permission android:name="android.permission.CAMERA" />
24```
25
26Add camera use to you iOS info.plist
27
28```xml
29<key>NSCameraUsageDescription</key>
30<string>We need access to your camera to analyse your performance.</string>
31```
32
33## Usage
34
35Request camera permissions
36
37```kotlin
38var permissionGranted by remember { mutableStateOf(false) }
39PermissionProvider().apply {
40 if (!hasCameraPermission()) RequestCameraPermission(onGranted = {
41 permissionGranted = true
42 }, onDenied = { permissionGranted = false }) else permissionGranted = true
43}
44```
45
46Create a Skeleton Repisitory
47
48```kotlin
49val skeletonRepository = remember { SkeletonRepository() }
50val customObjectRepository = remember { CustomObjectRespository() }
51```
52
53Initialise the camera feed
54
55```kotlin
56if (permissionGranted) {
57 CameraView(
58 skeletonRepository = skeletonRepository,
59 customObjectRepository = customObjectRespository,
60 )
61} else {
62 Text("Camera permission not granted")
63}
64```
65
66Create a Pose to detect
67
68```kotlin
69val upRightPose = Pose(
70 leftShoulder = Pose.PoseRange(0.0, 40.0),
71 rightShoulder = Pose.PoseRange(0.0, 40.0),
72 leftHip = Pose.PoseRange(160.0, 180.0),
73 rightHip = Pose.PoseRange(160.0, 180.0),
74 leftKnee = Pose.PoseRange(160.0, 180.0),
75 rightKnee = Pose.PoseRange(160.0, 180.0)
76)
77```
78
79Listen for skeleton updates and detect specific poses
80
81```kotlin
82val skeleton by skeletonRepository.skeletonFlow.collectAsState()
83val poseDetected = skeleton?.let {
84 upRightPose.matches(it)
85}
86```
87
88Analyse pre recorded video files.
89Initialise the video extraction for android with your application context.
90
91```kotlin
92VideoExtractionContext.setUp(applicationContext)
93```
94
95extract frames from the video and request analysis
96
97```kotlin
98rememberCoroutineScope().launch {
99 try {
100 extractFrame(url, frame, VideoExtractionContext)?.let { frame ->
101 frameAnalyser.analyseFrame(frame)?.let { skeleton ->
102 bitmap = frame.drawSkeleton(skeleton)
103 }
104 }
105 } catch (e: Exception) {
106 e.printStackTrace()
107 }
108}
109```
110
111Add a custom object detection model
112Initialse the custom models for ios and android respectively.
113For android you need to add a .tflite model file to your assets folder, then set androidModelPath to
114the name of the model file, including the .tflite extension.
115For iOS you need to add a .mlmodel model file to your Xcode project, then set iosModelPath to the
116name of the model file without the .mlmodel extension.
117
118```kotlin
119val generalModel = ObjectModelProvider.get(
120 ModelPath(
121 "yolov10n_float16.tflite",
122 "YOLOv3FP16"
123 )
124)
125```
126
127Once this is done, you can use the model to detect objects in the camera feed or video frames.
128
129Check out the sample app for full example of how to use the library.
130
131## License
132
133 Licensed under the Apache License, Version 2.0 (the "License");
134 you may not use this file except in compliance with the License.
135 You may obtain a copy of the License at
136
137 https://www.apache.org/licenses/LICENSE-2.0
138
139 Unless required by applicable law or agreed to in writing, software
140 distributed under the License is distributed on an "AS IS" BASIS,
141 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
142 See the License for the specific language governing permissions and
143 limitations under the License.