Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
Vennela Kodali authored and Vennela Kodali committed Mar 17, 2020
1 parent b92df6b commit 3e035af
Show file tree
Hide file tree
Showing 5 changed files with 17 additions and 6 deletions.
2 changes: 1 addition & 1 deletion CapMl.podspec
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
s.name = 'CapMl'
s.version = '0.0.1'
s.summary = 'Machine Learning Plugin for Capacitor'
s.license = 'Apache v2'
s.license = 'Hippocratic License Version 2.0'
s.homepage = '[email protected]:bendyworks/cap-ml'
s.author = 'Vennela Kodali'
s.source = { :git => '[email protected]:bendyworks/cap-ml', :tag => s.version.to_s }
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Machine Learning Plugin for Capacitor. Currently offered implementations include -
- Text Detector: Text Detection in still images.

We're using Apple's Vision Framework which has some limitations like not being able to detect cursive/handwriting font etc.
On the ios side, we're using Apple's Vision Framework and MLKit's Vision Framework on the Android side. Both have some limitations like not being able to detect cursive/handwriting font etc.

TextDetector expects the image to be sent in portrait mode only, i.e. with text facing up. It will try to process even otherwise, but note that it might result in gibberish.

Expand Down Expand Up @@ -81,6 +81,8 @@ After checking out the repo,
- run `npx cap sync` to sync with ios and install pods
Plugin should be ready at this point. To test it out -
- navigate to examples/text-detection/ImageReader

### ios Development
- run `npx capacitor open ios` to open up an XCode project.
- Run the XCode project either on a simulator or a device.
- For each change in the javascript part of the app, run `npm run build && npx cap sync ios` to deploy the corresponding changes to ios app
Expand All @@ -89,6 +91,16 @@ After checking out the repo,
Plugin code is located at Pods/DevelopmentPods/CapML
- `Plugin.swift` is the entry point to the Plugin.

### Android Development
- run `npx capacitor open android` to open up an Android Studio project.
- Run the project either on a simulator or a device.
- For each change in the javascript part of the app, run `npm run build && npx cap sync android` to deploy the corresponding changes to android app
(or)
- (recommended) Enable live reload of the app, using `ionic capacitor run android --livereload`
- Plugin code is located at `android-cap-ml/java/com.bendyworks.capML`
- `CapML.java` is the entry point to the Plugin.
(Note: When plugin code is updated, make sure to rebuild the project before running it.)

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/bendyworks/cap-ml.
Expand Down
3 changes: 1 addition & 2 deletions android/src/main/java/com/bendyworks/capML/TextDetector.kt
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ class TextDetector {
val rect: Rect = line.boundingBox ?: throw NoSuchPropertyException("FirebaseVisionTextRecognizer.processImage: could not get bounding coordinates")

val textDetection = mapOf(
// normalizing coordinates and switching to 1st quadrant ie origin(0,0) as bottom left
// in order to match the result with that of core-ml's Vision framework.
// normalizing coordinates
"topLeft" to listOf<Double?>((rect.left).toDouble()/width, (height - rect.top).toDouble()/height),
"topRight" to listOf<Double?>((rect.right).toDouble()/width, (height - rect.top).toDouble()/height),
"bottomLeft" to listOf<Double?>((rect.left).toDouble()/width, (height - rect.bottom).toDouble()/height),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ export class HomePage implements OnInit {
this.textDetections.forEach((detection: TextDetection) => {
// the received coordinates are normalized by Vision framework, proportionate to the width and height of the image itself. Hence here, x coordinate is multiplied by image width(scaleX), and y with height(scaleY) to obtain de-normalized coordinates for the chosen image scale.
const x = detection.bottomLeft[0] * this.scaleX
// In addition to de-normalizing, subtracting from scaleY because TextDetector(which uses Apple's Vision Framework) assumes bottom-left as origin (0,0) vs SVG rect which assumes top-left as origin (0,0)
// In addition to de-normalizing, subtracting from scaleY because cap-ml assumes bottom-left as origin (0,0) vs SVG rect which assumes top-left as origin (0,0)
const y = this.scaleY - (detection.topLeft[1] * this.scaleY)

// Similar to the x and y coordinates above, the received coordinates are normalized by Vision framework, proportionate to the width and height of the image itself. Hence here, difference between corresponding x-coordinates is multiplied by image width(scaleX), and difference between corresponding y-coordinates is multiplied by image height(scaleY) to obtain de-normalized dimensions for the chosen image scale.
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"prepublishOnly": "npm run build"
},
"author": "Vennela Kodali",
"license": "Apache v2",
"license": "Hippocratic License Version 2.0",
"dependencies": {
"@capacitor/core": "latest"
},
Expand Down

0 comments on commit 3e035af

Please sign in to comment.