Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript implementation #10

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions JavaScript/LiveCameraSample/LiveCameraResult.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<!DOCTYPE HTML>
<html>
<head>
<title>JavaScript - Live Camera Result</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
</head>
<body onload="cameraStart()">

<h2> Microsoft Cognitive Services </h2>
<h2> Computer Vision API v2.0 - Describe Image </h2>

<!-- Main -->
<div id="main">
<div class="camera-div">
<video id="video"></video>
<canvas hidden id="canvas" width="640" height="480"></canvas>
<photo hidden id ="photo" width="640" height="480"></photo>
</div>
<div class="result-div">
<h3>Captions:</h3>
<div class="captions"></div>

<h3>Tags:</h3>
<div class="tags"></div>
</div>
</div>

<!-- Scripts -->
<script src="jquery.min.js"></script>
<script src="camera.js"></script>

</body>
</html>
102 changes: 102 additions & 0 deletions JavaScript/LiveCameraSample/camera.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
// this sample API URL corresponds to Computer Vision API v2.0 - Describe Image
// please refer to the Computer Vision API Reference for API usage
var API_URL = 'https://westus.api.cognitive.microsoft.com/vision/v2.0/describe'

// replace this with your own API KEY
var API_KEY = '---YOUR_API_KEY---'

// this is the rate of how frequent your camera image is sent to the Computer Vision API
var CAPTURE_RATE = 3000 // milliseconds

// camera settings
var CONSTRAINTS = {
audio: false
, video: { facingMode: 'environment' } // using the rear camera
//, video: { facingMode: 'user' } // using the front camera
, video: { exact: {
height: 500 // change this to your desired image height
, width: 800 } // change this to your desired image width
}
}

async function cameraStart() {
try {
const stream = await navigator.mediaDevices.getUserMedia(CONSTRAINTS)

video.srcObject = stream
video.play()

var canvas = document.getElementById('canvas')
canvas.style.display = 'none'
var context = canvas.getContext('2d')

setInterval(
function () {
context.drawImage(video, 0, 0, 680, 480)
sendImageToAPI() // invoke the call to Computer Vision API
}
, CAPTURE_RATE
)

} catch (err) {
throw err
}
}

// send your camera image to Computer Vision API
function sendImageToAPI() {

var payload = dataURItoBlob()
console.log(payload)

$.ajax({
type: 'POST'
, url: API_URL
, headers: {
'Content-Type': 'application/octet-stream'
, 'Ocp-Apim-Subscription-Key': API_KEY
}
, data: payload
, processData: false
, success: function (data) {
console.log(data)

var captionsResult = ''
$.each(data.description.captions, function(index, value) {
captionsResult = captionsResult + '<p>' + value.text + ' [confidence: ' + value.confidence + '] </p>'
})
changeInnerHtml('.captions', captionsResult)

var tagsResult = ''
$.each(data.description.tags, function(index, value) {
tagsResult = tagsResult + value + ' '
})
changeInnerHtml('.tags', tagsResult)
}
, error: function (xhr, status) {
alert('API POST Error.')
}
})
}

// helper functions - DO NOT CHANGE IF YOU DONT UNDERSTAND THE CODE BELOW
function dataURItoBlob() {
var dataUri = canvas.toDataURL('image/jpeg')
var data = dataUri.split(',')[1]
var mimeType = dataUri.split('')[0].slice(5)

var bytes = window.atob(data)
var buf = new ArrayBuffer(bytes.length)
var byteArr = new Uint8Array(buf)

for (var i = 0; i < bytes.length; i++) {
byteArr[i] = bytes.charCodeAt(i)
}
return byteArr
}

function changeInnerHtml(elementPath, newText){
$(elementPath).fadeOut(500, function() {
$(this).html(newText).fadeIn(500)
})
}
2 changes: 2 additions & 0 deletions JavaScript/LiveCameraSample/jquery.min.js

Large diffs are not rendered by default.

Binary file added JavaScript/LiveCameraSample/screenshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
30 changes: 30 additions & 0 deletions JavaScript/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Video Frame Analysis Sample - JavaScript

<p align="center"><img alt="" src="https://github.com/aaronchong888/Cognitive-Samples-VideoFrameAnalysis/blob/javascript-implementation/JavaScript/LiveCameraSample/screenshot.png" width="50%"></p>

## Description

This sample demonstrates a simple HTML website with client side JavaScript codes for analyzing video frames from a webcam in near-real-time using Vision APIs from [Microsoft Cognitive Services][].

[Microsoft Cognitive Services]: https://azure.microsoft.com/en-us/services/cognitive-services/

## Getting Started

1. Get API keys for the Vision APIs from [Microsoft Cognitive Services][]. For video frame analysis, the applicable APIs are:
- [Computer Vision API][]
- [Face API][]
- [Custom Vision][]

2. Open `camera.js` in any code editor (e.g. [VSCode](https://code.visualstudio.com/)), change `API_URL` and `API_KEY` to your desired API and the corresponding key

3. Open `LiveCameraResult.html` in any web browsers (e.g. [Microsoft Edge](https://www.microsoft.com/en-us/windows/microsoft-edge)) to run the sample and see the results!

[Microsoft Cognitive Services]: https://azure.microsoft.com/en-us/services/cognitive-services/directory/vision/
[Computer Vision API]: https://azure.microsoft.com/en-us/try/cognitive-services/?api=computer-vision
[Face API]: https://azure.microsoft.com/en-us/try/cognitive-services/?api=face-api
[Custom Vision]: https://www.customvision.ai/

## Contributors

* **Aaron Chong** - *Initial work* - [aaronchong888](https://github.com/aaronchong888)
* **Claudia Leung** - *Initial work* - [calx3eung](https://github.com/calx3eung)