ESP32 CAM Server untuk Deteksi dan Lacak Warna

LAPORAN PRAKTIKUM

ESP32 Cam Server Untuk Deteksi dan Lacak Warna

Disusun Oleh :

Daniel Putra Ariyanto 1903421003

Ester Monica 1903421030

BM 6B

BROADBAND MULTIMEDIA

TEKNIK ELEKTRO

POLITEKNIK NEGERI JAKARTA

TAHUN AJARAN 2021/2022

I. Tujuan Praktikum

Pada praktikum ini akan dikenalkan alat/tool OpenCV.js dan OpenCV untuk lingkungan Server Web Kamera ESP32. Sebagai contoh, akan dibuat Server Web Kamera ESP32 sederhana yang menyertakan deteksi warna dan pelacakan objek bergerak. Diharapkan pengenalan ini akan menginspirasi kerja OpenCV tambahan dengan kamera ESP32.

II. Teori Singkat

A. ESP32 Cam

ESP32-CAM merupakan salah satu mikrokontroler yang memiliki fasilitas tambahan berupa bluetooth, wifi, kamera, bahkan sampai ke slot mikroSD. ESP32-CAM ini biasanya digunakan untuk project IoT (Internet of Things) yang membutuhkan fitur kamera. Modul ESP32CAM memiliki lebih sedikit pin I/O dibandingkan modul ESP32 produk sebelumnya, yaitu ESP32 Wroom. Hal ini dikarenakan sudah banyak pin yang digunakan secara internal untuk fungsi kamera dan fungsi slot kartu microSD. Selain itu, modul ESP32CAM juga tidak memiliki port USB khusus (mengirim program dari port USB komputer). Jadi untuk memprogram modul ini Anda harus menggunakan USB TTL atau kita dapat menambahkan modul tambahan berupa downloader khusus untuk ESP32-CAM.

Modul ESP32CAM memiliki 2 sisi dalam rangkaian modulnya. Di bagian atas terdapat modul kamera yang dapat dibongkar pasang dan ada microSD yang dapat diisi, serta flash sebagai lampu tambahan untuk kamera jika diperlukan. Di bagian belakang modul, terdapat antena internal, konektor untuk antena eksternal, pin male untuk I/O dan ESP32S sebagai otaknya. Lebih jelasnya, kita dapat melihat spesifikasinya sebagai berikut:

· 802.11b/g/n Wi-Fi

· Bluetooth 4.2 with BLE

· UART, SPI, I2C and PWM interfaces

· Clock speed up to 160 MHz

· Computing power up to 600 DMIPS

· 520 KB SRAM plus 4 MB PSRAM

· Supports WiFi Image Upload

· Multiple Sleep modes

· Firmware Over the Air (FOTA) upgrades possible

· 9 GPIO ports

· Built-in Flash LED

· Kamera

Referensi : https://indobot.co.id/blog/mengenal-esp32-cam-dan-bagaimana-cara-menggunakannya/

B. OpenCV.js

OpenCV (Open Source Computer Vision Library), adalah sebuah library open source yang dikembangkan oleh intel yang fokus untuk menyederhanakan programing terkait citra digital. Di dalam OpenCV sudah mempunyai banyak fitur, antara lain : pengenalan wajah, pelacakan wajah, deteksi wajah, Kalman filtering, dan berbagai jenis metode AI (Artificial Intellegence). Dan menyediakan berbagai algoritma sederhana terkait Computer Vision untuk low level API.

OpenCV merupakan open source computer vision library untuk bahasa pemrograman C/C++, dan telah dikembangkan ke phyton, java, matlab.

Referensi : https://binus.ac.id/malang/2017/10/introduction-to-open-cv/

Ø Open CV.js

Seperti yang dijelaskan dalam docs.opencv.org , " OpenCV.js adalah pengikatan JavaScript untuk subset yang dipilih dari fungsi OpenCV untuk platform web ". OpenCV.js menggunakan Emscripten, kompiler LLVM-ke-JavaScript, untuk mengompilasi fungsi OpenCV untuk pustaka API yang terus berkembang. OpenCV.js berjalan di browser yang memungkinkan uji coba cepat fungsi OpenCV oleh seseorang yang hanya memiliki latar belakang sederhana dalam HTML dan JavaScript. Mereka yang memiliki latar belakang di aplikasi Kamera Esp32 sudah memiliki latar belakang ini.

III. Diagram Rangkaian

IV. Langkah Percobaan

1. Pastikan Anda sudah menginstal Arduino IDE serta memasang board ESP32 CAM di Arduino IDE

2. Program ini dibagi menjadi dua file: the OCV_ColorTrack_P.ino file yang berisi program server dan index_OCV_ColorTrack.h file header yang berisi program klien (HTML, CSS dan JavaScript dengan OpenCV.js).

3. Buat sketsa Arduino baru yang disebut OCV_ColorTrack_P dan salin kode berikut.

/*********

The include file, index_OCV_ColorTrack.h, the Client, is an intoduction of OpenCV.js to the ESP32 Camera environment. The Client was

developed and written by Andrew R. Sass. Permission to reproduce the index_OCV_ColorTrack.h file is granted free of charge if this

entire copyright notice is included in all copies of the index_OCV_ColorTrack.h file.

Complete instructions at https://RandomNerdTutorials.com/esp32-cam-opencv-js-color-detection-tracking/

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files.

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

*********/

#include <WiFi.h>

#include <WiFiClientSecure.h>

#include "esp_camera.h"

#include "soc/soc.h"

#include "soc/rtc_cntl_reg.h"

#include "index_OCV_ColorTrack.h"

// Replace with your network credentials

const char* ssid = "REPLACE_WITH_YOUR_SSID";

const char* password = "REPLACE_WITH_YOUR_PASSWORD";

String Feedback="";

String Command="",cmd="",P1="",P2="",P3="",P4="",P5="",P6="",P7="",P8="",P9="";

byte ReceiveState=0,cmdState=1,strState=1,questionstate=0,equalstate=0,semicolonstate=0;

//ANN:0

// AI-Thinker

#define PWDN_GPIO_NUM 32

#define RESET_GPIO_NUM -1

#define XCLK_GPIO_NUM 0

#define SIOD_GPIO_NUM 26

#define SIOC_GPIO_NUM 27

#define Y9_GPIO_NUM 35

#define Y8_GPIO_NUM 34

#define Y7_GPIO_NUM 39

#define Y6_GPIO_NUM 36

#define Y5_GPIO_NUM 21

#define Y4_GPIO_NUM 19

#define Y3_GPIO_NUM 18

#define Y2_GPIO_NUM 5

#define VSYNC_GPIO_NUM 25

#define HREF_GPIO_NUM 23

#define PCLK_GPIO_NUM 22

WiFiServer server(80);

//ANN:2

void ExecuteCommand() {

if (cmd!="colorDetect") { //Omit printout

//Serial.println("cmd= "+cmd+" ,P1= "+P1+" ,P2= "+P2+" ,P3= "+P3+" ,P4= "+P4+" ,P5= "+P5+" ,P6= "+P6+" ,P7= "+P7+" ,P8= "+P8+" ,P9= "+P9);

//Serial.println("");

}

if (cmd=="resetwifi") {

WiFi.begin(P1.c_str(), P2.c_str());

Serial.print("Connecting to ");

Serial.println(P1);

long int StartTime=millis();

while (WiFi.status() != WL_CONNECTED)

{

delay(500);

if ((StartTime+5000) < millis()) break;

}

Serial.println("");

Serial.println("STAIP: "+WiFi.localIP().toString());

Feedback="STAIP: "+WiFi.localIP().toString();

}

else if (cmd=="restart") {

ESP.restart();

}

else if (cmd=="cm"){

int XcmVal = P1.toInt();

int YcmVal = P2.toInt();

Serial.println("cmd= "+cmd+" ,VALXCM= "+XcmVal);

Serial.println("cmd= "+cmd+" ,VALYCM= "+YcmVal);

}

else if (cmd=="quality") {

sensor_t * s = esp_camera_sensor_get();

int val = P1.toInt();

s->set_quality(s, val);

}

else if (cmd=="contrast") {

sensor_t * s = esp_camera_sensor_get();

int val = P1.toInt();

s->set_contrast(s, val);

}

else if (cmd=="brightness") {

sensor_t * s = esp_camera_sensor_get();

int val = P1.toInt();

s->set_brightness(s, val);

}

else {

Feedback="Command is not defined.";

}

if (Feedback=="") {

Feedback=Command;

}

void setup() {

WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0);

Serial.begin(115200);

Serial.setDebugOutput(true);

Serial.println();

camera_config_t config;

config.ledc_channel = LEDC_CHANNEL_0;

config.ledc_timer = LEDC_TIMER_0;

config.pin_d0 = Y2_GPIO_NUM;

config.pin_d1 = Y3_GPIO_NUM;

config.pin_d2 = Y4_GPIO_NUM;

config.pin_d3 = Y5_GPIO_NUM;

config.pin_d4 = Y6_GPIO_NUM;

config.pin_d5 = Y7_GPIO_NUM;

config.pin_d6 = Y8_GPIO_NUM;

config.pin_d7 = Y9_GPIO_NUM;

config.pin_xclk = XCLK_GPIO_NUM;

config.pin_pclk = PCLK_GPIO_NUM;

config.pin_vsync = VSYNC_GPIO_NUM;

config.pin_href = HREF_GPIO_NUM;

config.pin_sscb_sda = SIOD_GPIO_NUM;

config.pin_sscb_scl = SIOC_GPIO_NUM;

config.pin_pwdn = PWDN_GPIO_NUM;

config.pin_reset = RESET_GPIO_NUM;

config.xclk_freq_hz = 20000000;

config.pixel_format = PIXFORMAT_JPEG;

//init with high specs to pre-allocate larger buffers

if(psramFound()){

config.frame_size = FRAMESIZE_UXGA;

config.jpeg_quality = 10; //0-63 lower number means higher quality

config.fb_count = 2;

} else {

config.frame_size = FRAMESIZE_SVGA;

config.jpeg_quality = 12; //0-63 lower number means higher quality

config.fb_count = 1;

}

// camera init

esp_err_t err = esp_camera_init(&config);

if (err != ESP_OK) {

Serial.printf("Camera init failed with error 0x%x", err);

delay(1000);

ESP.restart();

}

//drop down frame size for higher initial frame rate

sensor_t * s = esp_camera_sensor_get();

WiFi.mode(WIFI_AP_STA);

WiFi.begin(ssid, password);

delay(1000);

long int StartTime=millis();

while (WiFi.status() != WL_CONNECTED) {

delay(500);

if ((StartTime+10000) < millis())

break;

}

if (WiFi.status() == WL_CONNECTED) {

Serial.print("ESP IP Address: http://");

Serial.println(WiFi.localIP());

}

server.begin();

}

void loop() {

Feedback="";Command="";cmd="";P1="";P2="";P3="";P4="";P5="";P6="";P7="";P8="";P9="";

ReceiveState=0,cmdState=1,strState=1,questionstate=0,equalstate=0,semicolonstate=0;

WiFiClient client = server.available();

if (client) {

String currentLine = "";

while (client.connected()) {

if (client.available()) {

char c = client.read();

getCommand(c);

if (c == '\n') {

if (currentLine.length() == 0) {

if (cmd=="colorDetect") {

camera_fb_t * fb = NULL;

fb = esp_camera_fb_get();

if(!fb) {

Serial.println("Camera capture failed");

delay(1000);

ESP.restart();

}

//ANN:1

client.println("HTTP/1.1 200 OK");

client.println("Access-Control-Allow-Origin: *");

client.println("Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept");

client.println("Access-Control-Allow-Methods: GET,POST,PUT,DELETE,OPTIONS");

client.println("Content-Type: image/jpeg");

client.println("Content-Disposition: form-data; name=\"imageFile\"; filename=\"picture.jpg\"");

client.println("Content-Length: " + String(fb->len));

client.println("Connection: close");

client.println();

uint8_t *fbBuf = fb->buf;

size_t fbLen = fb->len;

for (size_t n=0;n<fbLen;n=n+1024) {

if (n+1024<fbLen) {

client.write(fbBuf, 1024);

fbBuf += 1024;

}

else if (fbLen%1024>0) {

size_t remainder = fbLen%1024;

client.write(fbBuf, remainder);

}

esp_camera_fb_return(fb);

}

else {

//ANN:1

client.println("HTTP/1.1 200 OK");

client.println("Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept");

client.println("Access-Control-Allow-Methods: GET,POST,PUT,DELETE,OPTIONS");

client.println("Content-Type: text/html; charset=utf-8");

client.println("Access-Control-Allow-Origin: *");

client.println("Connection: close");

client.println();

String Data="";

if (cmd!="")

Data = Feedback;

else {

Data = String((const char *)INDEX_HTML);

}

int Index;

for (Index = 0; Index < Data.length(); Index = Index+1000) {

client.print(Data.substring(Index, Index+1000));

}

client.println();

}

Feedback="";

break;

} else {

currentLine = "";

}

else if (c != '\r') {

currentLine += c;

}

if ((currentLine.indexOf("/?")!=-1)&&(currentLine.indexOf(" HTTP")!=-1)) {

if (Command.indexOf("stop")!=-1) {

client.println();

client.stop();

}

currentLine="";

Feedback="";

ExecuteCommand();

}

delay(1);

client.stop();

}

void getCommand(char c){

if (c=='?') ReceiveState=1;

if ((c==' ')||(c=='\r')||(c=='\n')) ReceiveState=0;

if (ReceiveState==1) {

Command=Command+String(c);

if (c=='=') cmdState=0;

if (c==';') strState++;

if ((cmdState==1)&&((c!='?')||(questionstate==1))) cmd=cmd+String(c);

if ((cmdState==0)&&(strState==1)&&((c!='=')||(equalstate==1))) P1=P1+String(c);

if ((cmdState==0)&&(strState==2)&&(c!=';')) P2=P2+String(c);

if ((cmdState==0)&&(strState==3)&&(c!=';')) P3=P3+String(c);

if ((cmdState==0)&&(strState==4)&&(c!=';')) P4=P4+String(c);

if ((cmdState==0)&&(strState==5)&&(c!=';')) P5=P5+String(c);

if ((cmdState==0)&&(strState==6)&&(c!=';')) P6=P6+String(c);

if ((cmdState==0)&&(strState==7)&&(c!=';')) P7=P7+String(c);

if ((cmdState==0)&&(strState==8)&&(c!=';')) P8=P8+String(c);

if ((cmdState==0)&&(strState>=9)&&((c!=';')||(semicolonstate==1))) P9=P9+String(c);

if (c=='?') questionstate=1;

if (c=='=') equalstate=1;

if ((strState>=9)&&(c==';')) semicolonstate=1;

}

4. Kemudian, buka tab baru di Arduino IDE seperti yang ditunjukkan pada gambar berikut.

Kemudian beri nama index_OCV_ColorTrack.h.

Salin kode berikut ke dalam file tersebut.

/****************************

This include file, index_OCV_ColorTrack.h, the Client, is an intoduction of OpenCV.js to the ESP32 Camera environment. The Client was

developed and written by Andrew R. Sass. Permission to reproduce the index_OCV_ColorTrack.h file is granted free of charge if this

entire copyright notice is included in all copies of the index_OCV_ColorTrack.h file.

Complete instructions at https://RandomNerdTutorials.com/esp32-cam-opencv-js-color-detection-tracking/

*******************************/

static const char PROGMEM INDEX_HTML[] = R"rawliteral(

<!DOCTYPE html>

<html>

<head>

<title>ESP32-CAMERA COLOR DETECTION</title>

</head>

<style>

html {

font-family: Arial, Helvetica, sans-serif;

}

body {

background-color: #F7F7F2;

margin: 0px;

}

h1 {

font-size: 1.6rem;

color:white;

text-align: center;

}

.topnav {

overflow: hidden;

background-color: #0A1128;

}

.main-controls{

padding-top: 5px;

}

h2 {

color: #0A1128;

font-size: 1rem;

}

.section {

margin: 2px;

padding: 10px;

}

.column{

float: left;

width: 50%

}

table {

margin: 0;

width: 90%;

border-collapse: collapse;

}

th{

text-align: center;

}

.row{

margin-right:50px;

margin-left:50px;

}

#colorDetect{

border: none;

color: #FEFCFB;

background-color: #0A1128;

padding: 15px;

text-align: center;

display: inline-block;

font-size: 16px;

border-radius: 4px;

}

#restart{

border: none;

color: #FEFCFB;

background-color: #7B0828;

padding: 15px;

text-align: center;

display: inline-block;

font-size: 16px;

border-radius: 4px;

}

button{

border: none;

color: #FEFCFB;

background-color: #0A1128;

padding: 10px;

text-align: center;

display: inline-block;

border-radius: 4px;

}

</style>

<body>

<h1>ESP32-CAM Color Detection and Tracking</h1>

</div>

<table>

<tr>

</tr>

</table>

</div>

<h2>Video Streaming</h2>

</div>

<table>

<tr>

<td>Quality</td>

</tr>

<tr>

<td>Brightness</td>

</tr>

<tr>

<td>Contrast</td>

</tr>

</table>

</div>

<h2>RGB Color Trackbars</h2>

<table>

<tr>

</tr>

<tr>

</tr>

<tr>

</tr>

</table>

</div>

<h2>Threshold Minimum-Binary Image</h2>

<table>

<tr>

<td>Minimum Threshold:   <span id="THRESH_MINdemo"></span></td>

</tr>

</table>

</div>

<h2>Color Probe</h2>

<table>

<tr>

<td>X probe:   <span id="X_PROBEdemo"></span></td>

<td>Y probe:   <span id="Y_PROBEdemo"></span></td>

</tr>

</table>

</div>

</div>

<h2>Image Mask</h2>

</div>

<h2>Image Canvas</h2>

</div>

<table>

<tr>

<td><button type="button" id="invertButton" class="btn btn-primary">INVERT</button></td>

<td><button type="button" id="contourButton" class="btn btn-primary">SHOW CONTOUR</button></td>

<td><button type="button" id="trackButton" class="btn btn-primary">TRACKING</button></td>

</tr>

<tr>

<td>Invert: <span id="INVERTdemo"></span></td>

<td>Contour: <span id="CONTOURdemo"></span></td>

<td>Track: <span id="TRACKdemo"></span>

</td>

</tr>

</table>

</div>

<table>

<tr>

</tr>

</table>

</div>

</div>

</div>

</div>

</div>

var colorDetect = document.getElementById('colorDetect');

var ShowImage = document.getElementById('ShowImage');

var canvas = document.getElementById("canvas");

var context = canvas.getContext("2d");

var imageMask = document.getElementById("imageMask");

var imageMaskContext = imageMask.getContext("2d");

var imageCanvas = document.getElementById("imageCanvas");

var imageContext = imageCanvas.getContext("2d");

var txtcanvas = document.getElementById("textCanvas");

var ctx = txtcanvas.getContext("2d");

var message = document.getElementById('message');

var ifr = document.getElementById('ifr');

var myTimer;

var restartCount=0;

const modelPath = 'https://ruisantosdotme.github.io/face-api.js/weights/';

let currentStream;

let displaySize = { width:400, height: 296 }

let faceDetection;

let b_tracker = false;

let x_cm = 0;

let y_cm = 0;

let b_invert = false;

let b_contour = false;

var RMAX=50;

var RMIN=0;

var GMAX=50;

var GMIN=0;

var BMAX=50;

var BMIN=0;

var THRESH_MIN=120;

var X_PROBE=200;

var Y_PROBE=196;

var R=0;

var G=0;

var B=0;

var A=0;

colorDetect.onclick = function (event) {

clearInterval(myTimer);

myTimer = setInterval(function(){error_handle();},5000);

ShowImage.src=location.origin+'/?colorDetect='+Math.random();

}

//ANN:READY

var Module = {

onRuntimeInitialized(){onOpenCvReady();}

}

function onOpenCvReady(){

//alert("onOpenCvReady");

console.log("OpenCV IS READY!!!");

drawReadyText();

document.body.classList.remove("loading");

}

function error_handle() {

restartCount++;

clearInterval(myTimer);

if (restartCount<=2) {

message.innerHTML = "Get still error. <br>Restart ESP32-CAM "+restartCount+" times.";

myTimer = setInterval(function(){colorDetect.click();},10000);

ifr.src = document.location.origin+'?restart';

}

else

message.innerHTML = "Get still error. <br>Please close the page and check ESP32-CAM.";

}

colorDetect.style.display = "block";

ShowImage.onload = function (event) {

//alert("SHOW IMAGE");

console.log("SHOW iMAGE");

clearInterval(myTimer);

restartCount=0;

canvas.setAttribute("width", ShowImage.width);

canvas.setAttribute("height", ShowImage.height);

canvas.style.display = "block";

imageCanvas.setAttribute("width", ShowImage.width);

imageCanvas.setAttribute("height", ShowImage.height);

imageCanvas.style.display = "block";

imageMask.setAttribute("width", ShowImage.width);

imageMask.setAttribute("height", ShowImage.height);

imageMask.style.display = "block";

context.drawImage(ShowImage,0,0,ShowImage.width,ShowImage.height);

DetectImage();

}

restart.onclick = function (event) {

fetch(location.origin+'/?restart=stop');

}

quality.onclick = function (event) {

fetch(document.location.origin+'/?quality='+this.value+';stop');

}

brightness.onclick = function (event) {

fetch(document.location.origin+'/?brightness='+this.value+';stop');

}

contrast.onclick = function (event) {

fetch(document.location.origin+'/?contrast='+this.value+';stop');

}

async function DetectImage() {

//alert("DETECT IMAGE");

console.log("DETECT IMAGE");

/***************opencv********************************/

//ANN:4

let src = cv.imread(ShowImage);

arows = src.rows;

acols = src.cols;

aarea = arows*acols;

adepth = src.depth();

atype = src.type();

achannels = src.channels();

console.log("rows = " + arows);

console.log("cols = " + acols);

console.log("pic area = " + aarea);

console.log("depth = " + adepth);

console.log("type = " + atype);

console.log("channels = " + achannels);

/******************COLOR DETECT******************************/

//ANN:6

var RMAXslider = document.getElementById("rmax");

var RMAXoutput = document.getElementById("RMAXdemo");

RMAXoutput.innerHTML = RMAXslider.value;

RMAXslider.oninput = function(){

RMAXoutput.innerHTML = this.value;

RMAX = parseInt(RMAXoutput.innerHTML,10);

console.log("RMAX=" + RMAX);

}

console.log("RMAX=" + RMAX);

var RMINslider = document.getElementById("rmin");

var RMINoutput = document.getElementById("RMINdemo");

RMINoutput.innerHTML = RMINslider.value;

RMINslider.oninput = function(){

RMINoutput.innerHTML = this.value;

RMIN = parseInt(RMINoutput.innerHTML,10);

console.log("RMIN=" + RMIN);

}

console.log("RMIN=" + RMIN);

var GMAXslider = document.getElementById("gmax");

var GMAXoutput = document.getElementById("GMAXdemo");

GMAXoutput.innerHTML = GMAXslider.value;

GMAXslider.oninput = function(){

GMAXoutput.innerHTML = this.value;

GMAX = parseInt(GMAXoutput.innerHTML,10);

}

console.log("GMAX=" + GMAX);

var GMINslider = document.getElementById("gmin");

var GMINoutput = document.getElementById("GMINdemo");

GMINoutput.innerHTML = GMINslider.value;

GMINslider.oninput = function(){

GMINoutput.innerHTML = this.value;

GMIN = parseInt(GMINoutput.innerHTML,10);

}

console.log("GMIN=" + GMIN);

var BMAXslider = document.getElementById("bmax");

var BMAXoutput = document.getElementById("BMAXdemo");

BMAXoutput.innerHTML = BMAXslider.value;

BMAXslider.oninput = function(){

BMAXoutput.innerHTML = this.value;

BMAX = parseInt(BMAXoutput.innerHTML,10);

}

console.log("BMAX=" + BMAX);

var BMINslider = document.getElementById("bmin");

var BMINoutput = document.getElementById("BMINdemo");

BMINoutput.innerHTML = BMINslider.value;

BMINslider.oninput = function(){

BMINoutput.innerHTML = this.value;

BMIN = parseInt(BMINoutput.innerHTML,10);

}

console.log("BMIN=" + BMIN);

var THRESH_MINslider = document.getElementById("thresh_min");

var THRESH_MINoutput = document.getElementById("THRESH_MINdemo");

THRESH_MINoutput.innerHTML = THRESH_MINslider.value;

THRESH_MINslider.oninput = function(){

THRESH_MINoutput.innerHTML = this.value;

THRESH_MIN = parseInt(THRESH_MINoutput.innerHTML,10);

}

console.log("THRESHOLD MIN=" + THRESH_MIN);

//ANN:9A

var X_PROBEslider = document.getElementById("x_probe");

var X_PROBEoutput = document.getElementById("X_PROBEdemo");

X_PROBEoutput.innerHTML = X_PROBEslider.value;

X_PROBEslider.oninput = function(){

X_PROBEoutput.innerHTML = this.value;

X_PROBE = parseInt(X_PROBEoutput.innerHTML,10);

}

console.log("X_PROBE=" + X_PROBE);

var Y_PROBEslider = document.getElementById("y_probe");

var Y_PROBEoutput = document.getElementById("Y_PROBEdemo");

Y_PROBEoutput.innerHTML = Y_PROBEslider.value;

Y_PROBEslider.oninput = function(){

Y_PROBEoutput.innerHTML = this.value;

Y_PROBE = parseInt(Y_PROBEoutput.innerHTML,10);

}

console.log("Y_PROBE=" + Y_PROBE);

document.getElementById('trackButton').onclick = function(){

b_tracker = (true && !b_tracker)

console.log("TRACKER = " + b_tracker );

var TRACKoutput = document.getElementById("TRACKdemo");

TRACKoutput.innerHTML = b_tracker;

//var XCMoutput = document.getElementById("XCMdemo");

//XCMoutput.innerHTML = x_cm;

}

document.getElementById('invertButton').onclick = function(){

b_invert = (true && !b_invert)

console.log("TRACKER = " + b_invert );

var INVERToutput = document.getElementById("INVERTdemo");

INVERToutput.innerHTML = b_invert;

}

/**/

document.getElementById('contourButton').onclick = function(){

b_contour = (true && !b_contour)

console.log("TRACKER = " + b_contour );

var CONTOURoutput = document.getElementById("CONTOURdemo");

CONTOURoutput.innerHTML = b_contour;

}

/**/

let tracker = 0;

var TRACKoutput = document.getElementById("TRACKdemo");

TRACKoutput.innerHTML = b_tracker;

var XCMoutput = document.getElementById("XCMdemo");

var YCMoutput = document.getElementById("YCMdemo");

XCMoutput.innerHTML = 0;

YCMoutput.innerHTML = 0;

var INVERToutput = document.getElementById("INVERTdemo");

INVERToutput.innerHTML = b_invert;

var CONTOURoutput = document.getElementById("CONTOURdemo");

CONTOURoutput.innerHTML = b_contour;

//ANN:8

let M00Array = [0,];

let orig = new cv.Mat();

let mask = new cv.Mat();

let mask1 = new cv.Mat();

let mask2 = new cv.Mat();

let contours = new cv.MatVector();

let hierarchy = new cv.Mat();

let rgbaPlanes = new cv.MatVector();

let color = new cv.Scalar(0,0,0);

clear_canvas();

orig = cv.imread(ShowImage);

cv.split(orig,rgbaPlanes); //SPLIT

let BP = rgbaPlanes.get(2); // SELECTED COLOR PLANE

let GP = rgbaPlanes.get(1);

let RP = rgbaPlanes.get(0);

cv.merge(rgbaPlanes,orig);

// BLK BLU GRN RED

let row = Y_PROBE //180//275 //225 //150 //130

let col = X_PROBE //100//10 //100 //200 //300

drawColRowText(acols,arows);

console.log("ISCONTINUOUS = " + orig.isContinuous());

//ANN:9C

R = src.data[row * src.cols * src.channels() + col * src.channels()];

G = src.data[row * src.cols * src.channels() + col * src.channels() + 1];

B = src.data[row * src.cols * src.channels() + col * src.channels() + 2];

A = src.data[row * src.cols * src.channels() + col * src.channels() + 3];

console.log("RDATA = " + R);

console.log("GDATA = " + G);

console.log("BDATA = " + B);

console.log("ADATA = " + A);

drawRGB_PROBE_Text();

//ANN:9b

//*************draw probe point*********************

let point4 = new cv.Point(col,row);

cv.circle(src,point4,5,[255,255,255,255],2,cv.LINE_AA,0);

//***********end draw probe point*********************

//ANN:7

let high = new cv.Mat(src.rows,src.cols,src.type(),[RMAX,GMAX,BMAX,255]);

let low = new cv.Mat(src.rows,src.cols,src.type(),[RMIN,GMIN,BMIN,0]);

cv.inRange(src,low,high,mask1);

//inRange(source image, lower limit, higher limit, destination image)

cv.threshold(mask1,mask,THRESH_MIN,255,cv.THRESH_BINARY);

//threshold(source image,destination image,threshold,255,threshold method);

//ANN:9

if(b_invert==true){

cv.bitwise_not(mask,mask2);

}

/********************start contours******************************************/

//ANN:10

if(b_tracker == true){

try{

if(b_invert==false){

//ANN:11

cv.findContours(mask,contours,hierarchy,cv.RETR_CCOMP,cv.CHAIN_APPROX_SIMPLE);

//findContours(source image, array of contours found, hierarchy of contours

// if contours are inside other contours, method of contour data retrieval,

//algorithm method)

}

else{

cv.findContours(mask2,contours,hierarchy,cv.RETR_CCOMP,cv.CHAIN_APPROX_SIMPLE);

}

console.log("CONTOUR_SIZE = " + contours.size());

//draw contours

if(b_contour==true){

for(let i = 0; i < contours.size(); i++){

cv.drawContours(src,contours,i,[0,0,0,255],2,cv.LINE_8,hierarchy,100)

}

//ANN:12

let cnt;

let Moments;

let M00;

let M10;

//let x_cm;

//let y_cm;

//ANN:13

for(let k = 0; k < contours.size(); k++){

cnt = contours.get(k);

Moments = cv.moments(cnt,false);

M00Array[k] = Moments.m00;

// cnt.delete();

}

//ANN13A

let max_area_arg = MaxAreaArg(M00Array);

console.log("MAXAREAARG = "+max_area_arg);

//let TestArray = [0,0,0,15,4,15,2];

//let TestArray0 = [];

//let max_test_area_arg = MaxAreaArg(TestArray0);

//console.log("MAXTESTAREAARG = "+max_test_area_arg);

let ArgMaxArea = MaxAreaArg(M00Array);

if(ArgMaxArea >= 0){

cnt = contours.get(MaxAreaArg(M00Array)); //use the contour with biggest MOO

//cnt = contours.get(54);

Moments = cv.moments(cnt,false);

M00 = Moments.m00;

M10 = Moments.m10;

M01 = Moments.m01;

x_cm = M10/M00; // 75 for circle_9.jpg

y_cm = M01/M00; // 41 for circle_9.jpg

XCMoutput.innerHTML = Math.round(x_cm);

YCMoutput.innerHTML = Math.round(y_cm);

console.log("M00 = "+M00);

console.log("XCM = "+Math.round(x_cm));

console.log("YCM = "+Math.round(y_cm));

//fetch(document.location.origin+'/?xcm='+Math.round(x_cm)+';stop');

fetch(document.location.origin+'/?cm='+Math.round(x_cm)+';'+Math.round(y_cm)+';stop');

console.log("M00ARRAY = " + M00Array);

//ANN:14

//**************min area bounding rect********************

//let rotatedRect=cv.minAreaRect(cnt);

//let vertices = cv.RotatedRect.points(rotatedRect);

//for(let j=0;j<4;j++){

// cv.line(src,vertices[j],

// vertices[(j+1)%4],[0,0,255,255],2,cv.LINE_AA,0);

//}

//***************end min area bounding rect*************************************

//***************bounding rect***************************

let rect = cv.boundingRect(cnt);

let point1 = new cv.Point(rect.x,rect.y);

let point2 = new cv.Point(rect.x+rect.width,rect.y+rect.height);

cv.rectangle(src,point1,point2,[0,0,255,255],2,cv.LINE_AA,0);

//*************end bounding rect***************************

//*************draw center point*********************

let point3 = new cv.Point(x_cm,y_cm);

cv.circle(src,point3,2,[0,0,255,255],2,cv.LINE_AA,0);

//***********end draw center point*********************

}//end if(ArgMaxArea >= 0)

else{

if(ArgMaxArea==-1){

console.log("ZERO ARRAY LENGTH");

}

else{ //ArgMaxArea=-2

console.log("DUPLICATE MAX ARRAY-ELEMENT");

}

cnt.delete();

/******************end contours note cnt line one up*******************************************/

drawXCM_YCM_Text();

}//end try

catch{

console.log("ERROR TRACKER NO CONTOUR");

clear_canvas();

drawErrorTracking_Text();

}

}//end b_tracking if statement

else{

XCMoutput.innerHTML = 0;

YCMoutput.innerHTML = 0;

}

if(b_invert==false){

cv.imshow('imageMask', mask);

}

else{

cv.imshow('imageMask', mask2);

}

//cv.imshow('imageMask', R);

cv.imshow('imageCanvas', src);

//ANN:8A

src.delete();

high.delete();

low.delete();

orig.delete();

mask1.delete();

mask2.delete();

mask.delete();

contours.delete();

hierarchy.delete();

//cnt.delete();

RP.delete();

/********************END COLOR DETECT****************************/

/***************end opencv******************************/

setTimeout(function(){colorDetect.click();},500);

}//end detectimage

function MaxAreaArg(arr){

if (arr.length == 0) {

return -1;

}

var max = arr[0];

var maxIndex = 0;

var dupIndexCount = 0; //duplicate max elements?

if(arr[0] >= .90*aarea){

max = 0;

}

for (var i = 1; i < arr.length; i++) {

if (arr[i] > max && arr[i] < .99*aarea) {

maxIndex = i;

max = arr[i];

dupIndexCount = 0;

}

else if(arr[i]==max && arr[i]!=0){

dupIndexCount++;

}

if(dupIndexCount==0){

return maxIndex;

}

else{

return -2;

}

}//end MaxAreaArg

function clear_canvas(){

ctx.clearRect(0,0,txtcanvas.width,txtcanvas.height);

ctx.rect(0,0,txtcanvas.width,txtcanvas.height);

ctx.fillStyle="red";

ctx.fill();

}

function drawReadyText(){

ctx.fillStyle = 'black';

ctx.font = '20px serif';

ctx.fillText('OpenCV.JS READY',txtcanvas.width/4,txtcanvas.height/10);

}

function drawColRowText(x,y){

ctx.fillStyle = 'black';

ctx.font = '20px serif';

ctx.fillText('ImageCols='+x,0,txtcanvas.height/10);

ctx.fillText('ImageRows='+y,txtcanvas.width/2,txtcanvas.height/10);

}

function drawRGB_PROBE_Text(){

ctx.fillStyle = 'black';

ctx.font = '20px serif';

ctx.fillText('Rp='+R,0,2*txtcanvas.height/10);

ctx.fillText('Gp='+G,txtcanvas.width/4,2*txtcanvas.height/10);

ctx.fillText('Bp='+B,txtcanvas.width/2,2*txtcanvas.height/10);

ctx.fillText('Ap='+A,3*txtcanvas.width/4,2*txtcanvas.height/10);

}

function drawXCM_YCM_Text(){

ctx.fillStyle = 'black';

ctx.font = '20px serif';

ctx.fillText('XCM='+Math.round(x_cm),0,3*txtcanvas.height/10);

ctx.fillText('YCM='+Math.round(y_cm),txtcanvas.width/4,3*txtcanvas.height/10);

}

function drawErrorTracking_Text(){

ctx.fillStyle = 'black';

ctx.font = '20px serif';

ctx.fillText('ERROR TRACKING-NO CONTOUR',0,3*txtcanvas.height/10);

}

</script>

</body>

</html>

)rawliteral";

5. Kemudian Run, dan hasil akan muncul di Serial Monitor.

V. Hasil Praktikum

VI. Analisa

Pada praktikum kali ini kami melakukan percobaan untuk mendeteksi dan melacak warna pada objek menggunakan ESP32CAM Server. Pertama kami mengcopy code pada software Arduino IDE dengan 2 code yang tersedia dan jangan lupa untuk menghubungkan port ESP32Cam dengan Arduino IDE. Jika sudah berhasil code diupload maka pada bagian serial monitor akan muncul link website yang dapat kita buka pada browser untuk melakukan test pelacakan dan deteksi warna pada objek. Setelah web terbuka pada browser, maka kita dapat melakukan percobaannya dengan mengklik tombol “Color Detection” untuk mengambil gambar menggunakan kamera pada ESP32Cam, setelahnya kita mensetting nilai-nilai RGB sesuai yang diinginkan untuk melacak dan mendeteksi warna tertentu, disini kita akan mendeteksi warna biru pada objek. Jika sudah setting nilai RGB, selanjutya arahkan kamera ESP32Cam pada objek, disini kami menggunakan objek sebuah tutup botol yang terdapat warna biru. Karena kita akan mendeteksi dan melacak warna biru maka ESP32Cam hanya mendeteksi warna biru pada tutup botol dengan hasil deteksi warna dapat dilihat pada bagian Image Canvas dan hasil lacak warna dapat dilihat pada bagian Image Mask.

Search This Blog

Komputasi Pervasive

ESP32 CAM Server untuk Deteksi dan Lacak Warna

Comments

Post a Comment

Popular posts from this blog

LAPORAN PRAKTIKUM - Sistem Deteksi Objek dengan ESP32-CAM dan Tensorflow