Skip to main content

Buffing the unwrapper with a face recognition algorithm

Buffing the unwrapper with a face recognition algorithm

Boosting my previous code with Machine Learning in order to obtain a more precise result of the unwrapped face, the result is not necessarily better tho.



In fact, as you can see the result is quite disturbing.

Just a quick note before continuing, I am learning all of this with YouTube, reading the OpenCV docs which btw are really explicative and simple to understand, and googling stuff in StackOverflow that basically someone has asked before. I will add some resources at the end of the entry.

*This is a continuation of what I explained in my previous entry, so if you want a bit of background, go check that one out* (https://achefethings.blogspot.com/2021/03/unwrapping-someones-face-to-make.html)

So now what I did, I was watching a YouTube video and saw that you can do face recognition, among many other features such as car plates, eyes, hands, and many other cool things, so my first idea was to do some kind of alarm that would recognize household members (to do so it would be needed to have it trained with a large number of photos of each of the members that you want it to recognize) and to have this alarm system connected with a Whatsapp API that would send you a message every time someones not recognized enters the house or for instance to know who is at home at each moment. I think this will come in future entries, but for now, let's come back to the unwrapper.

Then while I was running a bit I thought that face recognition would make for a very cool way to improve my previous code's flaws. Being its main disadvantage that the video of the head-spinning would need to be right in the middle of the screen, or otherwise, it would just print out horrible things or at least not funny ones.

So that's what I did, I first learn how to use these libraries and I then implemented into my previous code. Here you have it fully in case you want some or inspiration, are curious about how it is done or simply want to try it for yourself.

import cv2 as cv
import numpy as np
import os

video = cv.VideoCapture('C:/Users/Usuario/PycharmProjects/wrapping/video/video.mp4')

success,image = video.read()
haar_cascade = cv.CascadeClassifier('haar_face.xml')

count = 0

while success:
  cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/frame/frame%d.jpg" % count, image)     # save frame as JPEG file
  success,image = video.read()
  count += 1
  print('Read Frame #', count)

numero = list(range(0,215))
j = 0


for i in numero:
  lectura = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/frame/frame" + str(i) + ".jpg")
  gray = cv.cvtColor(lectura,cv.COLOR_BGR2GRAY)
  haar_cascade = cv.CascadeClassifier('haar_face.xml')
  faces_rect = haar_cascade.detectMultiScale(gray, scaleFactor=1.1,minNeighbors=6)

  for (x,y,w,h) in faces_rect:
    cv.rectangle(lectura,(x,y),(x+w,y+h),(0,1,0),thickness=1)

  img_crop = lectura[(y-100):(y+h+300),(x-100):(x+w+300)]
  cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle%d.jpg" % i, img_crop)
  print("Face Frame #:", i)

frames_validos = []

for i in numero:
  lectura = "C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle" + str(i) + ".jpg"
  img_original = cv.imread(lectura)
  dimensions = img_original.shape

  [y, h, x ,w ] = [0,int(dimensions[0]),int((int(dimensions[1])/2)-1),1]
  img_crop = img_original[y:h,x:x+w]

  dim = img_crop.shape
  if dim[0] > 350:
    resized = cv.resize(img_crop, (1,400), interpolation=cv.INTER_AREA)
    cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/crop/crop%d.jpg" % i,resized)
    frames_validos.append(resized)
    print(f'Frame #: {i} added')
    j = j + 1
  else:
    os.remove("C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle" + str(i) + ".jpg")
    print(f'Frame #: {i} killed')

print(f'Number of frames added: {j}')
crop = []
n=0
for n in frames_validos:
  new_resize = cv.resize(n,(1,400),interpolation=cv.INTER_AREA)
  # cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/crop/crop%d.jpg" % n,new_resize)
  crop.append(new_resize)

im_h_1 = cv.hconcat(crop)
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/forward.jpg", im_h_1)
print("Concat Done")

im_h_2 = cv.flip(im_h_1, 1)
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/backward.jpg", im_h_2)


forward = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/forward.jpg")
backward = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/backward.jpg")

im_final = cv.hconcat([backward,forward])
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/final.jpg", im_final)
cv.imshow("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/final.jpg",im_final)


cv.waitKey(0)
cv.destroyAllWindows()

So, the first thing that changes with respect to the previous one is that top I bring this haar_face.xml file that you can find here (https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml) among many other recognition file thingies that I quite don't understand yet. I then apply the classifier which to the frames of my video together with the .xml file, this would create a rectangle where the classifier thinks my face is.

Since I found out that this rectangle was often too small in comparison to my actual face, I added some margins around it so the final result looks a bit nicer, and many other times the software would not be able to recognize me or simply it would locate my face somewhere else.*

Here a photo of a directory with many Pablo's recognized, and some that have not been recognized.


I had to filter all the "locations" of my face to discard the ones that were wrong, an if statement with some dimensions was enough to do it. And since the size of the detected faces in each frame would be different I had to resize them for them to have all the same size, otherwise, the concatenation thing would have failed.

From that point on, the code is basically the same, except for the fact that now it cut rights in the middle of my detected face instead of in the middle of the original frame.

Here the video I used to compose the picture above with my face being detected at every moment.


Before finishing, I will leave you here a link to all the files that I've used and that you will need in case that you want to try it out by yourself.

Here you go: https://drive.google.com/file/d/1wdMN5EicW1BgkARK_nV_8JNHPd95p4lF/view?usp=sharing

And some links to the resources that have helped me so much to my stupid projects (at least I'm learning, ok? it's that much of a waste of time):

  • https://www.youtube.com/watch?v=oXlwWbU8l2o&t=9863s (look for the Face Recognition section in the comments box)

  • https://docs.opencv.org/master/d6/d00/tutorial_py_root.html
  • https://es.stackoverflow.com/

So, hope you like this entry, for the next one I am preparing something with 3D printing, something useful and not those figurines and shit like that that people that do not knowhow of 3D printing think when talking about it. I'm also preparing for my Master Thesis defense, which will be next Monday, hopefully, when you hear about me again I have already graduated.

Bye, thanks for reading me, if you arrived here, I love you.


*Funny note here: I was trying to do it first with my bear but I found out that this would mislead the program so I proceeded to shave.

Labels:
Location: Castro Urdiales, Cantabria, España

Comments

Post a Comment

Popular posts from this blog

A first approach to IoT, connecting my 3D printer to the internet

My first approach to the IoT, connecting my 3D printer to the internet IoT is one of those fancy words that people like to talk about in conferences and in TedTalks without (apparently) having too much idea of what it is all about. Set up to manage the 3D printer through the internet This one is going to be a short entry where I don't go through code or anything, just wanted to talk about a bit about how I connected my 3D printer to the internet.  I've been in the 3D printing thing for a while now, about a year and I haven't stopped printing ever since I bought my Ender 3. Fortunately enough, I live in a big house where my room/working place is on the fourth floor and my 3D printing is on the first one. You might be thinking as well: "OK Pablo but where do you want to bring us? Go to the point" Well, as you might have noticed there are two floors in betw...
Post a Comment
Read more

Advent of Code, day 2

We made it to Day 2 of Advent of Code I have kept my promise for 2 days in a row. As weird as it seems for this day and age, I have kept my promise and today is the day 2 of the Advent of Code, this time with elves involved. There you go, the statement of today's first problem down below: And usual, my solution: import pandas as pd import re import numpy as np filepath = "/content/input 1.txt" with open (filepath, 'r' ) as file :     lines = [line.strip() for line in file .readlines()] lines_split = [] for line in lines:   token = re.findall(r '\w+|[^\s\w]' , line)   lines_split.append(token) blue_list = [] green_list = [] red_list = [] for line in lines_split:   blue,green,red = 0 , 0 , 0   for ix, word in enumerate (line):     if word == "blue" :       if blue < int (line[ix -1 ]):         blue = int (line[ix -1 ])     if word == "green" :       if green < int (...
Post a Comment
Read more

Computer vision to fight against Covid-19

Computer vision to fight against Covid-19 In this era technology can be very helpful in many ways, here is an example of how to reduce close contacts in the working place thanks to computer vision Welcome Manuela 15:04 (left side) and Not authorized Javier 15:05 (right side) There are many scientific publications that show how Covid-19 can survive in non-porous surfaces for some hours ( https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-research/surface-transmission.html ) this might be a problem especially if those surfaces are touched by a large number of people in a short period of time. That is the case of the clock-in machine that still exist in many working places all around the world, like this one below. If not most, many of them rely on physical contact for the employee to clock-in, either it is a finger-print scan, a card that you punch-in once you arrive, or NFC devices such as cards or tokens, but not many exist that use other methods such as face recognition or ...
Post a Comment
Read more