Skip to main content

Buffing the unwrapper with a face recognition algorithm

Buffing the unwrapper with a face recognition algorithm

Boosting my previous code with Machine Learning in order to obtain a more precise result of the unwrapped face, the result is not necessarily better tho.



In fact, as you can see the result is quite disturbing.

Just a quick note before continuing, I am learning all of this with YouTube, reading the OpenCV docs which btw are really explicative and simple to understand, and googling stuff in StackOverflow that basically someone has asked before. I will add some resources at the end of the entry.

*This is a continuation of what I explained in my previous entry, so if you want a bit of background, go check that one out* (https://achefethings.blogspot.com/2021/03/unwrapping-someones-face-to-make.html)

So now what I did, I was watching a YouTube video and saw that you can do face recognition, among many other features such as car plates, eyes, hands, and many other cool things, so my first idea was to do some kind of alarm that would recognize household members (to do so it would be needed to have it trained with a large number of photos of each of the members that you want it to recognize) and to have this alarm system connected with a Whatsapp API that would send you a message every time someones not recognized enters the house or for instance to know who is at home at each moment. I think this will come in future entries, but for now, let's come back to the unwrapper.

Then while I was running a bit I thought that face recognition would make for a very cool way to improve my previous code's flaws. Being its main disadvantage that the video of the head-spinning would need to be right in the middle of the screen, or otherwise, it would just print out horrible things or at least not funny ones.

So that's what I did, I first learn how to use these libraries and I then implemented into my previous code. Here you have it fully in case you want some or inspiration, are curious about how it is done or simply want to try it for yourself.

import cv2 as cv
import numpy as np
import os

video = cv.VideoCapture('C:/Users/Usuario/PycharmProjects/wrapping/video/video.mp4')

success,image = video.read()
haar_cascade = cv.CascadeClassifier('haar_face.xml')

count = 0

while success:
  cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/frame/frame%d.jpg" % count, image)     # save frame as JPEG file
  success,image = video.read()
  count += 1
  print('Read Frame #', count)

numero = list(range(0,215))
j = 0


for i in numero:
  lectura = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/frame/frame" + str(i) + ".jpg")
  gray = cv.cvtColor(lectura,cv.COLOR_BGR2GRAY)
  haar_cascade = cv.CascadeClassifier('haar_face.xml')
  faces_rect = haar_cascade.detectMultiScale(gray, scaleFactor=1.1,minNeighbors=6)

  for (x,y,w,h) in faces_rect:
    cv.rectangle(lectura,(x,y),(x+w,y+h),(0,1,0),thickness=1)

  img_crop = lectura[(y-100):(y+h+300),(x-100):(x+w+300)]
  cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle%d.jpg" % i, img_crop)
  print("Face Frame #:", i)

frames_validos = []

for i in numero:
  lectura = "C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle" + str(i) + ".jpg"
  img_original = cv.imread(lectura)
  dimensions = img_original.shape

  [y, h, x ,w ] = [0,int(dimensions[0]),int((int(dimensions[1])/2)-1),1]
  img_crop = img_original[y:h,x:x+w]

  dim = img_crop.shape
  if dim[0] > 350:
    resized = cv.resize(img_crop, (1,400), interpolation=cv.INTER_AREA)
    cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/crop/crop%d.jpg" % i,resized)
    frames_validos.append(resized)
    print(f'Frame #: {i} added')
    j = j + 1
  else:
    os.remove("C:/Users/Usuario/PycharmProjects/wrapping/rectangle/rectangle" + str(i) + ".jpg")
    print(f'Frame #: {i} killed')

print(f'Number of frames added: {j}')
crop = []
n=0
for n in frames_validos:
  new_resize = cv.resize(n,(1,400),interpolation=cv.INTER_AREA)
  # cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/crop/crop%d.jpg" % n,new_resize)
  crop.append(new_resize)

im_h_1 = cv.hconcat(crop)
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/forward.jpg", im_h_1)
print("Concat Done")

im_h_2 = cv.flip(im_h_1, 1)
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/backward.jpg", im_h_2)


forward = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/forward.jpg")
backward = cv.imread("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/backward.jpg")

im_final = cv.hconcat([backward,forward])
cv.imwrite("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/final.jpg", im_final)
cv.imshow("C:/Users/Usuario/PycharmProjects/wrapping/nuevo/final.jpg",im_final)


cv.waitKey(0)
cv.destroyAllWindows()

So, the first thing that changes with respect to the previous one is that top I bring this haar_face.xml file that you can find here (https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml) among many other recognition file thingies that I quite don't understand yet. I then apply the classifier which to the frames of my video together with the .xml file, this would create a rectangle where the classifier thinks my face is.

Since I found out that this rectangle was often too small in comparison to my actual face, I added some margins around it so the final result looks a bit nicer, and many other times the software would not be able to recognize me or simply it would locate my face somewhere else.*

Here a photo of a directory with many Pablo's recognized, and some that have not been recognized.


I had to filter all the "locations" of my face to discard the ones that were wrong, an if statement with some dimensions was enough to do it. And since the size of the detected faces in each frame would be different I had to resize them for them to have all the same size, otherwise, the concatenation thing would have failed.

From that point on, the code is basically the same, except for the fact that now it cut rights in the middle of my detected face instead of in the middle of the original frame.

Here the video I used to compose the picture above with my face being detected at every moment.


Before finishing, I will leave you here a link to all the files that I've used and that you will need in case that you want to try it out by yourself.

Here you go: https://drive.google.com/file/d/1wdMN5EicW1BgkARK_nV_8JNHPd95p4lF/view?usp=sharing

And some links to the resources that have helped me so much to my stupid projects (at least I'm learning, ok? it's that much of a waste of time):

  • https://www.youtube.com/watch?v=oXlwWbU8l2o&t=9863s (look for the Face Recognition section in the comments box)

  • https://docs.opencv.org/master/d6/d00/tutorial_py_root.html
  • https://es.stackoverflow.com/

So, hope you like this entry, for the next one I am preparing something with 3D printing, something useful and not those figurines and shit like that that people that do not knowhow of 3D printing think when talking about it. I'm also preparing for my Master Thesis defense, which will be next Monday, hopefully, when you hear about me again I have already graduated.

Bye, thanks for reading me, if you arrived here, I love you.


*Funny note here: I was trying to do it first with my bear but I found out that this would mislead the program so I proceeded to shave.

Labels:
Location: Castro Urdiales, Cantabria, España

Comments

Post a Comment

Popular posts from this blog

Advent of Code, day 1

 Day 1 of the Advent of Code A bit late for Christmas, isn't it? It's been a long time since I did my last post, some days of boredom have inspired me to get back at it. I am going to be trying to solve the last Christmas' Advent of Code. For those that you don't know the Advent of Code is an Advent calendar of small programming puzzles that can be solved the way you like, each day contains two puzzles in which the first is linked to the second. If you want to get more info about it, check out this link:  https://adventofcode.com/2023/about Without further ado, let's get at it, I'm going to copy down below the statement for the Day 1 Statement Input Basically we are given a long list of characters (the one shown in the picture) where each line contains numbers and letters, we first need to get just the numbers, then store somewhere else the first and last numbers in each row, and lastly sum all those values. A link to the input file:  https://adventofcode.com/20...
Post a Comment
Read more

A first approach to IoT, connecting my 3D printer to the internet

My first approach to the IoT, connecting my 3D printer to the internet IoT is one of those fancy words that people like to talk about in conferences and in TedTalks without (apparently) having too much idea of what it is all about. Set up to manage the 3D printer through the internet This one is going to be a short entry where I don't go through code or anything, just wanted to talk about a bit about how I connected my 3D printer to the internet.  I've been in the 3D printing thing for a while now, about a year and I haven't stopped printing ever since I bought my Ender 3. Fortunately enough, I live in a big house where my room/working place is on the fourth floor and my 3D printing is on the first one. You might be thinking as well: "OK Pablo but where do you want to bring us? Go to the point" Well, as you might have noticed there are two floors in betw...
Post a Comment
Read more

Advent of Code, day 2

We made it to Day 2 of Advent of Code I have kept my promise for 2 days in a row. As weird as it seems for this day and age, I have kept my promise and today is the day 2 of the Advent of Code, this time with elves involved. There you go, the statement of today's first problem down below: And usual, my solution: import pandas as pd import re import numpy as np filepath = "/content/input 1.txt" with open (filepath, 'r' ) as file :     lines = [line.strip() for line in file .readlines()] lines_split = [] for line in lines:   token = re.findall(r '\w+|[^\s\w]' , line)   lines_split.append(token) blue_list = [] green_list = [] red_list = [] for line in lines_split:   blue,green,red = 0 , 0 , 0   for ix, word in enumerate (line):     if word == "blue" :       if blue < int (line[ix -1 ]):         blue = int (line[ix -1 ])     if word == "green" :       if green < int (...
Post a Comment
Read more