Computer vision to fight against Covid-19

In this era technology can be very helpful in many ways, here is an example of how to reduce close contacts in the working place thanks to computer vision

Welcome Manuela 15:04 (left side) and Not authorized Javier 15:05 (right side)

There are many scientific publications that show how Covid-19 can survive in non-porous surfaces for some hours (https://www.cdc.gov/coronavirus/2019-ncov/more/science-and-research/surface-transmission.html) this might be a problem especially if those surfaces are touched by a large number of people in a short period of time. That is the case of the clock-in machine that still exist in many working places all around the world, like this one below.

If not most, many of them rely on physical contact for the employee to clock-in, either it is a finger-print scan, a card that you punch-in once you arrive, or NFC devices such as cards or tokens, but not many exist that use other methods such as face recognition or QR code scanning that would avoid that close proximity that might lead to dangerous outbreaks in companies (or anywhere else).

I thought that this problem could be easily solvable with some Python scripting and a very small investment, leaving aside the Python part of it, which I'll talk about in detail later, let's focus now on the hardware that would be required.

So, for the hardware part, just a few cheap things are required, a tiny microcomputer such as a RaspberryPi or one that is focused on this kind of thing such a JetsonNano is fundamental. You can get the former for about from 30 to 80 euros depending on the performance that you are looking for (https://shop.pimoroni.com/collections/raspberry-pi) in this case the cheap one will do, and for the latter, you'll need to invest around 60 euros (https://shop.pimoroni.com/collections/raspberry-pi), then we need to add to the basket a webcam, or we can just go with the ones that these tiny processors have available for them which costs below 2 euros. So far so good, for less than 100 euros we can have everyone safe and productive.

We'd also need to add to that tiny list, the QR codes in case people want to have them printed, if not, it would be as easy as entering here (https://es.qr-code-generator.com/) and create a code with the name of each individual, then download it and have it on your phone for instance (as I do in the video that I'll show later below), like myself that I have it a picture on my WhatsApp

Now for the coding part of it, the software is relatively easy to implement, we'll need to use the ol' good libraries that I explored previously, plus some new ones:

OpenCV (https://opencv.org/) for the camera, vision, detection, and image processing
Numpy (https://numpy.org/) for dealing with matrices and math stuff
pyzbar (https://pypi.org/project/pyzbar/)to decode the QR code/bar code
pandas (https://pandas.pydata.org/) because detecting is not enough, we also need to keep a record of those entries and exits
datetime (https://docs.python.org/3/library/datetime.html) this one to time/date stamp things
openpyxl (https://openpyxl.readthedocs.io/en/stable/) and then this one to transfer all the relevant information to an Excel file because not many people know how to work with pandas but everyone can read a .xlsx file and so is much more convinient.

Code time!

There are some things that you need to know before you try to do it by yourself:

You need to have Python plus all the libraries mentioned before installed
You need to create a .txt file with the names of the people that have access to whatever the place is, so if there are 3 workers, then write their names and generate the QR codes with those names.
You need to create an empty.xlsx file that will be filled at the end of each day with the entrance and exit data of each worker

(I'll later post a link to my drive so you can download it all together if you want)

I know, this is the boring part that will likely be read by no one, but I guess that if only a single person finds it useful, then cool.

First, importing all the relevant stuff and setting the basic things, just the things I mentioned above and set the things that the camera is capturing just so people can aim correctly with their cards/QR codes/ bar codes/ face/eye or whatever.

import cv2
import numpy as np
from pyzbar.pyzbar import decode
from datetime import datetime, date
import pandas as pd
from openpyxl import load_workbook

cap = cv2.VideoCapture(0)
cap.set(3, 640)
cap.set(4, 480)

Then, define some file paths that should be changed accordingly to your computer

with open("myDataFile.txt") as f:
    myDataList = f.read().splitlines()

filepath = r"C:\Users\Usuario\PycharmProjects\presencia\presence.txt"
new_df = pd.DataFrame()

This one below is important, as it allows you to define the minimum entry time and the maximum exit time, for instance, if you want people to start working after 7:00 am then you set so in these lines

end_journey = datetime(hour=20,minute=30,year=datetime.now().year,month=datetime.now().month,day=datetime.now().day)
begin_journey = datetime(hour=7,minute=00,year=datetime.now().year,month=datetime.now().month,day=datetime.now().day)

Then this small snippet just creates a data frame that will be displayed each time there is a valid entry or exit, so let's say the employees are Pablo and Manuela, well then if someone with QR code that says Javier tries to access it will neither be registered nor shown in the screen. This is useful to give some feedback to the user, as otherwise, he wouldn't have the security of having clocked-in succesfully

def data(filepath):
    today = date.today()
    header_list = ["Name", "Date", "Time", "Time today"]
    new_df = pd.read_csv(filepath, sep=";", names=header_list, index_col=False)
    return new_df

And now, the main function, this one will operate within the working hours that you've set before anbd is in charge of the most important things, such as detecting the QR code and registering the entries and exit in the function that is pasted above. It also gives feedback to the user as it squares the QR code and writes just on it who the person is, whether is authorized or not and the entrance time.

while (end_journey >datetime.now() and (datetime.now() > begin_journey)):
    now = datetime.now()
    current_time = now.strftime("%H:%M")
    today = date.today()
    success, img = cap.read()
    for barcode in decode(img):
        myData = barcode.data.decode("UTF-8")
        if myData in myDataList:
            myOutput = "Welcome"
            myColor = (0, 255, 0)

            with open("presence.txt") as f:
                presenceList = f.read().splitlines()
                escritura = f"{myData};{today};{current_time};"

            with open("presence.txt", "a") as f:
                if (escritura not in presenceList):
                    f.write(escritura)
                    f.write("\n")
                else:
                    pass
        else:
            myOutput = "Not authorized"
            myColor = (0, 0, 255)
        pts = np.array([barcode.polygon], np.int32)
        pts = pts.reshape((-1, 1, 2))
        cv2.polylines(img, [pts], True, (255, 0, 255), 5)
        pts2 = barcode.rect
        txt = myOutput + " " + myData + " " + current_time
        cv2.putText(img, txt, (pts2[0], pts2[1]), cv2.FONT_HERSHEY_PLAIN,
                    0.9, myColor, 2)



        df = new_df
        new_df = data(filepath)
        if (df.equals(new_df) == False):
            print("\n" * 100)
            print(new_df)

And finally, the4 last function, that is in charge of moving all the entries and exit of each person to an Excel file, in which each sheet will correspond to each day automatically, the data will also be stored in a .csv file, which in the case of a very large amount of people with numerous entries and exits might be handier to work with.

fin = pd.DataFrame()
header_list = ["Name", "Date", "Time","Time today"]
df = pd.read_csv(r"C:\Users\Usuario\PycharmProjects\presencia\presence.txt",sep=";",names = header_list,index_col=False)

for names in myDataList:
    rows = df["Name"]==names
    if ((df.loc[rows].shape[0] % 2) == 0):
        my_time_1 = time.strptime(df.loc[rows]["Time"].values.tolist()[-1], '%H:%M')
        my_time_2 = time.strptime(df.loc[rows]["Time"].values.tolist()[-2], '%H:%M')
        ndf = df.loc[rows]
        if ((my_time_1.tm_min - my_time_2.tm_min) < 10):
            ndf.loc[df.index[df['Name'] == names].tolist()[
                        -1], 'Time today'] = f"{my_time_1.tm_hour - my_time_2.tm_hour}:0{my_time_1.tm_min - my_time_2.tm_min}"
        else:
            ndf.loc[df.index[df['Name'] == names].tolist()[
                        -1], 'Time today'] = f"{my_time_1.tm_hour - my_time_2.tm_hour}:{my_time_1.tm_min - my_time_2.tm_min}"

        fin = pd.concat([fin, ndf])

book = load_workbook(r"C:\Users\Usuario\PycharmProjects\presencia\Presence.xlsx")
with pd.ExcelWriter(r"C:\Users\Usuario\PycharmProjects\presencia\Presence.xlsx", engine='openpyxl') as writer:
    writer.book = book
    writer.sheets = dict((ws.title, ws) for ws in book.worksheets)
    fin.to_excel(writer,date.today().strftime("%d_%m_%Y") )
    writer.save()

See it working!

In this video you can see how it would work, it will only register entries and exits once, and it would keep that data in the presence.txt file, later on, that file is sent to the Excel file for better access.

So, someone (Manuela) would enter the office with her QR code, then Pablo does the same, and someone who has a QR code but who is not authorized to enter the office is not stored in the dataframe. Then when they exit their exit is recorded and finally, an .xlsx file is an output, in which you can check all the details.

Files and resources

Just in case you want to try it out for yourself and you want to see the code once put all together and the complementary files needed for it to work, here you go, a link to my Google Drive where you can download it all in a .zip file. Once again let me ask you not to weird stuff over there (that's my personal Drive :D) https://drive.google.com/drive/folders/12oOjN250Kd7WWWSKmblEKl5xNJ5itbRD?usp=sharing

In the very unlikely case that you want to explore more on the topic I recommend you checking the official documentation to the packages that I've put before, and these videos that I have already mentioned in one of my first entries and which I still find very useful:

OpenCV course by freeCodeCamp.org: https://www.youtube.com/watch?v=oXlwWbU8l2o
Mustaza's Workshop YouTube: channel https://www.youtube.com/user/Mhproductionhouse
Curso completo de Machine Learning: Data Science en Python by Juan Gabriel Gomez Gomila in udemy.com: https://www.udemy.com/course/machinelearningpython/

If you made it here, thanks for reading me, I hope you found my post if not useful at least interesting or the other way around, I don't know what I'll bring up next as apparently I never do what I said I was gonna do I guess it's better to keep as a surprise :D

And btw, the reason I hadn't post anything last week is that I was taking some time to start learning Java, I heard is very useful and who knows, I might bring you something with it too.

Achefe

Search This Blog