Face_recognition and Multiprocessing troubles

[ghost]

• face_recognition version: 1.0
• Python version: 3.6
• Operating System: Arch Linux

Description

I'm trying to parallel process the comparison of facial encodings (in anticipation for a lot of faces), however when I try to use any sort of parallel processing library (i.e. multiprocessing, concurrent.futures, etc.) I get a TypeError: can't pickle cv2.VideoCapture objects. I'm incredibly confused, as I don't think I'm using any cv2 objects when I compare?

Here's the code snippets that generate the error:
(Sorry for the crazy code, I've been pulling my hair out over this issue)

import face_recognition as fr
import concurrent.futures
.......
 self.pool = concurrent.futures.ProcessPoolExecutors()
.......
#The function being called as a different proces:
def threadComparison(self, vals):
        boole = fr.compare_faces(vals[0], vals[2][0])
        val = self.RecFaces(boole)
        if val is not None:
            self.found.append(vals[1][val])

#The function handling data sorting and main processing:
def searchFace(self, encoding):
        #Pull faces from database
        values = self.fdr.select('faces', '*')
        counter = 0
        buff = []
        names = []
        bit = False
        for face in values:
            if len(self.found) > 0:
                bit = True
                break
            #Sorts facial encodings with names
            buff.append(np.array(face[1].split(' ')))
            names.append(face[0])
            #Tried debugging here, all are list, strings, etc. No cv2.VideoCapture types.
            #print(type(enc[0]))
            #print(type(buff[0]))
            #print(type(names[0]))
            if counter % 25 == 0:
                #Error generated here-------------------------
                self.pool.map(self.threadComparison, [buff,names,enc])
                #Error generated here^^^^^^^^^^^^^^^^^^^^^^^^^
                buff, names = []
            counter += 1
        self.pool.map(self.threadComparison, [buff, names, enc])
        if bit or len(found) > 0:
            print('Name: '+self.found[0])`

        else:
            print('No entry for face...')

Any help is greatly appreciated!

[ageitgey]

There's a couple of problems at least with the code that have to be worked out before I could try to debug that specific issue.

First, this line:

                self.pool.map(self.threadComparison, [buff,names,enc])

doesn't really make any sense. You are sending buff to one thread, names to the next thread, and enc (which is never defined any may be picking up something defined in global scope) to the next thread.

I'm guessing you might be trying to do something like where you send one element of each array together as three parameters to self.threadComparison?

    function_parameters = zip(
        buff,
        names,
        enc
    )

    pool.starmap(self.threadComparison, function_parameters)

... but I could be wrong. But what you are currently doing doesn't make sense to me.

Next, I'm not really sure what the self.threadComparison function is exactly doing by checking hard-coded fixed indexes, but this line in particular is a bad idea:

            self.found.append(vals[1][val])

When you call pool.map, Python will spin up multiple copies of the running code in different python instances. So each copy of the function will be running in a separate thread at that point. You won't be able to just write back to self there since there will be a different copy of self in each thread. You'll be saving the results inside just the copy of the object that exists in a temporary thread.

Instead, you need to return the result for one single lookup from each call to self.threadComparison (right now, nothing is being returned). As long as you return a result, the pool.map() will take care of combining all the results from all the different threads and giving you back one single combined list.

Next, you'll want to assign the result of the pool.map() function to a variable so that you can check the results. Right now, they are all just being thrown away.

So that doesn't exactly answer your question, but your code right now doesn't "work" in a conceptual sense, so I'm guessing the error is the side effect of some bad global data getting passed in the enc variable (or something). But first try fixing the code's basic logic and see if that doesn't take care of the error.

If that still doesn't fix it, you need to figure out what bad value is getting passed in as part of the second parameter to the pool.map() function.

Hope that helps! :)

[ghost]

Thank you! I'll try re-writing my code and get back to you soon.

[ghost]

Okay, I've refactored my code based on your recommendations. However, its still telling me that it cannot pickle cv2 objects for some reason?

I rewrote my code to the following:

def threadComparison(self, vals):
        retval = None
        boole = fr.compare_faces(vals[0], vals[2][0])
        val = self.RecFaces(boole)
        if val is not None:
            retval = vals[1][val]
        return retval

def searchFace(self, enc):
        #Pull faces from database
        values = self.fdr.select('faces', '*')
        counter = 0
        buff = []
        names = []
        findperson = None
        for face in values:
            buff.append(np.asarray(face[1].split(' '), dtype=float))
            names.append(face[0])
            #Tried debugging here, all are list, strings, etc. No cv2.VideoCapture types.
            print(type(enc))
            print(type(buff))
            print(type(names))
            if counter % 25 == 0:
                searched = self.pool.map(self.threadComparison, zip(buff, names, enc))
                if searched is not None:
                    findperson = searched
                    break
                buff, names = []
            counter += 1
        findperson = self.pool.map(self.threadComparison, zip(buff, names, enc))
        print(list(findperson))
        if findperson is not None:
            print('Name: '+ findperson)
        else:
            print('No entry for face...')

However, I think the problem is the way I'm passing the facial encodings somehow, because when it fails to recognize my face when a picture is taken the pickling error doesn't occur. I think that the encodings from my database are fine, along with the names. Is there some other way I could pass the encodings, or do you think there could still be something wrong with the way I'm handling it?

Here's the code for taking my picture and passing it to the main search function:

h = HelloWorld()
#Take picture, and pass face_recognition readable frame and original frame
pframe, pics = h.Blink()
#Grab face locations, if any.
loc = fr.face_locations(pics)
#Grab face encodings, if any.
enc = fr.face_encodings(pics, loc)
#Pass facial encodings to be searched
h.searchFace(enc)

[ghost]

Hello, I wanted to check back to see if you had a chance to look at this? I've been scratching my head over the problem for a while.

[kokilarathan-siva01]

`
def searchFace(self, enc):
# Pull faces from database
values = self.fdr.select('faces', '*')

# Convert encoded faces and names to numpy arrays
db_encs = np.asarray([np.asarray(face[1].split(' '), dtype=float) for face in values])
db_names = [face[0] for face in values]

# Compare input face with all faces in the database
distances = face_recognition.face_distance(db_encs, enc)
min_distance_index = np.argmin(distances)

# If a match is found, return the corresponding name
if distances[min_distance_index] < 0.6:
    return db_names[min_distance_index]
else:
    return "No entry for face..."

`

This code eliminates the need for a thread pool by using the face_distance function from the face_recognition library to compute the distance between the input face and all faces in the database at once. It then finds the index of the face with the smallest distance (i.e., the closest match), and returns the corresponding name if the distance is below a threshold of 0.6 (which can be adjusted based on your needs). If no match is found, it returns the string "No entry for face...".