Can you provide the processed data or the related processing code?

[Haoqing-Wang]

Nice job! Can you released the used dataset?

[HUAFOR]

+1

[DavidKong96]

hope to know which face det model used..

[lmpeng12]

+1

[MengShen0709]

+1

[MengShen0709]

@DavidKong96
I guess authors might use dlib to extract the landmarks

The partial landmarks are defined in their dataloader:

        landmarks_img = landmarks[13:48]
        landmarks_img2 = landmarks[0:4]
        landmarks_img = np.concatenate((landmarks_img2, landmarks_img))

I use dlib to extracts 68 points, and follow the authors to only keep some of them. It turns out, by doing so, we could remove bottom half of landmarks. This landmark masking is in accordance with the image masking for the model to learn how to generate lip movement.

Here is an illustration:

But I haven't reproduced the results, so it is just my guess.

Feel free to discuss.

[Haoqing-Wang]

before obtain landmark, we need to detect the facial RoI advance. But when the model can not detete the face, how to obtain the landmark? we use dlib to obtain landmark.

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
rect = detector(gray)[0]
shape = predictor(gray, rect)

rect could be None.

[jianmanLin]

@DavidKong96 I guess authors might use dlib to extract the landmarks

The partial landmarks are defined in their dataloader:

        landmarks_img = landmarks[13:48]
        landmarks_img2 = landmarks[0:4]
        landmarks_img = np.concatenate((landmarks_img2, landmarks_img))

I use dlib to extracts 68 points, and follow the authors to only keep some of them. It turns out, by doing so, we could remove bottom half of landmarks. This landmark masking is in accordance with the image masking for the model to learn how to generate lip movement.

Here is an illustration:

But I haven't reproduced the results, so it is just my guess.

Feel free to discuss.

Hello, did you successfully reproduce this paper? As a result of my training, the inpainting area will keep shaking, and then the training loss will drop rapidly at the beginning, and then it will shake within a small area. I feel very happy troubled

[jianmanLin]

before obtain landmark, we need to detect the facial RoI advance. But when the model can not detete the face, how to obtain the landmark? we use dlib to obtain landmark.

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
rect = detector(gray)[0]
shape = predictor(gray, rect)

rect could be None.

Hello, did you successfully reproduce this paper? As a result of my training, the inpainting area will keep shaking, and then the training loss will drop rapidly at the beginning, and then it will shake within a small area. I feel very happy troubled

[Utkarsh-shift]

same request