Skip to content

Using FaceMesh's refine_landmarks=True to detect irises #23

Open
@cancan101

Description

At present iris.py first uses FaceMesh to find the eyes and then uses another model to find (refine) the irises:

def detect_iris(eye_frame, is_right_eye=False):
side_low = 64
eye_frame_low = cv2.resize(
eye_frame, (side_low, side_low), interpolation=cv2.INTER_AREA
)
model_path = "models/iris_landmark.tflite"
if is_right_eye:
eye_frame_low = np.fliplr(eye_frame_low)
outputs = tflite_inference(eye_frame_low / 127.5 - 1.0, model_path)
eye_contours_low = np.reshape(outputs[0], (71, 3))
iris_landmarks_low = np.reshape(outputs[1], (5, 3))
eye_contours = eye_contours_low / side_low
iris_landmarks = iris_landmarks_low / side_low
if is_right_eye:
eye_contours[:, 0] = 1 - eye_contours[:, 0]
iris_landmarks[:, 0] = 1 - iris_landmarks[:, 0]
return eye_contours, iris_landmarks

However, per this comment, google-ai-edge/mediapipe#2605 (comment) when FaceMesh is run with refine_landmarks=True, it directly returns the irises. Is there a reason to not just use that directly?

The indices for the iris landmarks can then be found using these constants:

mp_face_mesh = mp.solutions.face_mesh
mp_face_mesh.FACEMESH_IRISES

For reference, this is how the mediapipe example code then plots these:

      mp_drawing.draw_landmarks(
          image=annotated_image,
          landmark_list=face_landmarks,
          connections=mp_face_mesh.FACEMESH_IRISES,
          landmark_drawing_spec=None,
          connection_drawing_spec=mp_drawing_styles
          .get_default_face_mesh_iris_connections_style())

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions