Week 4: February 22nd, 2023

Live Code Session :

Face Api, FaceMesh, HandPose

Working with: Face API, Face Mesh, HandPose

Idea: Colors based on height of the corners of mouth

• lerp?
• higher corners/smile, more red. lower corners/frown, more blue

p5 Sketch: Color Edit

I started by plotting circles on each face key point. Then I created a gradient across the face points.

//face mesh - ellipse at each keypoint 
  for (let i = 0; i < myResults.length; i += 1) {
    const prediction = myResults[i];
    for (let j = 0; j < prediction.scaledMesh.length; j += 1) {
      const keypoint = prediction.scaledMesh[j];
      noStroke();
      
      let startColor = color(startR, startG, startB)
      let endColor = color(endR, endG, endB)
      let grad = lerpColor(startColor, endColor, j/467)
      fill(grad)
      ellipse(keypoint[0], keypoint[1], 5);   
    }
  }

Using the face-landmarks-detection mesh_map, I found the exact keypoints for the corners of the mouth.

const leftMouth = myResults[0].scaledMesh[61];
    const leftMouthX = leftMouth[0];
    const leftMouthY = leftMouth[1];
    
    const rightMouth = myResults[0].scaledMesh[291];
    const rightMouthX = rightMouth[0];
    const rightMouthY = rightMouth[1];
    
    const mouth = myResults[0].scaledMesh[14];
    const mouthX = mouth[0];
    const mouthY = mouth[1];

I created circles on the left corner, upper center, and right corner of the mouth to check the placement of the key point.

Then, I drafted the color change using conditional statements and simple red and green.