Face rotation/angle detection #95

YanDevDe · 2018-06-04T22:14:44Z

It would be nice if there would be a way to detect the face's direction/rotation. You can calculate direction from both eyes.

YanDevDe · 2018-06-06T13:04:43Z

You can get eye position from Face Landmarks which allows you to calculate to get angle. So solved!

gauravdhra · 2020-09-18T11:14:46Z

Hi @YanDevDe .
I need the same solution. I have been trying to find the solution for 2 weeks.
Could you please tell me what is that calculation to find if face is looking forward / little up or not ?

I need that solution to find out multiple faces out from a video of student class room.

YanDevDe · 2020-09-18T11:31:59Z

Hello @gauravdhra
It was 2 years ago, so I might have forgotten how to solve it. Sorry!
But I'll try my best here:

I assume you can get position by using face landmark.

With those position you can either calculate...

the face rotations (by using both eye positions, this trick should works)
tilting face (up/down) by calculating the distance between both eyes and mouth/nose position. The shorter distance it is, the more you're looking down/up. To calculate distance between two points, you can check it out here

I'm sorry if I was not a help!

Regards,

Yan

YanDevDe · 2020-09-18T11:33:01Z

Also you may want to checkout at face-api.js since this face-recognition.js is obsolete.

gauravdhra · 2020-09-18T11:51:24Z

Hey @YanDevDe ,after two years you still have solution wow !!!

I have all the position let me find the value of c :)

eye_right [466.1014111340046, 199.98082201182842]
eye_left [446.8404211352269, 199.97677264114222]
nose [455.0013075934516, 208.0722832745976]
mouth [456.26030165702105, 215.79589595198632]

YanDevDe · 2020-09-18T11:55:32Z

@gauravdhra to be fair, I forgot that I was using this library lol. But this was a logical approach to detect the rotation/tilt of the faces.

Yeah those values are more than enough! Now calculating is the only part left! =)

Glad that I could help you!

Regards, Yan

gauravdhra · 2020-09-18T12:04:57Z

@YanDevDe

Could you clarify this The shorter distance it is, the more you're looking down/up .
how to know if distance is shorter or not ? I am confused :p
shorter distance > looking down

gauravdhra · 2020-09-18T12:07:49Z

the distance between both eyes should be equal to the distance between any eye and mouth .
it they are not equal or the smaller one then its looking down ..
am I right

YanDevDe · 2020-09-18T12:17:14Z

You can either calculate the center point (lets call cp) between both eyes. (Or if you want to be lazy, just use left or right eye position, but it would be better to get center point between both of the eyes for more accurate result)

For mouth position, let's call mp.

So that means in order to calculate the distance between eyes and mouth position, you need to do that:

const distance_x = mp.x - cp.x;
const distance_y = mp.y - cp.y;
const distance = Math.sqrt(distance_x * distance_x + distance_y + distance_y);

The distance is now the value which you can guess if the face is tilting or not. The smaller distance it is, the more you're looking down or looking up (by up, I mean like up and not front)

The distance value may be bit different depending on face's different position / distances. So using simple equal operator may be not enough. But that's up to you to solve it. 😅

gauravdhra · 2020-09-18T12:45:38Z

@YanDevDe I understand the distance will be calculated according to the close faces and the faces sitting behind .
that is now a difficult part ...oops

I have calculated the distance and I am stuck here how to decide the face rotation

YanDevDe · 2020-09-18T13:14:26Z

As far I've read from a while ago, almost every face has same eyes distance (left and right eye). So you can use eyes distance to tell on which distance your face is from camera. 🤷 But don't see this as confirmation, maybe I'm wrong as well.

You can also use the rectangle detection sizes as distance between face and camera, if that works for you.

gauravdhra · 2020-09-28T12:28:04Z

Hi @YanDevDe that was helpful for me .
Now I need to find if face is not turned to left or right .I need straight looking faces .

gauravdhra · 2020-09-30T13:11:40Z

@YanDevDe Please help me out
See some face are looking down but it still detecting it .So for the improvement little bit I want to find bottom looking side faces/ or just side faces that will be enough .

trongmanh19 · 2021-01-18T09:48:47Z

Hi @YanDevDe @gauravdhra, Please help me
I get position of mouth, left eye and right eye by blow code
`
const landmarks = await faceapi.detectFaceLandmarks(videoEl);
const mouth = landmarks.getMouth();

  const leftEye = landmarks.getLeftEye();

  const rightEye = landmarks.getRightEye();

 console.log("Mouth position ===>" + JSON.stringify(mouth));

  console.log("Left eye position ===>" + JSON.stringify(leftEye));

  console.log("Right eye position ===>" + JSON.stringify(rightEye));

Result:

// mouth position
[{"_x":203.203125,"_y":399.6875},{"_x":217.1484375,"_y":382.5},{"_x":230.7421875,"_y":383.4375},{"_x":245.625,"_y":376.5625},{"_x":254.296875,"_y":370.3125},{"_x":253.828125,"_y":365},{"_x":275.859375,"_y":371.875},{"_x":275.625,"_y":360.625},{"_x":283.125,"_y":396.875},{"_x":267.1874713897705,"_y":420.00003814697266},{"_x":258.28125,"_y":424.6875},{"_x":238.2421875,"_y":415.3125},{"_x":202.14845180511475,"_y":399.6875},{"_x":236.015625,"_y":381.5625},{"_x":246.5625,"_y":383.4375},{"_x":258.75,"_y":371.5625},{"_x":264.140625,"_y":366.5625},{"_x":269.296875,"_y":390.3125},{"_x":259.21875,"_y":405.9375},{"_x":252.1875,"_y":406.875}]

// leftEye position
[{"_x":98.14453125,"_y":344.0625},{"_x":106.99220180511475,"_y":331.875},{"_x":126.4453125,"_y":318.28125},{"_x":156.09375,"_y":314.0625},{"_x":140.390625,"_y":329.375},{"_x":112.32422590255737,"_y":333.75}]

// rightEye position
[{"_x":220.3125,"_y":286.7187690734863},{"_x":243.2812786102295,"_y":277.03125},{"_x":247.9687786102295,"_y":283.59375},{"_x":246.796875,"_y":269.0625},{"_x":243.2812786102295,"_y":292.1875},{"_x":243.2812786102295,"_y":286.25}]

`
How to calculate distance from left/right eye to mouth?
Thanks!

fredzhu008 · 2021-01-20T03:46:18Z

Hi,    Did you have the detailed requirements ?such as pdf. Could you send another email.

…

------------------ 原始邮件 ------------------ 发件人: "justadudewhohacks/face-recognition.js" <[email protected]>; 发送时间: 2021年1月18日(星期一) 下午5:49 收件人: "justadudewhohacks/face-recognition.js"<[email protected]>; 抄送: "Subscribed"<[email protected]>; 主题: Re: [justadudewhohacks/face-recognition.js] Face rotation/angle detection (#95) Hi @YanDevDe @gauravdhra, Please help me I get position of mouth, left eye and right eye by blow code const mouth = landmarks.getMouth(); const leftEye = landmarks.getLeftEye(); const rightEye = landmarks.getRightEye(); console.log("Left eye position ===>" + JSON.stringify(leftEye)); console.log("Right eye position ===>" + JSON.stringify(mouth)); console.log("Mouth position ===>" + JSON.stringify(mouth)); Result: // mouth position [{"_x":203.203125,"_y":399.6875},{"_x":217.1484375,"_y":382.5},{"_x":230.7421875,"_y":383.4375},{"_x":245.625,"_y":376.5625},{"_x":254.296875,"_y":370.3125},{"_x":253.828125,"_y":365},{"_x":275.859375,"_y":371.875},{"_x":275.625,"_y":360.625},{"_x":283.125,"_y":396.875},{"_x":267.1874713897705,"_y":420.00003814697266},{"_x":258.28125,"_y":424.6875},{"_x":238.2421875,"_y":415.3125},{"_x":202.14845180511475,"_y":399.6875},{"_x":236.015625,"_y":381.5625},{"_x":246.5625,"_y":383.4375},{"_x":258.75,"_y":371.5625},{"_x":264.140625,"_y":366.5625},{"_x":269.296875,"_y":390.3125},{"_x":259.21875,"_y":405.9375},{"_x":252.1875,"_y":406.875}] // leftEye position [{"_x":98.14453125,"_y":344.0625},{"_x":106.99220180511475,"_y":331.875},{"_x":126.4453125,"_y":318.28125},{"_x":156.09375,"_y":314.0625},{"_x":140.390625,"_y":329.375},{"_x":112.32422590255737,"_y":333.75}] // rightEye position [{"_x":220.3125,"_y":286.7187690734863},{"_x":243.2812786102295,"_y":277.03125},{"_x":247.9687786102295,"_y":283.59375},{"_x":246.796875,"_y":269.0625},{"_x":243.2812786102295,"_y":292.1875},{"_x":243.2812786102295,"_y":286.25}] How to calculate distance from left/right eye to mouth? Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

fredzhu008 · 2021-01-20T03:53:00Z

add my weichat 13501901060

…

------------------ 原始邮件 ------------------ 发件人: "justadudewhohacks/face-recognition.js" <[email protected]>; 发送时间: 2021年1月18日(星期一) 下午5:49 收件人: "justadudewhohacks/face-recognition.js"<[email protected]>; 抄送: "Subscribed"<[email protected]>; 主题: Re: [justadudewhohacks/face-recognition.js] Face rotation/angle detection (#95) Hi @YanDevDe @gauravdhra, Please help me I get position of mouth, left eye and right eye by blow code const mouth = landmarks.getMouth(); const leftEye = landmarks.getLeftEye(); const rightEye = landmarks.getRightEye(); console.log("Left eye position ===>" + JSON.stringify(leftEye)); console.log("Right eye position ===>" + JSON.stringify(mouth)); console.log("Mouth position ===>" + JSON.stringify(mouth)); Result: // mouth position [{"_x":203.203125,"_y":399.6875},{"_x":217.1484375,"_y":382.5},{"_x":230.7421875,"_y":383.4375},{"_x":245.625,"_y":376.5625},{"_x":254.296875,"_y":370.3125},{"_x":253.828125,"_y":365},{"_x":275.859375,"_y":371.875},{"_x":275.625,"_y":360.625},{"_x":283.125,"_y":396.875},{"_x":267.1874713897705,"_y":420.00003814697266},{"_x":258.28125,"_y":424.6875},{"_x":238.2421875,"_y":415.3125},{"_x":202.14845180511475,"_y":399.6875},{"_x":236.015625,"_y":381.5625},{"_x":246.5625,"_y":383.4375},{"_x":258.75,"_y":371.5625},{"_x":264.140625,"_y":366.5625},{"_x":269.296875,"_y":390.3125},{"_x":259.21875,"_y":405.9375},{"_x":252.1875,"_y":406.875}] // leftEye position [{"_x":98.14453125,"_y":344.0625},{"_x":106.99220180511475,"_y":331.875},{"_x":126.4453125,"_y":318.28125},{"_x":156.09375,"_y":314.0625},{"_x":140.390625,"_y":329.375},{"_x":112.32422590255737,"_y":333.75}] // rightEye position [{"_x":220.3125,"_y":286.7187690734863},{"_x":243.2812786102295,"_y":277.03125},{"_x":247.9687786102295,"_y":283.59375},{"_x":246.796875,"_y":269.0625},{"_x":243.2812786102295,"_y":292.1875},{"_x":243.2812786102295,"_y":286.25}] How to calculate distance from left/right eye to mouth? Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

trongmanh19 · 2021-01-21T02:17:19Z

@fredzhu008 I need to find if face is turned to left or right or straight looking faces.
Can you help me?

fredzhu008 · 2021-01-21T04:49:52Z

Could you send the original project description or other indormation,or example for that draw by handwriting? I cannot get the goal for this project. It's you lab practice or commercial project?

…

------------------ 原始邮件 ------------------ 发件人: "justadudewhohacks/face-recognition.js" <[email protected]>; 发送时间: 2021年1月21日(星期四) 上午10:17 收件人: "justadudewhohacks/face-recognition.js"<[email protected]>; 抄送: "Fred"<[email protected]>;"Mention"<[email protected]>; 主题: Re: [justadudewhohacks/face-recognition.js] Face rotation/angle detection (#95) @fredzhu008 I need to find if face is turned to left or right or straight looking faces. Can you help me? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

YanDevDe closed this as completed Jun 6, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Face rotation/angle detection #95

Face rotation/angle detection #95

YanDevDe commented Jun 4, 2018

YanDevDe commented Jun 6, 2018 •

edited

Loading

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020

gauravdhra commented Sep 18, 2020

YanDevDe commented Sep 18, 2020

gauravdhra commented Sep 18, 2020

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

gauravdhra commented Sep 28, 2020

gauravdhra commented Sep 30, 2020 •

edited

Loading

trongmanh19 commented Jan 18, 2021 •

edited

Loading

fredzhu008 commented Jan 20, 2021 via email

fredzhu008 commented Jan 20, 2021 via email

trongmanh19 commented Jan 21, 2021

fredzhu008 commented Jan 21, 2021 via email

Face rotation/angle detection #95

Face rotation/angle detection #95

Comments

YanDevDe commented Jun 4, 2018

YanDevDe commented Jun 6, 2018 • edited Loading

gauravdhra commented Sep 18, 2020 • edited Loading

YanDevDe commented Sep 18, 2020 • edited Loading

YanDevDe commented Sep 18, 2020

gauravdhra commented Sep 18, 2020

YanDevDe commented Sep 18, 2020

gauravdhra commented Sep 18, 2020

gauravdhra commented Sep 18, 2020 • edited Loading

YanDevDe commented Sep 18, 2020 • edited Loading

gauravdhra commented Sep 18, 2020 • edited Loading

YanDevDe commented Sep 18, 2020 • edited Loading

gauravdhra commented Sep 28, 2020

gauravdhra commented Sep 30, 2020 • edited Loading

trongmanh19 commented Jan 18, 2021 • edited Loading

fredzhu008 commented Jan 20, 2021 via email

fredzhu008 commented Jan 20, 2021 via email

trongmanh19 commented Jan 21, 2021

fredzhu008 commented Jan 21, 2021 via email

YanDevDe commented Jun 6, 2018 •

edited

Loading

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

gauravdhra commented Sep 18, 2020 •

edited

Loading

YanDevDe commented Sep 18, 2020 •

edited

Loading

gauravdhra commented Sep 30, 2020 •

edited

Loading

trongmanh19 commented Jan 18, 2021 •

edited

Loading