| 1. |
|
|
| 2. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Audio–visual person authentication using lip-motion from orientation maps
- 2007
-
Ingår i: Pattern Recognition Letters. - Amsterdam : North-Holland. - 0167-8655 .- 1872-7344. ; 28:11, s. 1368-1382
-
Tidskriftsartikel (refereegranskat)abstract
- This paper describes a new identity authentication technique by a synergetic use of lip-motion and speech. The lip-motion is defined as the distribution of apparent velocities in the movement of brightness patterns in an image and is estimated by computing the velocity components of the structure tensor by 1D processing, in 2D manifolds. Since the velocities are computed without extracting the speaker’s lip-contours, more robust visual features can be obtained in comparison to motion features extracted from lip-contours. The motion estimations are performed in a rectangular lip-region, which affords increased computational efficiency. A person authentication implementation based on lip-movements and speech is presented along with experiments exhibiting a recognition rate of 98%. Besides its value in authentication, the technique can be used naturally to evaluate the “liveness” of someone speaking as it can be used in text-prompted dialogue. The XM2VTS database was used for performance quantification as it is currently the largest publicly available database (≈300 persons) containing both lip-motion and speech. Comparisons with other techniques are presented.
|
|
| 3. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Lip Biometrics for Digit Recognition
- 2007
-
Ingår i: Computer Analysis of Images and Patterns, Proceedings. - Berlin : Springer Berlin/Heidelberg. - 9783540742715 ; , s. 360-365
-
Konferensbidrag (refereegranskat)abstract
- This paper presents a speaker-independent audio-visual digit recognition system that utilizes speech and visual lip signals. The extracted visual features are based on line-motion estimation obtained from video sequences with low resolution (128 ×128 pixels) to increase the robustness of audio recognition. The core experiments investigate lip motion biometrics as stand-alone as well as merged modality in speech recognition system. It uses Support Vector Machines, showing favourable experimental results with digit recognition featuring 83% to 100% on the XM2VTS database depending on the amount of available visual information.
|
|
| 4. |
|
|
| 5. |
- Faraj, Maycel Isaac, 1979-
(författare)
-
Lip-motion and speech biometrics in person recognition
- 2006
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Biometric identification techniques are frequently used to improve security, e.g. in financial transactions, computer networks and secure critical locations. The purpose of biometric authentication systems is to verify an individual by her biological characteristics including those generating characterisitic behaviour. It is not only fingerprints that are used for authentication; our lips, eyes, speech, signatures and even facial temperature are now being used to identify us. This presumably increases security since these traits are harder to copy, steal or lose.This thesis attempts to present an effective scheme to extract descriminative features based on a novel motion estimation algorithm for lip movement. Motion is defined as the distribution of apparent velocities in the changes of brightness patterns in an image. The velocity components of a lip sequence are computed by the well-known 3D structure tensor using 1D processing, in 2D manifolds. Since the velocities are computed without extracting the speaker's lip contours, more robust visual features can be obtained. The velocity estimation is performed in rectangular lip regions, which affords increased computational efficiency.To investigate the usefulness of the proposed motion features we implement a person authentication system based on lip movements information with (and without) speech information. It yields a speaker verification rate of 98% with lip and speech information. Comparisons are made with an alternative motion estimation technique and a description of our proposed feature fusion technique is given. Beside its value in authentication, the technique can be used naturally to evaluate the liveness i.e. to determine if the biometric data is be captured from a legitimate user, live user who is physically present at the point of acquisition, of a speaking person as it can be used in a text-prompted dialog.
|
|
| 6. |
- Faraj, Maycel Isaac, 1979-
(författare)
-
Lip-motion biometrics for audio-visual identity recognition
- 2008
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Biometric recognition systems have been established as powerful security tools to prevent unknown users from entering high risk systems and areas. They are increasingly being utilized in surveillance and access management (city centers, banks, etc.) by using individuals' physical or biological characteristics. The present study reports on the use of lip motion as a standalone biometric modality as well as a modality integrated with audio speech for identity and digit recognition. First, we estimate motion vectors from a sequence of lip-movement images. The motion is modelled as the distribution of apparent line velocities in the movement of brightness patterns in an image. Then, we construct compact lip-motion features from the regional statistics of the local velocities. These can be used alone or merged with audio features to recognize individuals or speech (digits). In this work, we utilized two classifiers for identification and verification of identity as well as with digit recognition. Although the study is focused on processing lip movements in a video sequence, significant speech processing is a prerequisite given that the contribution of video analysis to speech analysis is studied in conjunction with recognition of humans and what they say (digits). Such integration is necessary to understand multimodel biometric systems to the benefit of recognition performance and robustness against noise. Extensive experiments utilizing one of the largest available databases, XM2VTS, are presented.
|
|
| 7. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Lip Motion Features for Biometric Person Recognition
- 2009
-
Ingår i: Visual Speech Recognition. - Hershey, PA : Medical Information Science Reference. - 9781605661865 - 9781605661872 ; , s. 495-532
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- The present chapter reports on the use of lip motion as a stand alone biometric modality as well as a modality integrated with audio speech for identity recognition using digit recognition as a support. First, the auhtors estimate motion vectors from images of lip movements. The motion is modeled as the distribution of apparent line velocities in the movement of brightness patterns in an image. Then, they construct compact lip-motion features from the regional statistics of the local velocities. These can be used as alone or merged with audio features to recognize identity or the uttered digit. The author’s present person recognition results using the XM2VTS database representing the video and audio data of 295 people. Furthermore, we present results on digit recognition when it is used in a text prompted mode to verify the liveness of the user. Such user challenges have the intention to reduce replay attack risks of the audio system. © 2009, IGI Global.
|
|
| 8. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Motion Features from Lip Movement for Person Authentication
- 2006
-
Ingår i: The 18th International Conference on Pattern Recognition. - Washington, D.C. : IEEE Computer Society. - 0769525210 ; 3, s. 1059- 1062
-
Konferensbidrag (refereegranskat)abstract
- This paper describes a new motion based feature extraction technique for speaker identification using orientation estimation in 2D manifolds. The motion is estimated by computing the components of the structure tensor from which normal flows are extracted. By projecting the 3D spatiotemporal data to 2D planes, we obtain projection coefficients which we use to evaluate the 3D orientations of brightness patterns in TV like image sequences. This corresponds to the solutions of simple matrix eigenvalue problems in 2D, affording increased computational efficiency. An implementation based on joint lip movements and speech is presented along with experiments which confirm the theory, exhibiting a recognition rate of 98% on the publicly available XM2VTS database
|
|
| 9. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Person verification by lip-motion
- 2006
-
Ingår i: Conference on Computer Vision and Pattern Recognition Workshop. - Piscataway, N.J. : IEEE Press. - 0769526462 ; , s. 37-37
-
Konferensbidrag (refereegranskat)abstract
- This paper describes a new motion based feature extraction technique for speaker recognition using orientation estimation in 2D manifolds. The motion is estimated by computing the components of the structure tensor from which normal flows are extracted. By projecting the 3D spatiotemporal data to 2-D planes we obtain projection coefficients which we use to evaluate the 3-D orientations of brightness patterns in TV like 2D image sequences. This corresponds to the solutions of simple matrix eigenvalue problems in 2D, affording increased computational efficiency. An implementation based on joint lip movements and speech is presented along with experiments which confirm the theory, exhibiting a recognition rate of 98% on the publicly available XM2VTS database.
|
|
| 10. |
- Faraj, Maycel Isaac, 1979-, et al.
(författare)
-
Speaker and Digit Recognition by Audio-Visual Lip Biometrics
- 2007
-
Ingår i: Advances in biometrics. - Berlin : Springer. - 9783540745488 ; , s. 1016-1024
-
Konferensbidrag (populärvet., debatt m.m.)abstract
- This paper proposes a new robust bi-modal audio visual digit and speaker recognition system by lip-motion and speech biometrics. To increase the robustness of digit and speaker recognition, we have proposed a method using speaker lip motion information extracted from video sequences with low resolution (128 ×128 pixels). In this paper we investigate a biometric system for digit recognition and speaker identification based using line-motion estimation with speech information and Support Vector Machines. The acoustic and visual features are fused at the feature level showing favourable results with digit recognition being 83% to 100% and speaker recognition 100% on the XM2VTS database.
|
|
