Unlike the frontal face detection, multi-pose face detection and recognition techniques, still face the following challenges: large variability of environments such as pose, illumination and backgrounds, and unconstrained capturing of facial images. We introduced a new system to deal with this problem. First, a two-step color-based approach is used to find a candidate area of face from original picture. Then a rough estimator of five poses is created using AdaBoost technique. In order to accurately locate the candidate face, multiple statistical shape models-ASM (Active Shape Models) are proposed to estimate an accurate pose of model of the input image and to extract facial features as well. In the recognition step, we use a geometrical mapping technique to deal with the pose variation and face identification.