Abstract

Handwriting analysis, or graphology, studies an individual’s psychological traits through handwriting patterns and features. It is used in forensic science, criminology, and disease diagnosis.

Previous studies have evaluated the correlation between psychological questionnaires and manual handwriting analysis, but results were inconsistent due to its limitations and human error. This research addresses these challenges by developing an automated handwriting analysis system using deep learning to predict multi-label personality traits based on the Big Five Factor Model (BFFM).

The proposed model is built on the Transformer OCR (TrOCR) architecture, pre-trained on diverse datasets, including handwritten texts like IAM. In this study, the text generation function is replaced with a classification approach to predict levels (Low, Average, High) of BFFM traits from handwriting samples. The model uses Focal Loss to handle class imbalance and Binary Cross-Entropy with Logits for accurate classification.

The dataset includes 873 French and 181 English handwriting samples from CENPARMI, originally labeled for Extraversion and Conscientiousness. It has been expanded to cover all five BFFM traits: Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness to Experience, totaling 1,054 samples. Each sample is segmented into individual lines to improve generalization.

The model's performance is compared with ResNet50 and Vision Transformers (ViT Base 16 - 224 and 384). Results show that TrOCR outperforms them in accuracy and overall performance. For two personality traits, it achieves 90.05% accuracy, AUROC of 0.97, and F-Score of 89%. For all five traits, it reaches 89.01% accuracy, AUROC of 0.95, and F-Score of 87%. Extraversion shows the weakest performance (AUROC of 91), while Agreeableness performs best (AUROC of 97). These results highlight the model's effectiveness in classifying BFFM traits despite class imbalance.