Login | Register

Shop Signboards Detection and Classification Framework (SSDCF) based on AI approach and Typeface Analysis


Shop Signboards Detection and Classification Framework (SSDCF) based on AI approach and Typeface Analysis

Almuhajri, Mrouj (2022) Shop Signboards Detection and Classification Framework (SSDCF) based on AI approach and Typeface Analysis. PhD thesis, Concordia University.

[thumbnail of Almuhajri_PhD_F2022.pdf]
Text (application/pdf)
Almuhajri_PhD_F2022.pdf - Accepted Version
Available under License Spectrum Terms of Access.


Rapid advancements in artificial intelligence algorithms have sharpened the focus on street signs due to their prevalence. This research was driven by beneficial applications of end-to-end systems to humans, municipal agencies, and automobiles. However, the variation of materials, shapes, colors, and fonts in some signs, such as shop signboards, have presented complicated challenges to AI-based systems to detect and classify them. Previous studies built classification models by considering the whole storefront. Their classification results were negatively impacted by the inclusion of other components within the storefront. This research focuses on shop signboards as they are much more consistent.

The main objective of this research is to detect and classify shop signboards based on deep learning and machine learning techniques. To achieve that, data acquisition was necessary for models training purposes. Therefore, the Shop Signboard ShoS dataset was collected from Google street images. A total of 10k store signboards were captured within 7500 images. All the collected images were fully annotated and made available for the public for several research purposes.

Then, the Shop Signboard Detection and Classification Framework SSDCF was designed and built to tackle most of the existing challenges. Three main components were fully implemented and evaluated: signboard detector, text extractor, and shop classifier to classify commercial stores based on the textual information. For signboard detector, two models were trained and tested utilizing the ShoS dataset. Findings surpassed the performance of YOLOv3 without any color preparation. For text extractor, the evaluation of Google Vision OCR showed better results even with the existence of influential factors, such as stylized fonts and skewed images. For shop classifier, out of the two trained and tested classifiers, SVM showed great performance even with classes that have some difficulty factors. The performance of the classifier had been enhanced by 4\% approximately after adding the augmented data which was generated by the Random Deletion method and a novel Thesauruses-inspired method named \textit{OCR-Thesauruses}. Each component has been trained and tested individually at first. Then, the full end-to-end framework was implemented and evaluated using the SVT public dataset, and the outcome reached an F1-score=89\%. The classification performance was also compared with human performance based on the texts extracted from the signs. Human subjects were provided with textual information only and were not exposed to shop sing images. The results showed that our classifier exceeded human performance by about 15\% due to the prior knowledge the classifier learned from all text data during training.

Finally, the results of the second component of our framework, the text extractor, were statistically analyzed to check the impact of typeface styles used in shop signboards on the recognition rates. The findings showed a significant association between the typeface style and the recognition rate. So, it is recommended to use ''Serif" and ''Sanserif" styles over ''Script" and ''Decorative" in designing shop signboards. If using stylized fonts is a must, it is advised to add keywords that distinguish a store class from another using a better typeface design, such as ''Serif" or ''Sanserif" styles.

Divisions:Concordia University > Gina Cody School of Engineering and Computer Science > Computer Science and Software Engineering
Item Type:Thesis (PhD)
Authors:Almuhajri, Mrouj
Institution:Concordia University
Degree Name:Ph. D.
Program:Computer Science
Date:14 July 2022
Thesis Supervisor(s):Suen, Ching Y.
ID Code:991147
Deposited On:27 Oct 2022 14:34
Last Modified:27 Oct 2022 14:34
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top