Real Time Road Sign Recognition System

Real Time Road Sign Recognition System Using Artificial Neural Networks For Bengali Textual Information Box An Automated Road Sign Recognition system using Artificial Neural Network for the Textual Information box inscribing in Bengali is presented on the paper. Signs are visual languages that represent some special circumstantial information of environment. Road signs, being among the most important around us primarily for safety reasons, are designed, and manufactured and installed according to tight regulations. The system captures real time images every two seconds and saves them as JPG format files. Firstly some road sign are already stored in the memory. Like: Warning Sign, Prohibition Sign, Obligation Sign and Informative Sign. Car Driver concentration and illiterateness isn’t always focused on what it should be and not always notice the road signs. For these reasons, automation of Bangla Road Sign Recognition system is highly essential. Previously several works are done by Mueller, Piccioli, Novovicova, Yuille, Escalera and others. But those are not in Bengali. Real Time Road Sign Recognition System Using Artificial Neural Networks for Bengali Textual Information Box which is done by Mohammad Osiur Rahman, Fouzia Asharf Mousumi, Edgar Scavino, Aini Hussain, Hassan Basri whose are from the Department of Computer Science and Engineering, University of Chittagong, Chittagong-4331, Bangladesh, Faculty of Engineering, University Kebangsaan Malaysia. For doing this they divide the total Concept in Steps: 1. Image Acquisition: From several video sequences from a moving vehicle for a certain period are consecutive frames recorded within 2 seconds are similar. For this they have used Application Programming Interface functions of VB 6. 0. Every 2-second a frame is collected and stored in JPG format. 2. Preprocessing: Median filter is used to reduce impulsive or salt-and-pepper type noise from captured images and then normalized into 320 X 240 pixels. 3. Text Detection and Extraction: An algorithm was developed for textual information detection and extraction from Bangla Road Signs on the basis of the Sobel Edge Detection technique. Like the following: I. Read input image in . jpg format II. Convert colored image into gray scale image III. Apply 3Ãâ€"3 median filter convolution masks on gray scale image IV. Calculated edges by applying Sobel convolutions mask V. Thicken the calculated edges by dilation VI. Apply vertical Sobel projection filter on dimmed image VII. Create a histogram by computing projection values VIII. Find the threshold value of the image IX. Loop on the possible positive identifications based on the histogram values X. Extract the possible positive identifications based on the histogram values XI. Apply Sobel horizontal edge-emphasis for other possible text area searches XII. Convert detected text region into binary image XIII. Calculate height and width of detected region of text XIV. Crop the image 4. Bangla OCR using MLP: An ANN based approach is used for Bangla OCR of road signs’ text. It has 3 sub modules – Character segmentation, Feature Extraction and Character Recognition by MLP NN. 5. Confirmation of Textual Road Signs and Conversion 6. Speech synthesis The Proposed system works like the following: 1. From video sequences capture a single frame in JPG format in each 2 seconds. 2. Preprocess the captured image each time . Detect the Text and Extract that and then Extracted Text will recognize by Bengali Optical Character Recognition System. 4. Recognized characters of textual information compared with the stored knowledge and then give decision valid or invalid. 5. If Valid then recognize and according to users choice it provide Bengali or it convert to English and provide audio stream. The system processes the images to find out whe ther they contain images of road signs or not. The textual information of the road signs is detected and extracted from the images. The Bengali OCR system takes the textual information as an input to recognize individual Bengali characters. The Bengali OCR is implemented using Multi-layer Perceptron. The output of the Bengali OCR system is compared with the previously enrolled standard Bengali textual road signs. The throughput which comes from the matching process is used as input for the speech synthesizer and finally the system delivers the audio stream to the driver, either in Bengali or in English based on the user settings. After testing this system, the obtained accuracy rate was evaluated at 91. 48%. Our Idea by using Hopfield Associative Memory Our work to done this thesis by using Associative Memory. Which are two types – Hetero Associative Memory & Auto Associative Memory. We will use the Auto-associative / Autocorrelators Memory for our purposes. It’s now most easily recognized by the title of HAM(Hopfield Associative Memory), were introduced as a theoretical notation by Donald Hebb. To do this we need to first generate Matrices (Row or Column Matrix) in the Bipolar Boolean format (-1 and +1) from the Image. Then the matrices need to transpose of each of the matrices and then create the encoding process (The Connection Matrix) by [pic] And then need to Recognized of the stored patterns or feed each of the matrix by [pic] Introducing the Bipolar Function to [pic]. If [pic] >= 0 set the value +1 otherwise set the value -1 for each of the Element of the Matrix of [pic]. Now Recognition of Noisy Patterns by finding the Hamming Distance (HD) with the Given Noisy Pattern N by [pic] Which Hamming Distance of noisy and stored pattern are less the probability of matching to noisy pattern with the stored pattern are most. And then need to Recognized of the Noisy patterns or feed each of the matrix with Encoding Process by [pic] By using Bipolar Function to [pic]. If [pic] > 0 set the value +1 otherwise set the value -1 for each of the Element of the Matrix of [pic]. In this method we need to store all road sign text segmented by each blank will generate Matrices. And by the above method generate correlation matrix. If the Bipolar Noisy Matrix matched with the Transposed Matrix of the stored Image Transpose Matrix, in the case of partial vectors, an Auto-Correlator results in the refinement of the pattern or removal of noise to retrieve the closest matching stored pattern. Our Idea by using WANG et al. ’s Multiple training encoding strategy (WANG MTES): The algorithm of the WANG MTES is like the following: Step-1:Initialize the correlation matrix M to null matrix M ( [0]. Step-2:Compute the M as, For I ( 1 to N M ( M ( [qi * (Transpose Xi’) ( Yi’] [where Xi’ and Yi bipolar patterns] End Step-3:Read input bipolar Pattern A’ Step-4:Compute A_M where A_M ( A’ ( M Step-5:Apply threshold function ( to A_M to get B’ [(=bipolar of Matrices] Step-6:Output B’ which is the associated Pattern Pair. In this method, as like the HOPFIELD ASSOCIATIVE MEMORY we need to store all road sign text segmented by each character will generate Matrices Associated with the equivalent ASCII of Bengali Character Matrix. And by the above method generate correlation matrix of the stored Pattern. Now from the input image text need to generate matrix of called noisy pattern will must in bipolar form. And Feed with the Correlation Matrix. Equation like the following: [pic] qi’s are positive real number called generalized correlation matrix, will be change according to the improving feeding necessity. Figure: Schematic view of Bangla Road Sign Recognition System ———————– Speech Language Choose? Speech synthesis Convert into equivalent English text English Bengali Audio stream Valid Bangla road Sign Recognized Unrecognized Yes Prememorized Knowledge (Bangla Sign Textual info Database) Image (JPG format) Processing Text detection& extraction Matching Bangla OCR using WANG MTES Extracted Text Recognized Characters of Texture Information Single Frame Video Sequences No