Optical Character Recognition (OCR) – How it works

• Loading image as bitmap from given source. Source can be a file or a pointer to a memory block, also good OCR system must understand a lot of image formats: BMP, TIFF (both one-page and multi-pages images), JPEG, PNG and so on. PDF files must be supported as well, many documents are stored as images in PDF format and the only way to extract text from such files is to perform OCR.
• Detecting the most important image features like resolution and inversion. Many OCR algorithms expect some predefined range of font sizes and foreground/background colors so the image must be rescaled and inverted before processing when necessary.
• Image can be skewed or it can have a lot of noise, so deskew and denoising algorithms are applied to improve the image quality.

• Many OCR algorithms require bi-tonal image, therefore color or gray image must be converted to black-white image. This process is called "binarization" and it is very important step because incorrect binarization will cause a lot of problems.

• Lines detection and removing. This step is required to improve page layout analysis, to achieve better recognition quality for underlined text, to detect tables, etc.

• Page layout analysis; this steps is also called "zoning". At this stage OCR system must detect positions and types of all important areas on the image.
• Detection of text lines and words. Sometimes it is not an easy task because of different font sizes and small spaces between words.
• Combined-broken characters analysis. It is very common situation when some characters are broken to several parts, or when a few characters touch each one; it is necessary to detect such cases and find correct position of every character.

• Recognition of characters. This is the main algorithm of OCR; an image of every character must be converted to appropriate character code. Sometimes this algorithm produces several character codes for uncertain images. For instance, recognition of the image of "I" character can produce "I", "|" "1", "l" codes and the final character code will be selected later.
• Dictionary support. This step can improve recognition quality, some characters like "1" and "I", "C" and "G" can look very similar and the dictionary can help to make the decision.
• Saving results to selected output format, for instance, searchable PDF, DOC, RTF, TXT. It is important to save original page layout: columns, fonts, colors, pictures, background and so on.