Capturing and Evaluating Surveys and Questionnaires with FormPro

In this article we show you how filled-out questionnaires, surveys and forms can be captured fully automatically with our software FormPro. We start with designing the optimal form layout and guide you from creating a reading template to scanning filled-out documents and finish off with data extraction.

Download Demo

Purchase FormPro

Optimal Layout Design

As one of its many strengths FormPro does not impose any restrictions on the design of your questionnaires or surveys. Neither must your pages contain any special markers for their automatic distinction or adjustment nor do the checkmarks have to adhere to a pre-defined grid. Furthermore, you do not have to use FormPro to design your questionnaire but can scan it or import it from a PDF file. In most cases you can use your existing layout without any modifications. This enables you to use FormPro even when your questionnaires have already been printed and distributed to the participants of your survey.

However, if you are in the comforting situation to have influence on the design of the questionnaire you should pay attention to the following hints. This will make it easier for your participants to fill out the questionnaires and will later reduce the processing efforts considerably.

Text anchors at top and bottom

By default, FormPro uses the pre-printed text for distinguishing and adjusting pages. It is therefore very important that each page contains some text in the header and footer areas, for example hints for the person filling out the questionnaire. The text must be part of the blank form known to FormPro and should be updated even after minor layout changes. Otherwise, there may be problems with page adjustment.

Symbolic anchors for multi-lingual forms

If your survey is multi-lingual you should create a separate form template for each language so FormPro can use the pre-printed text for page adjustment. Only if all language variants are superimposable such that all fields are at identical positions you can use a single, common template. In this case, you should print filled squares, angles or cross hairs into all four corners of each page. Make the symbols between 5 and 10 mm in size and let FormPro use them for page adjustment.

Divide handprint fields

If your questionnaire contains handprint fields (customer ID, date, time etc.) you should divide these fields into boxes for the individual characters. This will inform your participants which entry format is expected and they will automatically fill out the fields much more accurately. Apart from that, touching characters or writing outside the designated areas will be avoided. Make sure the boxes are at least 10 mm high and 8 mm wide!

Dropout colors for handprint fields

To achieve the best possible accuracy for reading handwritten values you can print the frames or boxes of these fields in a scanner dropout color (red or green). When you then scan your questionnaires in color mode FormPro can filter out the frames completely without damaging the filled-out characters. Dots within date or time fields, however, should still be printed in black to be recognized.

Creating Reading Templates

Befor you can start with capturing and evaluating filled-out questionnaires and surveys you must create a reading template in the definition mode of FormPro. This template consists of a blank form with all pages of the questionnaire, the positions of the capturing fields and a link to a database, CSV file or Excel sheet to export your data to. Please have a look at the following tips before you create your first reading template!

Use a blank form

Whenever you create a new form template in FormPro please use a blank, that is not a filled-out document! The best way of doing this is to import a PDF file with your questionnaire layout or to scan a blank questionnaire. Unfortunately, the most common user mistake is to use an already filled-out document which will often result in problems with data filtering.

Remove variable text

Please ensure that the blank form you provide to FormPro contains only text that will appear on all questionnaires at exactly the same positions and with the same formatting. If your layout has been changed since you created the template please update your blank form! On the other hand, if the blank form contains some text that does not appear on all questionnaires please remove it using the eraser feature!

Keep the frames

Please do not remove lines, frames, pre-printed text or check boxes that are within or close to reading fields! This kind of contents will be used by FormPro for field positioning and filtering.

Link your database

Capturing data from a survey will always result in a table with one record per questionnaire. This table will be the basis of your evaluation. FormPro will write the table to a CSV file by default but can just as well export to an XML file, Excel sheet or via ODBC into a database. When exporting to Excel or a database please create the table with all its columns first and link your form in FormPro with that table before you create reading fields. When exporting to CSV or XML files this step is not necessary.

Create reading fields

The final step is to create reading fields on each page. To do so draw rectangles with your mouse around the areas you want to read and assign the correct type and a database field to each area. Checkmarks that are mutually-exclusive (multiple choice) can be grouped by assigning the same database field to all of them.

Scanning Questionnaires

To scan the filled out questionnaires you can use a document scanner with automatic document feeder (ADF) or a multi-functional device (MFP). Which kind of device you use only has a minor impact on the quality of the result as the requirements on image quality are rather low for data capturing. If your scanner is connected to your computer via USB or is within the same network you can control it directly from FormPro. In all other cases, you should scan your documents with some other software or directly on the multi-functional device and save the pages batch-wise in PDF or TIFF format. Before you start, however, please consider the following hints!

Color depth and resolution

For most surveys it will be convenient to scan the filled-out questionnaires in gray-scale mode with a resolution of 300 dpi. Only if you have red or green lines or boxes around the handprint fields and if your scanner is not capable of filtering those colors you should scan in color mode. Please do not use the pure black-and-white mode as this would result in information loss.

ISIS vs TWAIN

If your scanner can be controlled directly from FormPro via USB or your local network you should use an ISIS driver instead of TWAIN. In this case, the lesser-known ISIS is not an islamic terror organization but a powerful device interface for document scanners. You should find the drivers for both standards on the website of your scanner manufacturer.

Consider compression

Especially when scanning with other software or on multi-functional devices you should pay attention to the image file compression. Often, scanners will use lossy compression methods even for PDF or TIFF files to reduce the file size. This may result in blurred letter edges and can degrade recognition quality. Please use a lossless compression method or select maximum image quality!

Capturing Data

The decisive step of each survey evaluation is the extraction of data and the conversion of images into records. This takes place in the processing mode of FormPro when filled-out questionnaires are scanned or image files are processed. In this phase, page by page will be adjusted, assigned to the corresponding page of the blank form and analyzed in detail. The data entries and checkmarks will be recognized using OCR, ICR, OMR and barcode technology.

As a result, FormPro will create one record per questionnaire and will arrange these records in a table. When you click on any value the corresponding image section of the scan will be displayed next to the table making the original documents dispensable. Any unrecognized values or ambiguous checkmarks will be displayed in red and can be selected by pressing the Enter key. Once you have verified and corrected the data you can export your records into a database, Excel sheet or CSV file and start with your actual evaluation.

Identify problems

Let FormPro guide you in the search for unrecognized or ambiguous values. To do so click in the first cell of the table and press the Enter key to get to the next problem. Correct any errors either in table view or click on checkboxes directly in image view. Continue this process until there are no more errors. If your questionnaire contains many different types of fields it may be more effective to search for problems column by column.

Automatic error detection

To ease the search for errors you should define as many rules as possible within your reading template. You can, for instance, define the length or value range in the zone settings or assign lookup lists. For structured data like customer IDs, date and time values it can make sense to define their format by providing a Regular Expression (“Syntax” in FormPro). Using these rules FormPro can identify erroneous values and mark them red.

Export image files

FormPro links each questionnaire to the image file it originated from or that was saved during scanning. If you want to keep those links to later assign the exported records to the corresponding questionnaires you can insert a special image file zone at any position into your form template and activate the Save whole image option. This will result in a new column in your export table with the image file names.