Reference Image Mapper ​

Reference Image Mapper is a powerful tool to automatically map gaze onto features/objects in the environment.

A heatmap of gaze data mapped onto the reference image can be generated, and mapped gaze and fixation data can further be downloaded as CSV files.

Setup ​

As described in the setup video, you will need two things in addition to your eye tracking recording(s) to produce a Reference Image Mapper enrichment:

1. A reference image
2. A scanning video of the object/feature(s) taken with Neon’s scene camera

In this guide, we will show you how the Reference Image Mapper can be used in a few different situations. This should help you get set up using the enrichment in your own testing environment!

Below, gaze is mapped in four very different environments: to a magazine cover, a basketball backboard, a supermarket shelf, and even a whole building!

Let's take a look at what the Reference Image and Scanning Recording look like to produce these heatmaps.

1. Magazine Reading ​

Reference image

First, we need a high-resolution .jpeg of the page.

Scanning video

Then, we need a scanning recording, about 15 s long, taken on a blank background and in good lighting (natural light works well). Note that the magazine page is clear with good contrast

2. Basketball ​

Reference image

Here we can take a photo of the basketball hoop and court background.

Scanning video

The scanning recording, in this case, is about 45 s long – note that the angles and distances cover what a player might see when dribbling towards the basket or taking a shot

3. Supermarket Shelf ​

Reference image

This photo captures the assortment of packagíng in the coffee aisle of a supermarket.

Scanning video

Notice that the scanning recording is fairly slow to reduce motion blur for this feature-rich shelf.

4. An Entire Building ​

Reference image

This is a photo of the entire building

Scanning video

We take a longer scanning recording, about 2 min. The angles and distances cover what a person might see whilst walking past or standing in front of the building.

Scanning Best Practices ​

Try to follow these best practices when recording a scanning video:

• Make the recording while holding Neon in your hand rather than wearing it on your head.
• Record the object of interest from all possible angles and from all distances a subject may view them. More diversity is better. Collecting sufficiently diverse viewpoints for a large object, like a building, may require you to move accordingly large distances.
• Move the glasses slowly while recording to avoid motion blur.
• Make sure to have good contrast and that your scene lighting during scanning is similar to that during mapping.

What works and what doesn’t?

• The scene must be feature-rich like in the examples above. There has to be enough salient visual content to produce a successful mapping.
• The scene needs to have relatively static features in the environment. If there is a lot of movement or the objects change in appearance or shape, the mapping can fail.

Validate the Gaze Mapping ​

To check if gaze has been mapped successfully, use the side-by-side view:

1. Select a recording.
2. Select the Reference Image Mapper Enrichment.
3. Select the Scene / Reference Image View.

Now when you play back the recording you can see where gaze is mapped to on your reference image for validation.

Occlusions ​

Sometimes an object will occlude the feature/object of interest. The reference image mapper may not recognize this, so a false positive mapping could occur.

If you need to remove the falsely mapped data points, there are a few workarounds.

1. Use Events and Sections to trim out false positive sections.
2. Manually remove the affected data points in the reference image mapper export, by finding the timestamp and deleting the row in the .csv export.

Repetitions ​

In cases such as supermarket shopping, where features of the environment like freezers and aisles are repetitive and overlapping, it can be useful to divide recordings into shorter sections using Events for enrichment. This way you can ensure gaze is only mapped to portions of the recording during which you know the user is looking at a particular part of the store.

Export Format ​

gaze.csv ​

This file contains all the mapped gaze data from all sections.

fixations.csv ​

This file contains fixation events detected in the gaze data stream and mapped to the reference image.

Reference Image ​

The reference image that was used for defining the enrichment. The file is named reference_image.jpeg|png.