# Plugin API
The usage of plugins has multiple advantages. For the user, they make it easy to turn features on and off as required. For the developer, it increases maintainability through separation.
Plugins can also be loaded at runtime, extending Pupil's functionality by sharing a simple Python file. See our pupil-community repository for a list of third-party plugins. See below on how to add them.
# Adding A Plugin
Each Pupil Core software creates its own user directory. It is directly placed in your
user's home directory and follows this naming convention:
Each user directory has a
plugins subdirectory into which the plugin files need to be
placed. The Pupil Core software will attempt to load the files during the next launch.
If the plugin was installed correctly, it should appear in the Plugin Manager
of the corresponding Pupil Core software. Check the log file (
~/pupil_<name>_settings/<name>.log) for errors if this is not the case.
The bundled applications use their own isolated Python environment, i.e. the plugin will
not recognize your local
pip installation! Any additional dependencies need to be
installed into the
plugins folder, next to the plugin.
For the plugin development process, we recommend to run from source.
Pupil is written in
Python 3.6, but no "heavy lifting" is done in Python. High performance computer vision, media compression, display libraries, and custom functions are written in external libraries or c/c++ and accessed though cython. Python plays the role of "glue" that sticks all the pieces together.
We also like writing code in Python because it's quick and easy to move from initial idea to working proof-of-concept. If proof-of-concept code is slow, optimization and performance enhancement can happen in iterations of code.
# Process Structure
When Pupil Capture starts, in default settings two processes are spawned:
# Eye Process
The eye process only has one purpose - to detect the pupil and broadcast its position. The process breakdown looks like this:
- Grabs eye camera images from eye camera video stream
- Find the pupil position in the image
- Broadcast/stream the detected pupil position.
See the terminology section for the difference between pupil and gaze data.
# World Process
This is the workhorse.
- Grabs the world camera images from the world camera video stream
- Receives pupil positions from the eye process
- Performs calibration mapping from pupil positions to gaze positions
- Loads plugins - to detect fixations, track surfaces, and more...
- Records video and data. Most, and preferably all coordination and control happens within the World process.
# API Reference
All plugins are required to be subclasses from the root Plugin class. It is available during runtime:
from plugin import Plugin class MyCustomPlugin(Plugin): pass
# Pupil Detection Plugins
Starting with version v2.6, Pupil Core supports custom pupil detection plugins that run in the eye process. These plugins can be used to implement custom algorithms for extracting pupillometry data from the eye images and feeding it to the rest of the gaze mapping pipeline.
Pupil detection plugins are supported in Pupil Capture, Pupil Player, and Pupil Service. Similar to regular user plugins, custom pupil detection plugins should be placed inside
pupil_capture_settings/plugins for Pupil Capture,
pupil_player_settings/plugins for Pupil Player, and
pupil_service_settings/plugin for Pupil Service, all of which are located in the user directory of your system. The plugins are automatically loaded when any of the applications are started.
Conceptually, pupil detection plugins are wrappers around pupil detectors to make them work within Pupil Core. Pupil detectors are objects that are responsible for extracting the raw pupillometry data from the eye images. Pupil detector plugins use pupil detectors for extracting the raw data, package that data into pupil datums compatible for the gaze mapping pipeline, and integrates the detector into the rest of the lifecycle of a plugin.
See Pupil Datum Format for a list of required keys of the pupil datum dictionary returned by the
from pupil_detector_plugins import PupilDetectorPlugin class MyCustomPupilDetectorPlugin(PupilDetectorPlugin): def __init__(self, g_pool): super().__init__(g_pool) # In some cases, it might be desirable to disable other pupil detectors running in the eye process # e.g. to increase performance on systems with limited computing resources. # In such cases, these pupil detection plugins can be disabled with the helper method: self._stop_other_pupil_detectors() def _stop_other_pupil_detectors(self): # Getting the Plugin_List instance from the g_pool object plugin_list = self.g_pool.plugins # Iterating over every plugin in the Plugin_List instance for plugin in plugin_list: # Deactivating every PupilDetectorPlugin instances except self if isinstance(plugin, PupilDetectorPlugin) and plugin is not self: plugin.alive = False # Forcing Plugin_List instance to remove deactivated plugins plugin_list.clean() @property def pupil_detector(self): # This read-only property must be implemented by the custom subclass. # # Returns an instance of pupil detector; # See: https://github.com/pupil-labs/pupil-detectors pass def detect(self, frame, **kwargs): # This method must be implemented by the custom subclass. # # Returns a pupil datum dictionary containing the pupil location and related metadata. # See Pupil Datum Format for a list of required keys: # https://docs.pupil-labs.com/developer/core/overview/#pupil-datum-format pass
See examples of custom pupil detection plugins here.