In this work it is proposed to support MARG-sensor based heading estimates by zero rotation updates measured and indicated by visual fixation detection. The detection of visual fixation (given in subsection A) is used to feed the previous estimate of heading from the MARG-sensor fusion process (given in subsection B) recursively to the filter itself to reduce accumulation of gyroscope bias related heading errors. We calculate an IMU heading vector ${}^{N}{\overrightarrow{N}}_{IMU,k}$ based on the previous estimate of the heading that represents the direction of a horizontalized heading vector in the North East Down (NED) frame. This heading vector can be used as a complete substitute to the magnetometer based horizontalized north direction vector in the MARG-equation of an adapated form of Madgwick’s Gradient Descent filter stage.

Synchronization of both systems is achieved through timestamp based message filtering. The data of the mobile eye tracker as well as the used MARG- or IMU-sensor should be accessible in real time by the manufacturers application programming interface (API) and provide a timestamp that can be used for synchronization processes, for example, using the message filter package from the robot operating system (ROS) framework. An angular rate threshold based switching can be either implemented on the MARG-sensor or host computer to account for possible latency issues between both systems. This threshold is based on the median head motion noise in static motion phases indicated by visual fixations. If the gyroscope raw signal exceeds the median noise level, the zero orientation update is turned off. This median noise threshold is also used to address the special case that the pupil position does not change while the head and eyes are following a moving target at the same rotational speed. During these special motions the magnitude of the measured angular rate will exceed the median gyroscope threshold which in turn disables the zero orientation update.

#### 3.1. Visual Zero Rotation Detection

The trigger signal for the zero rotation update is based on an online visual fixation detection algorithm that utilizes dispersion (spatial movement,

$t{h}_{d}$) and duration (

$t{h}_{t}$) thresholds to identify fixations. These thresholds define the total amount of allowed gaze or pupil position differences (

$\Delta p$) between time successive eye camera images (

$\Delta t$). The algorithm utilizes a sliding window which length is determined by the duration threshold

$t{h}_{t}$ and sampling frequency. Dispersion

p is calculated as the sum of the differences between consecutive pupil positions

where

x and

y are the eye tracker cameras pixel positions. The dispersion is compared to the maximum dispersion threshold

$t{h}_{d}$. Fixations are identified as such if the dispersion stays below

$t{h}_{d}$. This results in an expansion of the window to the right until the dispersion exceeds this threshold. If no fixation is detected the window does not expand but moves forward in time [

21].

This kind of algorithm has proven to be very accurate and robust regarding online fixation identification but needs careful parameter setting [

21]. While the visual fixation stays on a target and inside the dispersion threshold boundaries, the head is assumed to be stationary. The threshold parameter ratings for the magnitude of dispersion in time is given due to involuntary movement, for example, microsaccades and tremor. However, these involuntary movements usually consist of rather small duration in the range of 20–30 ms and amplitudes peaking in a visual angle of

$0.2\xb0$ [

22]. The fixation detection parameters should be chosen in a way that fixations are still detected even in the presence of microsaccades and tremor. A fixation is identified and labeled as such, as soon as the fixation duration threshold is reached. Upon this a trigger signal (

${S}_{t}$) is emitted indicating a zero orientation update cycle for the MARG-sensor data fusion process.

The trigger starts an acquisition cycle that stores gyroscope raw data while the fixation holds true. When a sufficient amount of gyroscope samples has been recorded, updated motion noise parameters are sent to the MARG-sensor to update the threshold to account for desynchronization and latency issues between both systems and their different sampling rates. This procedure ensures adaptive and individual noise parameterization for the current user and use case and enables a real-time support.

#### 3.2. MARG-Sensor Datafusion

In general the approach can be used independently of the underlying MARG-sensor data fusion process, since it indicates whether the users head is in dynamic or static motion phases. In this work, we exploit the approach on an adaptation of Madgwick’s gradient descent algorithm (GDA) based filter.

Figure 2 depicts the complete filter fusion approach that will be explained in detail in the following subsection.

As proposed by Madgwick et al. a quaternion

${}_{B}^{N}\mathbf{q}$ is computed by solving a minimization problem

that rotates a vector

${}^{N}\overrightarrow{d}$ into the orientation of a reference vector

${}^{B}\overrightarrow{s}$
where

${}_{B}^{N}$ denotes the orientation of the global navigation frame relative to the body frame and

${}_{B}^{N}\mathbf{q}$ is the four component quaternion

A possible solution to the optimization problem in Equation (

3) can be given by gradient descent based solving for the obtained magnetometer and accelerometer vector measurements respectively

where

${\mu}_{t}$ denotes the stepsize of the gradient function. For a complete mathematical explanation of the filter see Reference [

11] or Reference [

8]. The GDA filter stage computes a complete quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ either based on gyroscope, magnetometer and accelerometer (MARG-case) or gyroscope and accelerometer only data (IMU-case). This is to reduce errors in the heading estimate from magnetic disturbances but requires two different sets of equations [

8]. This is due to the missing magnetometer measurement vector in the IMU case set of equations and therefore needs an adapted objective function and Jacobian respectively.

In this work, we propose calculating an IMU heading vector that substitutes the magnetometer vector while magnetic disturbance is present to reduce the needed sets of equations as well as to guarantee convergence and a continuous quaternion solution to the minimization problem. We use the north direction vector

${}^{N}{\overrightarrow{N}}_{m}$ from the NED formulation through accelerometer and magnetometer measurements while no disturbance is present. The north direction vector is defined as the cross product between the down and east vector,

where the down vector is defined as the inverse of the acceleration measurement vector (

${}^{B}\overrightarrow{a}$)

and the east vector is defined as the cross product between the down vector and the magnetometer measurement vector (

${}^{B}\overrightarrow{m}$)

During rotation the acceleration vector will be subject to motion acceleration and therefore does not accurately reflect the direction of gravity. This effect however is typically reduced by low pass filtering the acceleration vector. Most modern sensors provide onboard low pass filter banks that can be configured by the user to the appropriate needs. Furthermore, the rotational accelerations will be rather small compared to the dominant acceleration originating from gravity. This is due to the small distance (in this case $0.1$ m) between the rotational center of the head and the position of the MARG-sensor resulting in minor inaccuracies during dynamic motion. The influence of these incaccuracies during dynamic motion is further reduced by the subsequent data fusion filter. The data fusion filter usually does emphasize gyroscope data integration during fast dynamic motion to reduce inaccuracies from the motion acceleration on the orientation estimation.

We calculate a substitute to the north direction vector, termed IMU heading vector ${}^{N}{\overrightarrow{N}}_{IMU}$, based on the orientation estimation from the gyroscope and accelerometer measurements. This is achieved through the following process.

We extract the heading information of the output quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ by calculating a three component vector (

${}^{N}{\overrightarrow{N}}_{IMU,k}$) describing heading information in the NED frame. First the heading information (yaw angle,

${\psi}_{E}$) from the quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ is converted to Euler angle representation

When a zero rotation update is triggered, the fusion process samples the current output angle ${\psi}_{E}$ from the last output quaternion ${}_{B}^{N}{\mathbf{q}}_{k}$ of the GDA filter stage and holds it while the trigger ${S}_{t}$ is true. The subscript E indicates that the angle $\psi $ is not updated if the sample and hold mechanism is activated. If the trigger signal is false, indicating head motion, the fusion process updates the angle ${\psi}_{E}$ with every new output quaternion ${}_{B}^{N}{\mathbf{q}}_{k}$.

Second we convert the iterative updated roll (

${\varphi}_{k}$) and pitch (

${\theta}_{k}$) angles derived from the current quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ by the following process

From the yaw (

${\psi}_{E}$) angle and the current roll (

${\varphi}_{k}$) and pitch (

${\theta}_{k}$) angles we build a new temporary quaternion shown in Equation (

12),

where

c and

s are sine and cosine functions respectively.

This quaternion is now applied to a x-axis unit vector because the north direction vector defines the sensors body x-axis, resulting in

where • indicates quaternion multiplication and

$\dot{\mathbf{q}}$ represents the conjugate quaternion to

$\mathbf{q}$. The vector

${}^{N}{\overrightarrow{N}}_{IMU,k}$ now represents the direction as a substitute to the magnetometer based north direction vector in the NED frame, as can be seen in

Figure 3.

Since the vectors

${}^{N}{\overrightarrow{N}}_{m}$ and

${}^{N}{\overrightarrow{N}}_{IMU}$ lie in the same plane it is possible to calculate a deviation angle (

$\u03f5$) that can be used to determine magnetic disturbance due to sudden changes in the direction of the north direction vector in contrast to the IMU heading vector. The deviation angle is calculated as follows

where • represents the dot product respectively.

If magnetic disturbance is present, the deviation angle $\u03f5$ will increase. If it exceeds a threshold $\Delta \theta $, the filter switches towards the virtual sensor vector based quaternion calculation and vice versa if it vanishes. This procedure enables the calculation of a complete and continuous quaternion solution that involves current sensor measurements from the accelerometer and the extracted heading information from the previous quaternion. It is possible to use the same set of equations without any adaptation and switch from magnetometer based north direction vector to the IMU heading vector without divergence of the quaternion. While the zero rotation trigger is enabled, the fusion process holds the recent calculated yaw angle ${\psi}_{E}$. This ensures that the GDA based calculation of the new quaternion ${}_{B}^{N}{\mathbf{q}}_{k}$ is less affected by possible drift in the heading direction due to uncorrected gyroscope bias but will however be corrected in the remaining axes through accelerometer updates and preserves a continuous solution and convergence. While no trigger is emitted, the fusion approach simply updates the measurement quaternion with every iteration based on either magnetic north direction vector when no disturbance is present or the IMU heading vector from the current orientation. It is known that Euler angle representation is subject to gimbal lock if two rotation axis align. This effect can be dealt with in two different ways. Either by designing the experiment in a way that does not include head rotations around the pitch exceeding $\pm {90}^{\circ}$, which causes gimbal lockin the chosen rotation order (Z-Y-X), or by formulating a quaternion based heading orientation estimation method. The quaternion based solution can be found in the following paragraph.

The heading information (yaw angle,

${\psi}_{E}$) from the quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ is converted to a quaternion representing only the yaw rotation (

${}_{B}^{N}{\mathbf{q}}_{\psi ,E}$) by deriving it from the corresponding Euler angle representation [

23]. A unit quaternion representing heading information (

${}_{B}^{N}{\mathbf{q}}_{\psi}$) is expressed as a rotation

$\psi $ around the z-axis

We can express the heading quaternion

${}_{B}^{N}{\mathbf{q}}_{\psi ,E}$ without trigonometric functions by substituting the corresponding Euler angle Equation (

10) with (

15) and normalize it, resulting in

To get the half rotation angle from Equation (

10) we add a unit quaternion and normalize the result

Likewise, to the Euler angle solution the zero rotation update trigger samples the current output quaternion ${}_{B}^{N}{\mathbf{q}}_{\psi ,E}$ from the last output quaternion ${}_{B}^{N}{\mathbf{q}}_{k}$ of the GDA filter stage and holds it while it is activated. If the trigger signal is deactivated, the fusion process updates the heading quaternion ${}_{B}^{N}{\mathbf{q}}_{\psi ,E}$ with every new output quaternion ${}_{B}^{N}{\mathbf{q}}_{k}$.

We calculate a quaternion (

${}_{B}^{N}{\mathbf{q}}_{\varphi ,\theta ,k}$) representing the iterative updated roll (

${\varphi}_{k}$) and pitch (

${\theta}_{k}$) angles based on the current quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$. This is achieved by removing the yaw component of the current quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$ through conjugate quaternion multiplication. We calculate a yaw quaternion

${}_{B}^{N}{\mathbf{q}}_{\psi ,k}$ based on the Equation (

16) and apply the conjugate to the current quaternion

${}_{B}^{N}{\mathbf{q}}_{k}$
where • indicates quaternion multiplication and

$\dot{\mathbf{q}}$ represents the conjugate quaternion to

$\mathbf{q}$.

The final quaternion

${}_{B}^{N}{\mathbf{q}}_{E,k}$ can be computed by combining the heading quaternion

${}_{B}^{N}{\mathbf{q}}_{\psi ,E}$ and the iterative updated quaternion representing only roll and pitch

${}_{B}^{N}{\mathbf{q}}_{\varphi ,\theta ,k}$ through quaternion multiplication,

The quaternion

${}_{B}^{N}{\mathbf{q}}_{E,k}$ now represents a complete orientation expressed as quaternion and combines heading information from the sample and hold mechanism with current updates regarding roll and pitch information from the filter’s output quaternion. This solution does not suffer from gimbal lock and can be used as the input quaternion to Equation (

13). Both methods, Euler angle conversion or the complete quaternion based heading calculation, are valid and can be chosen based on the desired application and design of experiment.