In the framework of the European project GENESIS, led by the BCI team of the University of Lille (UMR CRIStAL) in collaboration with Inria, ETH Zürich, Koç University and University of Essex, we propose a thesis offer on the subject :
“Detecting neuromarkers through a passive Brain-Computer Interface for improving user experience in Virtual Reality”
If you wish to apply for the subject, please contact us by email at first. The description of the research topic is also available in a PDF document.
Abstract
This subject of this thesis is at the middle of two research domains, namely Brain-Computer Interfaces (BCI) and Virtual Reality (VR).
On the one hand, a BCI analyzes users’ brain activity to enable them to interact, without muscle activity, with a computer system [1]. Originally designed to assist people with severe motor disabilities, a new trend is emerging for the use of BCIs by a wider public. For example, through passive BCI systems, it is possible to transparently monitor users’ mental states to estimate their mental workload or their level of engagement in a task [2].
On the other hand, VR allows users to be immersed in computer-generated 3D environments, enabling total control over the displayed environment, so that the users feel present, or more precisely, immersed, in the virtual space. This immersion opens up new fields of application, ranging from training and education to social networking and entertainment [3]. Given the growing interest of society and the investments of major industrial groups, VR is considered a major revolution in human-computer interaction.
To date, however, VR is struggling to reach the expected level of democratization and remains largely confined to the state of an entertaining experience. This can be explained by the difficulty in characterizing the mental state of users during interaction and the lack of adaptation, inherent in the presentation of generically developed virtual content. For example, studies have shown that multiple users may have very different feelings when experiencing the same VR system [4].
In addition, about 60% of users may suffer from “cybersickness” [5], a term that refers to the set of deleterious symptoms that can occur after more or less prolonged use of virtual reality systems. Users can also suffer from breaks in presence and immersion, due to anomalies in rendering and interaction, which can lead to a bad feeling of embodiment in their virtual avatars. In both cases, the user experience can be severely affected because the VR experience relies heavily on, among other things, the concepts of telepresence and immersion.
The objective of this thesis is to study and design VR systems that analyze the electrophysiological activity of a user’s brain, through a passive BCI, in order to improve immersion in virtual environments. The objective will be to integrate into the VR system new ways to assess the user’s mental state through real-time classification of EEG data. The ultimate goal will be to improve user immersion by reducing or preventing cybersickness and increasing levels of embodiment through real-time adaptation of virtual content to mental states.
Work environment
The work related to this thesis will be carried out within the BCI team of the CRIStAL laboratory (University of Lille, CNRS). For more than ten years, the researchers of the BCI team have been studying the possibilities offered by BCIs to compensate for severe motor handicaps. In particular, they have evaluated the potential contributions of hybrid BCIs, acquiring a solid experience in the processing of EEG data, and in the hybridization of BCIs with various interaction technologies. The thesis is funded by the European project GENESIS, in partnership with Inria, ETH Zürich, Koç University and University of Essex.
Work description
In the context of the GENESIS project, the work related to this thesis will focus on the identification, characterization and real-time detection of neurophysiological markers associated with users’ mental states. The bibliographical study will focus on both methodological and theoretical aspects. First, it will be a question of establishing a state of the art of the different dimensions associated with the user experience in VR, the factors that influence their perception as well as the “offline” markers already highlighted. This will be the first step towards the design of an experimental protocol to study them. This experimental protocol, which will take place essentially in a virtual environment, will be used to capture electroencephalographic (EEG) data associated with different mental states of VR users. The in-depth analysis of these data will allow to highlight the existence of neuromarkers associated with the dimensions of the user experience, in particular the feeling of embodiment and cybersickness.
This data analysis will be carried out using signal processing techniques. New specific methods, based for example on Riemannian geometry tools, will be developed and validated in collaboration with the University of Koç and the University of Essex. It will be necessary to assess that it is possible to detect in real time neurophysiological changes according to different virtual environments and configurations, and thus to automatically detect changes in the user experience and to evaluate it objectively. Today, this evaluation is mainly based on subjective questionnaires, or on averaged experimental data. Through this thesis, we aim at proposing new objective metrics, able to evaluate in real-time the evolution of the feeling of embodiment or the risk of occurrence of cybersickness. Finally, the final objective will be to test different strategies and mechanisms, based on the literature in Human-Computer Interaction and in collaboration with ETH Zürich, allowing to re-engage the users in their experience or to reinforce their feeling of embodiment according to the results obtained by the quantification model.
Expected skills:
Technical skills:
- Notions of signal processing and/or machine learning,
- Python, Matlab and/or R development skills,
- Experience in conducting user studies and analyzing collected EEG data, as well as in Unity and C# development will be highly appreciated.
- Background in cognitive sciences and/or neurophysiology is optional.
Spoken languages: French/English
Contact
Bibliography
[1] Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical neurophysiology, 113(6), 767-791.
[2] Zander, T. O., & Kothe, C. (2011). Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. Journal of neural engineering, 8(2), 025005.
[3] Wexelblat, A. (Ed.). (2014). Virtual reality: applications and explorations. Academic Press.
[4] Dewez, D., Fribourg, R., Argelaguet, F., Hoyet, L., Mestre, D., Slater, M., & Lécuyer, A. (2019, October). Influence of personality traits and body awareness on the sense of embodiment in virtual reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 123-134). IEEE.
[5] Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and cybersickness in virtual reality are negatively related: a review. Frontiers in psychology, 10, 158.
Version française