Brain–Computer interfaces for games
Parul University, Vadodara.
Milan Jasani Hetvi Parikh
In past few years we have seen an increase interest in brain–computer interfacing for human–computer interaction and potential game applications. So far, we have only seen proof of concepts where a single BCI model is demonstrated to work as a simple control mechanism, as a measurement of user state. There has been very few attempts to design BCI games where BCI is considered to be one of multiple possible input modalities that can be used to control the game. In this paper we discuss current BCI research from the viewpoint of games.
Brain–computer interfacing, Multimodal interaction, Game design
Providing Brain Computer Interfacing (BCI) is finding its way in human computer interaction. This paper is discussed about the use of BCI in game and game like applications. These applications are not that different from medical or military BCI applications . Medical applications, aiming at handicapped patients with communication and movement skills, have seen many research efforts.
But this paper can also say that gamers, soldiers or, in fact, anybody is handicapped, in the sense that they will meet situations where it is desirable to have more skills and communication means than are available when using the usual verbal and nonverbal interaction modalities.
There are other reasons that make games, gamers and the game industry interesting for BCI research and development. In particular, gamers are early adopters. They are quite happy to play with technology, to accept that great efforts have to be made in order to gain minimal advantage, and they are used to the fact that games have to be mastered by training, allowing them to go from one level to the next level and to get a higher ranking than their competitors .
This Paper can use information made available to us from brain activity to adapt the interface to the user or to issue commands to the interface. Brain activity, whether it is consciously controlled and directed by the user or just recorded in order to obtain information about the users affective state, should be modeled and embedded in more general models of interaction in order to provide appropriate adaptation, feedback and a context where brain activity information is one of the many multi-modal interaction modalities that are provided to the gamer .
Gamer becomes occupied in the game, forgetting about time and the real world. Until now research aiming to understand this flow experience has concentrated on using more traditional physiological information, attempting to attain a user’s affective state from, for example, heart rate, sweating, respiration, and blood pressure .
The point of this review is to distinguish the value of significant games for training experts in the medical and, in particular, the surgical field. The first objective is to assess the framework of serious games for the purpose of training experts in medicine and their usability in surgical postgraduate training. The second objective is to assess the reliability of serious games as a educating method according to criteria regarded as best evidence .
BCI for controlling and adapting games
When we look at possible BCI games, we need a BCI that distinguishes and employee’s activities in different regions of the brain, and maps these activities to commands used to control or adapt a game. Generally, the BCI is a feedback loop that starts with brain signals generated by the user performing a conceptual task . When looking at game applications we need to take into account that gamers will prefer not to game in a MRI scanner, that gamers will not want to wear heavy head sets that measure their brain waves, and that not all gamers are yet willing to undergo surgery to have implants that will improve measuring their brain waves or improve their brain functions .
Figure 1 : BCI user EFG cap Interface
In this paper assume that brain activity is measured using an EEG cap. Such a cap has electrodes attached to it that measure activity in different regions of the brain. We can ‘read’ such information and make it available to a game engine that controls the environment, to use to adapt the game to a recognized conceptual state of the user or to translate consciously produced activity to commands that allow agamer to change the environment, to navigate, and to make decisions that allow him or her to survive in the game .
Internally evoked brain signals
This Paper distinguishes between internally and externally evoked brain signals. Motor imagery is one of the possible ways in which we can have internally evoked potentials. In our motor cortex we can find a mapping from possible movements made by our body parts to brain regions in this cortex .The mapping of ‘thoughts’ to actions in a virtual game environment does not necessarily have to be to ‘natural’. A gamer can be asked to perform a difficult calculation or to imagine a rotation of a geometric object . But preferably a required conceptual effort should be naturally place in a game because this helps a great deal to make the required game actions believable to the gamer and helps to keep the gamer occupied in the game. Conceptual efforts related to calculation or rotation, or other conceptual activities that have not yet been investigated, can become embedded in game environments .
Figure 2:Brain Performance
Feature of Motor Imagery
Figure 3: Motor Imagery
Externally evoked brain signals
There are also many forms of externally evoked potentials that can be exploited by game designers. When looking at evoked potentials we should take into account what can be measured in a game situation.The stimuli that cause these potentialscan be auditory or somatosensory or tactile, and combinations of these stimuli. Steady-state visually evoked potentials (SSVEPs), like flickering lights on a computer screen, have been used to allow a gamer to make decisive decisions .
Response of Whole Brain
Figure 4: Response of whole brain
BCI model categorized according to interaction characteristics.
(tiresomeness, tension , drift)
worldwide brain activity
(tension, awareness, composer)
(SSVEP, P300, N400)
Conceptual tasks (imaginary
movement, conceptual calculation”,
Figure 5: Example of games and virtual environment using imaginary
Games that employee BCI
In the previous section paper declare different ways in which BCI can be employed in games. In research and confirmation contexts there are many attempts in which researchers play with budding BCI game applications. Sometimes this is done for further developing ideas for research or medical software. Game-like conditions have also been designed to illustrate BCI research .
In this research application, EEG data is made available to the user in a graphical or auditory way with the point to instruct the user to perform in a desired way. Desirable behavior leads to immediate rewards, undesirable behavior is discouraged . neuron feedback is usually based on asking the user to control slow brainwave activity. Generally, slow brain activity is associated with relaxation, drowsiness, or sleep. Training brainwave activity may make it easier to enter a state of alertness, or make it easier to enter a relaxed state .
Figure 6:Medical Games diagram
A BCI game with brain activity related to real moment has been investigated in externally evoked potentials have also been used in game-like implementations .Simple and familiar video games have been given BCI control by researchers. The Berlin brain–computer interface has used motor imagery to play Pacman and Pong and comparably familiar games such as Tetris. Motor imagery applications exist for a first Person Shooter game..
Figure 7: How to do research?
Commercial game environments: controlling the game
More recently, however, we see large software companies such as IBM and Microsoft, big console game companies such as Sony and Nintendo, and smaller specialized companies such as Emotive, Neuron Sky, and OCZ entering and defining a market for commercial BCI games and other non-medical applications .
Mapping between mental tasks and the thought commands is done during a short training phase. One issue that may pop up with these kinds of games is how to remember the mental tasks and perform them in a consistent way. In the case of movements it may help to actually perform the corresponding movements .
Environmental communication need not be comic, nor does it need to be vocal. A swishing blade and a humming motor sound signify an industrial fan in Blood, while a phone may be ringing intermittently. A character may pass by an alien hive, with pods emitting a terrifying whine .
2) Nijholt, D. Tan, J. del R. Millán, B. Graimann, M. Jackson, Brain–computerinterfaces for HCI and games, in: Proceedings ACM CHI 2008: Art. Science.Balance, 2008, pp. 3925–3928.
3) Nijholt, D. Tan, G. Pfurtscheller, C. Brunner, J.D.R. Millán, B. Allison, B.Graimann, F. Popescu, B.Blankertz, K.-R. Müller, Brain–computer interfacingfor intelligent systems, IEEE Intelligent Systems (2008)
4) K. Gilleade, A. Dix, J. Allanson, Affective videogames and modes of affectivegaming: assist me, challenge me, emote me, in: Proceedings of DIGRA’2005
5) M. Lotze, U. Halsband, Motor imagery, Journal of Physiology 99 (4–6) (2006)386–395.
6) J.A. Pineda, D.S. Silverman, A. Vankov, J. Hestenes, Learning to control brainrhythms: making a brain–computer interface possible, IEEE Transactions onNeural Systems and Rehabilitation Engineering 11 (2) (2003) 181–184.
7) G. Müller-Putz, R. Scherer, G. Pfurtscheller, Game-like training to learn singleswitch operated neuroprosthetic control, in: BRAINPLAY 07 Brain–ComputerInterfaces and Games Workshop at Advances in Computer Entertainment”,2007.
8) Reuderink, A. Nijholt, M. Poel, Affective Pacman: a frustrating game forbrain–computer interface experiments, in: Intelligent Technologies forInteractive Entertainment, vol. 9, 2009, pp.
9) Bode S, Sewell DK, Lilburn S, Forte JD, Smith PL, Stahl J (2012) Predicting perceptual decision biases from early brain activity. J Neurosci 32:12488–12498. CrossRef Medline.
10) Kelly SP, Lalor EC, Reilly RB, Foxe JJ (2006) Increases in_ oscillatory powerreflect an active retinotopic mechanism for distracter suppression duringsustained visuospatial attention. J Neurophysiol 95:3844 3851. CrossRefMedline
11) Bergeron BP. Learning & retention in adaptive serious games. Stud Health Technol Inform 2008;
12) Badurdeen S, Abdul-Samad O, Story G, Wilson C, Down S, Harris A. Nintendo Wii video-gamingability predicts laparoscopic skill. Surg Endosc2010; 24(8): 1824–1828.
13) Schlickum MK, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L. Systematic video game trainingin surgical novices improves performance in virtual reality endoscopic surgical simulators: Aprospective randomized study. World J Surg 2009; 33(11): 2360–2367.
14) Boot WR, Kramer AF, Simons DJ, Fabiani M, Gratton G. The effects of video game playing on attention”,
15) memory, and executive control. Acta Psychol (Amst) 2008; 129(3): 387–398.
16) Anton Nijholt *, Danny Plass-Oude Bos, Boris ReuderinkUniversity of Twente, Faculty EEMCS, P.O. Box 217, 7500 AE, Enschede, The Netherlands
17) Simon P. Kelly1 and Redmond G. O’Connell 21Department of Biomedical Engineering, City College of the City University of New York, New York, New York 10031, and 2Trinity College Institute of
18) Neuroscience and School of Psychology, Trinity College Dublin, Dublin 2, Ireland