(1)
Laboratory of Anatomy, Biomechanics and Organogenesis, Université Libre de Bruxelles, Brussels, Belgium
(2)
Department of Electronic and Informatics – ETRO, Vrije Universiteit Brussel, Brussels, Belgium imec, Leuven, Belgium
“Let’s go invent tomorrow instead of worrying about what happened yesterday.” – Steve Jobs
1.1 The Computers
Without computers anything of this would be possible! The growth of the computational power of modern computers is exponential implying continuously increasing progress in the gaming industry as well. The first computer—the ENIAC (Electronic Numerical Integrator and Computer)—has been built in 1946 and occupied a full room for a weight of 30 Tons. The computer games evolved from early days of computers and forked from research applications like simulations and demonstrations developed to entertain the public. In 70 years, the computational power has been drastically increasing and we moved from text-based games to realistic virtual and immersive environments. However, considering the exponential growth, in a few decades will people look at the current games the same way as we look to the “Pong” or the “Pacman.” In parallel to this growth, the size and the price of the technology is decreasing at the same rate.
The first microprocessor was built in 1971; since that date the power of processors have been increasing exponentially. Moore’s law stipulated that the power of computer is doubled every 18 months since every innovation boosts the creation of new products (Moore 1965).
The increase of power calculation allowed the development of applications requiring a lot of computational power such as visual graphics, particularly important for the development of video games.
1.2 The Graphical (R)evolution
The evolution of computer graphics can be seen more easily than the computational power. The first video games were developed from the 1950s and evolved mainly from research/instructional programs, simulations, and demonstrators. They were mostly text based shaping its appearance to remind simple geometrical objects1. The need for visualization required for many applications created the basis for 2D graphics technology. Associated to this technology’s evolution, the scenario and the complexity of the games never stopped and in the modern games the player is literally immersed in a gaming environment which looks closer to the reality. There are plenty of serious games that are developed and used in various fields (e.g., to increase workers’ safety by simulating of various situations, to increase performance in sports (e.g., driving simulation), to practice safely prior being in real situation (e.g., flight simulator training, etc.)).
1.3 The Three-Dimensional Environment
The 3D computer graphics, pioneered by William Fetter, put extra demand on the computational resources creating a gap of few decades between the first computers and consumer 3D applications. Although the first publicly available 3D graphics software and games is from the 1970s, the popularity rose in 1990s and with the new millennia. Wide public could observe the high-quality 3D animations with series of successful blog buster movies (Tron, Terminator 2, Jurassic Park, Toy Story, Avatar, etc.). However, the industry that drives and demonstrates the evolution of 3D technologies the most is video gaming industry. The graphics has evolved from games that used only a few simple primitives (e.g., lines in the game Battlezone) through richer graphics (e.g., Wolfenstein 3D) to realistic environments with procedurally generated content and advanced artificial intelligence (e.g., The Elder Scrolls IV: Oblivion). The power to render realistic environments is now embedded into many households in the form of a game console or an entertainment system (e.g., MS Xbox One, Sony PlayStation 4). The upcoming trend is involving immersive 3D in the form of virtual reality, often based on a stereoscopic image requiring headset or glasses. Increased immersion is often achieved by including natural interactions (for humans) between humans and computer. This involves free movement of the player’s body tracking allowing new and unconventional kinds of interactions.
Majority of the modern computer games use advanced 3D systems (LaViola 2008), and it is expected that in the near future this technology will be adopted within our daily activities including health-related applications. Scientists from around the world are working on many examples, e.g., for people with depth perception impairment or other cognitive and physical disabilities.
A lot of the low-cost gaming devices have already spawned completely new research tracks. Gaming input devices, often resemble medically used devices. Open the potential for everyday use. For instance, MS Kinect allows real-time body tracking, the Wii balance board tracking center of the pressure, but also smart phones—equipped by IMU—tracking orientation and positional information of the device. Also, novel output devices offer potential changes in perception on tap. For instance, the Oculus Rift™ is a pioneering device, a headset using virtual reality, that allows (along an immersive experience) rapid changes in environment at rate that would not be possible in the physical world.
Scientists and researchers are adopting these disruptive technologies in plenty of research projects and clinical application. By using these devices allowing complete immersion of the patients, they allow them to experience various situations that could be hardly achieved during traditional treatment or rehabilitation exercises. Therefore, some other neuronal circuits and zones of the brain can be trained.
1.4 The Internet
The development and the generalization of high-speed Internet connection had changed the traditional games to a tool for social interaction. With the development of smartphones and mobile broadband connection people can interact not only in the real-time but also on the go. This network enables interconnect mobile devices with remote servers and server farms with much performant machines—distributing the high computational demand in the mobile devices. Therefore, a new approach to game development emerges where most of the computation can be done on the cloud.
Offloading the mobile devices to the cloud helps the game development studios establish a new business model where the game is offered as a service rather than of-the-shelf package. Another consequence is that the cloud empowers even broader social interactions among the players. The Massively Multiplayer Online Game (MMOG) and the Massively Multiplayer Online Role-Playing Games (MMORPG) are now one of the keystones in the video games industry. The revenue generated by MMOG and MMORPG never ceased to increase since introduced to the market in the 1990s. In 2012, the revenue for games industries was about 21 billion $2; the part for MMOG and MMORPG is approximately 25%. In 2014, the numbers were 25.3 billion $ with 27% for online games3.
Of course, serious games specifically developed for patients currently do not reach the same levels but we can observe the same trends in the serious games focused on health. The MMOG have even the potential to engage the disabled patients by supporting interactions with their healthy counterparts within a single environment. Through the anonymity of the Internet, this may offer a new socializing tool transparent in terms of the game, but still containing the serious aspects of the therapy.
1.5 The Commercial Video Games
Commercial video games have significantly evolved over the last decades to one of the biggest markets (generated almost $100 billion in revenues in 2016 globally4) in the entertainment industry with the accelerating annual growth of 8.5%. However, as the world becomes more connected and more of our activities are performed on mobile devices, some experts predict decline in sales of gaming consoles and game-purposed computers. The manufacturers of the gaming hardware therefore tend to concentrate their focus on the most compelling gaming experience and disruptive innovations. For instance, game controllers (Nintendo Wii Fit™, Microsoft Xbox Kinect™, Sony Play Station™, etc.) change the traditional passive video gameplay (i.e., controlling the game with mouse or a keyboard) to an active experience where the players are required to move in order to interact with games.
The new trends in game controllers is not only transforming the way the games are played, but it also led scientists to focus their attention on these devices due to their potential in clinical and/or scientific applications. Businesses have already started to focus on providing clinically relevant data collected from these devices. Therefore, in the next sections we are going to briefly present the game consoles coming with novel controllers that gained popularity in motion analysis and physical therapy in particular. We also describe our expectations on how these controllers could be, or already are, used for health-related applications mainly in physical rehabilitation.
1.5.1 From Games to Sciences?
This new trend of games controlling and controllers has not only completely modified the way of playing video games, but also led scientists to focus on these new devices for future potential clinical and/or scientific applications. Indeed, we are going to see that some game controllers present interesting characteristics similar to the devices that are used to perform motion analysis in clinics. Therefore in the next subchapters, we are going to present game consoles and controllers from a gaming point of view and then describe how these controllers could be used, or are already used, for other applications such as motion analysis, patients evaluation, and follow-up or rehabilitation.
1.5.2 Nintendo Wii
1.5.2.1 The Gaming Aspects
The Nintendo Wii is one of the first video game consoles using motion of the human body and novel controllers that were successfully adopted by the mass market. This console has been released on November 2006 and turned out to be a success with immediate worldwide sales. Unlike previous consoles, the games have been controlled by the Wii remote controller which detects motions of the players in three dimensions using inertial measurement sensors. In July 2007, the Wii Balance Board (WBB), a force plate that measures center of pressure, was released allowing players to control the games using center of pressure displacement. Most of the games using these controllers focus on fitness-related games, body movement, and burn calories of the player.
1.5.2.2 The Scientific Aspects
The evaluation of balance and postural control is an important field in various domains such as health (e.g., preventions of falls in elderly people), rehabilitation (e.g., balance training after stroke), and sports (e.g., to increase athlete’s performance or decrease injuries’ risk). Despite this potential huge field of application, it appears that balance assessment using a force plate (FP) (i.e., during quantitative functional evaluation) in laboratory is not as used as it should be in clinics for patients’ evaluation or follow-up. Despite this, the measurement of the center of pressure (CP) using FP is considered as gold standard to assess balance (Haas and Burden 2000). This is probably due to the fact that FPs are, most of the time, not transportable due to their embedment in the laboratory floor. Their relatively high price is also blocking their widespread use outside the laboratory. Access to this kind of tool is therefore limited and does not allow regular measurement for patient follow-up or evaluation of a treatment if a specially equipped laboratory is not available.