Touch interfaces are replacing physical buttons, dials, and switches in the new generation of cars, aircraft, and vessels. However, vehicle vibrations and accelerations perturb finger movements and cause erroneous touchscreen inputs by users. Furthermore, unlike physical buttons, touchscreens cannot be operated by touch alone and always require users' visual focus. Hence, despite their numerous benefits, touchscreens are not inherently suited for use in vehicles, which results in an increased risk of accidents. In a recently awarded research project titled "Right Touch Right Time: Future In-vehicle Touchscreens (FITS)", we aim to address these problems by developing novel in-vehicle touchscreens that actively predict and correct perturbed finger movements and simulate physical touch interactions with artificial tactile feedback. 

This study explores the impact of whole-body vibrations induced by external vehicle perturbations, such as aircraft turbulence, on the perception of electrovibration displayed on touchscreens. Electrovibration holds promise as a technology for providing tactile feedback on future touchscreens, addressing usability challenges in vehicle cockpits. However, its performance under dynamic conditions, such as during whole-body vibrations induced by turbulence, still needs to be explored. We measured the absolute detection thresholds of 15 human participants for short- and long-duration electrovibration stimuli displayed on a touchscreen, both in the absence and presence of two types of turbulence motion generated by a motion simulator. Concurrently, we measured participants' applied contact force and finger scan speeds. Significantly higher (38%) absolute detection thresholds were observed for short electrovibration stimuli than for long stimuli. Finger scan speeds in the direction of turbulence, applied forces, and force fluctuation rates increased during whole-body vibrations due to biodynamic feedthrough. As a result, turbulence also significantly increased the perception thresholds, but only for short-duration electrovibration stimuli. The results reveal that whole-body vibrations can impede the perception of short-duration electrovibration stimuli, due to involuntary finger movements and increased normal force fluctuations. Our findings offer valuable insights for the future design of touchscreens with tactile feedback in vehicle cockpits. 

Thermal feedback has been proven to enhance user experience in human-machine interactions. Yet state-of-the-art thermal technology has focused on the single finger or palm in static contact, overlooking dynamic and multi-finger interactions. The underlying challenges include incompatible designs of conventional interfaces for providing salient thermal stimuli for such interactions and, thereby, a lack of knowledge on human thermal perception for relevant conditions. Here we present the ThermoSurf, a new thermal display technology that can deliver temperature patterns on a large interface suitable for dynamic and multi-finger interactions. We also investigate how user exploration affects the perception of the generated temperature distributions. Twenty-three human participants interacted with the device following three exploration conditions (static-single finger, dynamic-single finger, and static-multi finger) and evaluated 15 temperature differences. Our results showed that human sensitivity against thermal stimuli is significantly greater for static-single finger contact compared to the other tested conditions. In addition, this interaction type resulted in higher thermal discrimination thresholds than the ones reported in the literature. 

Pressing the fingertips into surfaces causes skin deformations that enable humans to grip objects and sense their physical properties. This process involves intricate finger geometry, non-uniform tissue properties, and moisture, complicating the underlying contact mechanics.  Here, we explore the initial contact evolution of dry and hydrated fingers to isolate the roles of governing physical factors. Two participants gradually pressed an index finger on a glass surface under three moisture conditions:  dry, water-hydrated, and glycerin-hydrated.  Gross and real contact areas were optically measured over time, revealing that glycerin hydration produced strikingly higher real contact area, while gross contact area was similar for all conditions. To elucidate the causes for this phenomenon, we investigated the combined effects of tissue elasticity, skin-surface friction, and fingerprint ridges on contact area using simulation.  Our analyses show the dominant influence of elastic modulus over friction and an unusual contact phenomenon, which we call friction-induced hinging.

Electrovibration holds great potential for creating vivid and realistic haptic sensations on touchscreens. Ideally, a designer should be able to control what users feel independent of the number of fingers they use, the movements they make, and how hard they press. We proved for the first time that both the number of contacting fingers and whether each finger moves significantly affect what the user feels. Part of this change comes from the fact that relative motion greatly increases the electrical impedance between a finger and the screen. These findings help lay the groundwork for delivering consistent haptic feedback via electrovibration.

One may notice a relatively wide range of tactile sensations even when touching the same hard, flat surface in similar ways. Little is known about the reasons for this variability, so here we investigated how the perceptual intensity of light stickiness relates to the physical interaction between the skin and the surface. Our results show that stickiness perception mainly depends on the pre-detachment pressing duration, the time taken for the finger to detach, and the impulse in the normal direction after the normal force changes sign; finger-surface adhesion seems to build with pressing time, causing a larger normal impulse during detachment and thus a more intense stickiness sensation. 

Effective utilization of electrovibration can only be accomplished by simultaneously investigating both the physical and perceptual aspects of the finger-touchscreen interaction. Towards this goal, present work blends the available knowledge on electromechanical properties of the human finger and human tactile perception with the results of new psychophysical experiments and physical measurements. By following such an approach that combines both theoretical and experimental information, the study proposes new methods and insights on generating realistic haptic effects, such as textures and edges on these displays. 

Generating realistic texture feelings on tactile displays using data-driven methods has attracted a lot of interest in the last decade. However, the need for large data storage and transmission rates complicates the use of these methods for future commercial displays. Here, we propose a new texture rendering approach that can compress the texture data significantly for electrostatic displays. Using three sample surfaces, we first explain how to record, analyze and compress the texture data, and render them on a touchscreen. Then, through psychophysical experiments conducted with nineteen participants, we show that the textures can be reproduced by a significantly fewer number of frequency components than the ones in the original signal without inducing perceptual degradation. 

Here, we investigated whether it is possible to change detection threshold of electrovibration at fingertip of index finger via remote masking, i.e. by applying a (mechanical) vibrotactile stimulus on the proximal phalanx of the same finger. We found that vibrotactile masking stimuli generated sub-threshold vibrations around fingertip, and hence did not mechanically interfere with the electrovibration stimulus. However, there was a clear psychophysical masking effect due to central neural processes. Electrovibration absolute threshold increased approximately 0.19 dB for each dB increase in the masking level.

Realistic display of tactile textures on touch screens is a big step forward for haptic technology to reach a wide range of consumers utilizing electronic devices on a daily basis. Since the texture topography cannot be rendered explicitly by electrovibration on touch screens, it is important to understand how we perceive the virtual textures displayed by friction modulation via electrovibration. Here, we investigated the roughness perception of real gratings made of plexiglass and virtual gratings displayed by electrovibration through a touch screen for comparison. The results showed that the roughness perception of real and virtual gratings are different. We argue that this difference can be explained by the amount of fingerpad penetration into the gratings. For real gratings, penetration increased tangential forces acting on the finger, whereas for virtual ones where skin penetration is absent, tangential forces decreased with spatial period. 

Future touch screen applications will include multiple tactile stimuli displayed simultaneously or consecutively to a single finger or multiple fingers. These applications should be designed by considering human tactile masking mechanism since it is known that presenting one stimulus may interfere with the perception of the other. Here, we investigate the effect of masking on tactile perception of electrovibration displayed on touch screens. Moreover, in order to investigate the effect of tactile masking on our haptic perception of edge sharpness, we compared the perceived sharpness of edges separating two textured regions displayed with and without various masking stimuli. Our results suggest that sharpness perception depends on the local contrast between background and foreground stimuli, which varies as a function of masking amplitude and activation levels of frequency-dependent psychophysical channels.

Here, we investigated the effect of input voltage waveform on our haptic perception of electrovibration on touch screens. We found that the subjects were more sensitive to stimuli generated by square wave voltage than sinusoidal one for frequencies lower than 60 Hz. Using Matlab simulations, we showed that the sensation difference of waveforms in low fundamental frequencies occurred due to the frequency-dependent electrical properties of human skin and human tactile sensitivity. To validate our simulations, we actuated the touch screen at the threshold voltages and then measured the contact force and acceleration acting on the index fingers of the subjects moving on the screen with a constant speed. We analyzed the collected data in the frequency domain using the human vibrotactile sensitivity curve. The results suggested that Pacinian channel was the primary psychophysical channel in the detection of the electrovibration stimuli caused by all the square-wave inputs tested in this study.