Human factors of Virtual-Reality (VR) and studying humans using VR

1.

Human factors of
Virtual-Reality (VR) and
studying humans using VR
Tadamasa Sawada
School of Psychology
Higher School of Economics

2.

Devices of Virtual-Reality (VR)
• Head-mounted display: a pair of small displays (one for
each eye; e.g. Oculus-rift, HTC-Vive, Sony-PS-VR, Google
Cardboard)
• Large display (or displays) with stereo-shutter/polarized
glasses (e.g. CAVE)
http://www.techworld.com/picturegallery/cloud/virtual-reality-gets-real-3400143/

3.

• Theories of VR
• Problems of VR
• Applications of VR

4.

From a Display to a VR-device
Large Field of View
(Movie)
Stereo
(3D-Movie)
Motion-parallax
Interaction
and more…
Virtual-Reality
© Nabeel Hyatt

5.

Large field of view
A large field of view is important for
perceiving “vection” (visually-evoked-action)
from a visual stimulus.
Väljamäe et al. (2008)
Palmisano et al. (2015)

6.

Stereo (binocular-disparity)
3~14% of people are stereo-blind/-deficient (Gaudia et al., 2014).
Synopter for 3D perception.
Beam-splitter
(Moritz von Rohr, patented by CarlZeiss, 1907)
Wijntjes et al. (2016), Koenderink, van Doorn, & Kappers (1994)
http://binnenland.eenvandaag.nl/radio-items/60101/ik_neem_je_mee_synopter_in_het_rijksmuseum
http://www2.aimnet.ne.jp/nakahara/3dart/3genri6.html

7.

Motion-parallax
https://youtu.be/Jd3-eiid-Uw
https://youtu.be/leg2gS6ShZw
We perceive 3D if an image on a display changes synchronously with our
head motion. If the image does not change…

8.

Images viewed from correct/incorrect positions
We perceive a correct 3D scene only from a correct viewing position.
If we change a viewing position, an image on an display should
synchronously change.

9.

Other types of sensory information for VR
• Vestibular information
• Somatosensory information
(including tactile information)
• Auditory information
• Haptic information
• Olfactory information
• Gustatory information

10.

Vestibular information
The vestibular system (semicircular-canals, utricle, saccule) is for
perception of body/head orientation and motion. In short, it is an
accelerometer for rotational (3df) and linear (3df) motions. It cannot
detect position or speed but acceleration.
© Nevit Dilmen

11.

Vestibular information
The vestibular system is controlled in scientific experiments often by (1)
using the gravity by changing a head/body orientation or (2) physically
moving the head/body.
Demo: Make your face upward. The vestibular system detects
acceleration from front to back of your face because of the gravity.

12.

Vestibular information
The vestibular system can be stimulated artificially using GalvanicVestibular-Stimulation and it causes reflexive actions of a body.
However, mechanism of this phenomenon and a relation between the
stimulation and the response of the system is still unclear (Fitzpatrick &
Day, 2006).
https://youtu.be/oefVaHDo5xg
https://youtu.be/yZ_G_zbObWU

13.

Somatosensory information
Somatosensory information is about changes of body parts
(inside/outside). For VR, mechanical movements of a body and physical
interaction of the body with a 3D scene (e.g. air pressure).

14.

Somatosensory information
Interface for the human somatosensory system.
http://www.techworld.com/picturegallery/cloud/virtual-reality-gets-real-3400143/

15.

Somatosensory information
Interface for the human somatosensory system.
http://kaji-lab.jp/en/index.php?research
https://www.xataka.com/gale
ria/birdly-vr/2/

16.

Visual, Vestibular, and Somatosensory information
Water: Vestibular = 1G↓, Somatosensory ≈ Neutral
(pressure from all directions)
If you have any vestibular problem, you should not do scuba diving.
It is dangerous because you may not well perceive vertical direction.

17.

Visual, Vestibular, and Somatosensory information
Space: Vestibular = 0G, Somatosensory = Neutral
(no pressure from any direction)
http://www.artemjew.ru/

18.

Visual, Vestibular, and Somatosensory information
Space: Vestibular = 0G, Somatosensory = Neutral
(no pressure from any direction)
This situation can be roughly emulated in an airplane dropping down.
http://www.faust-ag.jp/soul/adventure/soul142.php

19.

Visual, Vestibular, and Somatosensory information
Drop1: Vestibular = 0G, Somatosensory = motion↓ (wind)
Drop2: Vestibular = 1G↓, Somatosensory = motion↓
(Downward speed becomes constant because of air resistance)
Skydive Lillo Centro de Paracaidismo

20.

Visual, Vestibular, and Somatosensory information
Drop1: Vestibular = 0G, Somatosensory = motion↓ (wind)
Drop2: Vestibular = 1G↓, Somatosensory = motion↓
(Downward speed becomes constant because of air resistance)
Parachute1: Vestibular = xG↑, Somatosensory = motion↓ + hanged
Parachute2: Vestibular = 1G↓, Somatosensory = motion↓ + hanged

21.

Visual, Vestibular, and Somatosensory information
Drop1: Vestibular = 0G, Somatosensory = motion↓ (wind)
Drop2: Vestibular = 1G↓, Somatosensory = motion↓
(Downward speed becomes constant because of air resistance)
Parachute1: Vestibular = xG↑, Somatosensory = motion↓ + hanged
Parachute2: Vestibular = 1G↓, Somatosensory = motion↓ + hanged
http://www.ultravr.org/virtualreality/top-things-to-do-virtual-reality/

22.

Visual, Vestibular, and Somatosensory information
All visual, vestibular, and somatosensory (action) information contributes
to perception of a self-spatial position (Dorsal hippocampus).
Aghajan et al. (2015)
Ravassard et al. (2013)
Cushman et al. (2013)

23.

Visual, Vestibular, and Somatosensory information
Perception of a visual stimulus on a computer screen is mostly affected
by somatosensory/tactile information from a bar grabbed (Harris et al.,
2017).
The somatosensory information is more effective under a condition of
this study. It is unclear how much this result can be generalized to other
conditions. Namely, we may observe different results in an experiment
with different design.
Note that vestibular system can be tuned for ordinal voluntary head
motions (Barlow & Mollon, 1982). If acceleration of a head is very different
from this, the vestibular system may not work well.

24.

Visual, Vestibular, and Somatosensory information
Stimulating all the sensory systems properly in VR is still a difficult
problem. Especially, a viewing position is forced to change quickly by
some outside factor.
https://youtu.be/q4ZfnZf8osA

25.

Visual, Vestibular, and Somatosensory information
Stimulating all the sensory systems properly in VR is still a difficult
problem. Especially, a viewing position is forced to change quickly by
some outside factor.
http://japanese.engadget.com/2016/04/12/vr-vr-zone-project-i-can/

26.

Auditory information
Binaural recording considers shapes of the human ears.
http://twogoodears.blogspot.ru/2016_03_01_arch
ive.html
To listen to binaural
recording sounds, you
should choose proper audio
devices.

27.

Auditory information
Binaural sound in VR?
There are already some video games with Binaural sound.

28.

Problems of VR

29.

VR-sickness (Cyber-sickness)
Its “reported symptoms include stomach awareness, burping, salivation,
drowsiness, nausea and occasionally even vomiting, as well as
disorientation, dizziness, headaches, difficulty focussing, blurred vision,
and eyestrain” (Barrett, 2004, see also Read et al., 2015).
A user of a head-mounted display should take a rest without wearing
the head-mounted display every 20 minutes (Kennedy et al., 2000) or
whenever she/he experiences any nausea (Hu & Hui, 1997).

30.

VR-sickness (Cyber-sickness)
Sources and potential sources of VR-sickness
1) Conflict among Visual, Vestibular, and Somatosensory information
2) Temporal delay between head-motion and update of an image
3) Slow Screen refresh-rate
4) Low Screen resolution
5) Narrow Field of View
Display
6) Constant lens accommodation
7) Approximate model of the eye
Eye-ball

31.

VR-sickness (Cyber-sickness)
Sources and potential sources of VR-sickness
1) Conflict among Visual, Vestibular, and Somatosensory information
2) Temporal delay between head-motion and update of an image
3) Slow Screen refresh-rate
4) Low Screen resolution
5) Narrow Field of View
Display
6) Constant lens accommodation
7) Approximate model of the eye
Eye-ball
Problems 1) and 2) are especially critical for VR-sickness.
Problems 2), 3), 4), and 5) are very technical and are matter of
time and money. These problem are getting resolved.
Problems 6) and 7) may be factors causing VR-sickness. But, they
are theoretically difficult to be studied and to be resolved.

32.

Age limit for 3D contents?
e.g. Sony PSVR (12yo), Oculus-rift (13yo), Nintendo 3DS (6yo)

33.

Age limit for 3D contents?
Strabismus is developed before 6 years old of life (Hollwich, 1983/1986).
60% of cases before 2yo and the other 40% before 6yo.
http://d.hatena.ne.jp/koikesan/20140910
At least, there is one case report (Tsukuda & Murai, 1988). A 4yo child
developed strabismus (acute onset esotropia) after watching a 3D movie
(probably, 15 min long cartoon animation using anaglyph).
But why? We see everything with two eyes everyday!

34.

Development of the visual system
interpupillary distance (mm)
Inter-pupillary (inter-ocular) distance increases as a person gets older.
Pointer (1999)
https://www.slideshare.net/GauriSShrestha/
progressive-lens-measurement
Usually, the VR devices assume that the inter-pupillary distance is
60~65mm. It is too large for young children and this discrepancy causes
conflict of 3D information in the visual system.
The visual system of children should be flexible to accommodate with
the growing inter-pupillary distance.

35.

Two centers of the eye
The human eye has two centers: optical-center and rotation-center.
Object
Retina
A
d
c
e
B
C
b
a
Optical-center
D
E

36.

Two centers of the eye
The human eye has two centers: optical-center and rotation-center.
Object
Retina
A
d
c
e
B
C
b
a
Optical-center
Rotation-center
D
E

37.

Two centers of the eye
The human eye has two centers: optical-center and rotation-center.
Object
Retina
A
d
c
e
B
C
b
a
Optical-center
Rotation-center
D
E
(Fry & Hill, 1963)
The distance between these two centers is around 6.3mm (TschermakSeysenegg, 1952) or around 11mm (Bingham, 1993).

38.

Two centers of the eye
Retina
The human eye has two centers: optical-center and rotation-center.
Optical-center
Rotation-center
(Fry & Hill, 1963)
The distance between these two centers is around 6.3mm (TschermakSeysenegg, 1952) or around 11mm (Bingham, 1993).
However, the current VR devices assume that the distance between the
two centers is 0 (or use another approximation). This approximation can
be critical for head-mounted displays, whose screens are just a few cm
away from the eyes.

39.

Lens focus
If a lens of your eye focuses at some object, other objects at different
distances (closer/further) will be blurred in your retinal image.
http://www.youtube.com/watch?v=GJvH8CiExiM
Using the current VR devices, the eye-lens always focuses at a fixed
distance (display + display-lens) no matter how close/far something
appears.

40.

Lens focus
If a lens of your eye focuses at some object, other objects at different
distances (closer/further) will be blurred in your retinal image.
http://www.youtube.com/watch?v=GJvH8CiExiM
If the objects are placed at 2m and 5m, their images can be blurred
very differently (it depends on where you focus and your pupil size).
If the objects are placed at 200m and 500m, their images will be
almost equally blurred (or equally focused).

41.

Photosensitive epilepsy
Epileptic seizures can be induced by image flickers. An image flicker
between 2 Hz and 55 Hz should be avoided for the seizures. This
frequency range to avoid is specified in the section 508 of the
Rehabilitation Act in the USA (https://www.section508.gov/content/quickreference-guide, see also Walter, Dovey, & Shipton, 1946; Epilepsy action, 2010).
https://en.wikipedia.org/wiki/Denn%C5%8D_Senshi_Porygon
http://gizmodo.com/a-grand-jury-just-called-tweeting-an-animated-gif-assau-1793477149

42.

Applications of VR/AR

43.

Applications of VR/AR

44.

Applications of VR/AR for Scientific research
• Strabismus and Amblyopia (Vivid Vision; Foss, 2017; Gargantini et al., 2017)
http://3d4amb.unibg.it/principles.html

45.

Applications of VR/AR for Scientific research
• Psycho exposure therapy: Phobia, PTSD, Other types of anxiety
https://www.army.mil/article/144048/Virtual_R
eality_Exposure_Therapy_helps_resolve_PTSD
http://anxietynetwork.com/content/persistent-negativeattitude-can-undo-benefits-exposure-therapy

46.

Applications of VR/AR for Scientific research
• Phantom-limb (Dunn, 2017)
It can be an alternative method to Mirrortherapy for Phantom-limb. However, recent
results of meta-analysis (Rothgangel, Braun,
Beurskens, Seitz, & Wade, 2011) showed that effect
of Mirror-therapy is, at most, very weak.
https://www.pinterest.com/jeffreybado/phantom-pain/
http://www.bbc.co.uk/news/magazine-15938103

47.

Talking with people in different fields…

48.

My (very close) future presentation about VR
Using Virtual-Reality (VR) for Psychological studies: it is more than a
display hanging in front of your eyes
May/17 (Wed) 15:10
Room 106 in HSE School of Psychology (Volgogradskiy Prospect, 46B)
Abstract: Devices of Virtual-Reality (VR), especially head-mounted displays (HM-display), are
becoming common. How different are they from existing computer displays? What becomes
newly available with the VR devices? Can the VR devices provide any novel method to test
humans in scientific studies? What are human-factor problems/questions of the VR devices? I
will discuss these questions of the VR devices with reviewing recent psychological studies using
the VR devices and with introducing ongoing projects of VR at HSE.

49.

Спасибо!
[email protected]
[email protected]
https://www.hse.ru/en/staff/tsawada