Transparent interaction and human-robot collaboration for military operations

1.

Transparent interaction and
human-robot collaboration
for military operations
Shan G. Lakhmani, Julia L. Wright,
Jessie Y.C. Chen
U.S.
Army
Research
Laboratory,
Human
Research
and
Engineering
Directorate, Orlando, FL, United States
Student
OREL, Egor (1836231)
Teacher
IOCCHI, Luca

2.

WHY ROBOTS ARE IMPORTANT IN THE MILITARY?
THEY KEEP BOTH SOLDIERS AND
CIVILIANS SAFE
• clearing buildings;
• search and rescue in disaster
areas and battlefields;
• detonation
and
disposal
of
explosives;
• reconnaissance and surveillance.
THEY GO WHERE SOLDIERS CANNOT
THEY DO WHAT SOLDIERS CANNOT
THEY INCREASE THE SOLDIERS’
SCOPE OF INFLUENCE
gathering data to support soldiers’
situation awareness,
transporting soldier equipment,
distributing supplies to soldiers in
the most forward resupply positions,
facilitating commanders’ decisionmaking (collecting, organizing, and
prioritizing data)
keeping soldier’s safe by providing
greater stand-off distance from the
enemy for maneuvers and convoys

3.

ROBOTICS PROGRESS
TIMELINE
Past
Autonomous
teammates
Nowadays
HRI PROGRESS
TIMELINE
Future
Human and
robot share
the task load

4.

HR BEHAVIORS
TASKWORK
Work related activities needed to
accomplish the team’s goals
Robots are already being
designed to do taskwork
TEAMWORK
Coordination, sharing knowledge, and
all the other actions needed for
interdependent operation
Team behaviors, such as communication
and coordination, can be simulated by
robots to support important facets of
HRI, such as mutual predictability
and shared knowledge
Transparent HRI
Transparency is an emergent property of the HRI process whereby a human operator has a clear and
accurate understanding of how the robot gathers information, processes that info, and makes decisions
4

5.

AUTONOMOUS SYSTEMS
a system that can use its understanding
of the world, itself, and its situation
to independently compose and select
among different courses of action to
accomplish its goals
This combination of independence and
flexibility is particularly desirable
in complex, dynamic environments, the
kind in which the US military
anticipates operating
MIXED-INITIATIVE SYSTEMS
adaptive automation
(the level of automation is
changeable by the system)
+
adjustable automation
(the level of automation is
changeable by an external operator
or system)
Challenges: mode confusion – when an
operator believes the automation is in a
different mode than it currently is
METHODS OF HRI
TELEOPERATION
a human (i.e., teleoperator) can
mechanically manipulate items or sense
objects at a different location than
where they are currently located, using
a mechanical or robotic apparatus
Challenges: operators experience
issues related to cognitive
tunneling, decreased field of
vision, degraded sense of spatial
orientation, attention switching,
and motion sickness
SUPERVISORY CONTROL
an operator manages multiple robots by
interacting with them individually
Challenges: as the number of robots being
supervised increases, the operators’
workload increases, their situation
awareness decreases, their response times
increase, the number of tasks that can be
successfully completed within a designated
time interval decreases, and the number of
system failures and accidents increase

6.

TEAMWORK
AUTONOMOUS ROBOTS+HUMAN
Effective teamwork between humans and robots, like effective teamwork between
people, depends on a mutual understanding of what all the team members are capable
of doing, what they are responsible for doing, and what they are currently doing
Neither human nor robot can do the
task by themselves but can do so
jointly as a team
One is better than another in
dependence on time
TASKS
environment
Human does better
Machine does better
Both the human and the
robot are capable of
doing a task suitably
THE BEST VARIANT
A human has to work with a robot that:
• supports mixed-initiative interaction,
• keeping the human apprised of the overall
situation,
• soliciting additional information as needed,
• completing predefined tasks without human
intervention,
• dynamically negotiating for initiative when
appropriate
6

7.

THE LOOP – WHAT IS IT AND WHY WE WANT TO BE IN IT
“The loop of control” – the interactions between a human operator and automation while
conducting one or more tasks
A HUMAN “OUT OF THE LOOP”
when the human loses
awareness of the shared task
A HUMAN “IN THE LOOP”
so long as he is aware of what
the automation is doing, how the
task is progressing, and can
reasonably estimate what the next
step or outcome will be
REASONS
Bad HR communication;
a robot is modeled to
be “strong and silent”;
human factor
RESULTS
reduced performance;
reduced
situation
awareness;
increased
automation
bias;
undesired actions

8.

COMMUNICATION BETWEEN HUMANS AND ROBOTS
This increased independence can make it difficult for people working with these robots to stay
informed about the robot’s actions and the decision making processes that lead to those actions.
Thus, we need extensive interaction models for complete understanding between “teammates”
In future soldiers and robots will work
in mixed teams with shared capabilities
SHARED MENTAL
MODELS
ROBOT’S GESTURES
Animal-like or
human-like
gestures
AUDITORY CUES
Synthetic and
human voices,
tones, alarms
SENSOR-BASED
DECISION MAKING
HAPTIC FEEDBACKS
Vibrations and
controller
torque
A robot sees an environment
through sensors and applies
some logics on it:
NLP, gestures detection...
INPUT DEVICES
Keyboard,
mouse, touch
screen
Mutual predictability
using neuroengineering
technologies

9.

IMPORTANCE OF THE PAPER
FOR THE GROUP’S PROJECT
RoboNurse
• The RoboNurse can be exploited in dangerous environments, harmful to
the ordinary nurse (COVID-19 hospitals, battlefields, for example);
• Mixed-initiative
interaction is a promising approach that allows
providing qualified assistance to a larger number of hospitalized
people with a limited number of medical staff, but considering
doctor’s solutions regarding special situation;
• We’ll implement the protocol of mixed-initiative interaction in the
project.
9

10.

English Русский Правила