3 States and a Plan: The AI of F.E.A.R.
Survey Says!
Survey Says!
FSM: 3 States
FSM: 3 States
FSM: 3 States
FSM: 2 States!
FSM: Transition Logic
Shogo, 1998
No One Lives Forever, 2000
F.E.A.R., 2005
Halo 2, 2004 Allowable behaviors for infantry, drivers, and passengers
FSM vs Planning
FSM vs Planning
Motivation
Here’s the Plan:
What is Planning?
STRIPS Planning
STRIPS Planning
STRIPS Planning
States
States: FSM
States: Planning
States: Planning
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
FPS Statistics
FPS Statistics
FPS Statistics
FPS Statistics
FPS Statistics
Here’s the Plan:
Design Philosophy
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #2
Planning Video #3
Planning Video #4
Planning Video #4
Benefits of Planning
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Benefits of Planning
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Benefits of Planning
Dynamic Problem Solving
Dynamic Problem Solving
Differences From STRIPS
Cost Per Action
Cost Per Action
Cost Per Action
Cost Per Action
Cost Per Action
Cost Per Action
Differences From STRIPS
No Add/Delete Lists
No Add/Delete Lists
No Add/Delete Lists
No Add/Delete Lists
No Add/Delete Lists
No Add/Delete Lists
No Add/Delete Lists
Differences From STRIPS
Procedural Preconditions
Procedural Preconditions
Differences From STRIPS
Procedural Effects
Procedural Effects
Squad Behaviors
Global Coordinator
Global Coordinator
Squad Behaviors
Squad Behaviors
Simple Squad Behaviors
Simple Squad Behaviors
Get-to-Cover
Get-to-Cover
Get-to-Cover
Get-to-Cover
Get-to-Cover
Get-to-Cover
Get-to-Cover
Get-to-Cover
Squad Behaviors
Complex Squad Behaviors
Complex Squad Behaviors
Complex Squad Behaviors
Complex Squad Behaviors
Squad Behavior Implementation
Squad Communication
Squad Communication
Squad Communication
Squad Communication
Squad Communication
Squad Communication
Here’s the Plan:
“… just like the marines in Half-Life 1”
“… just like the marines in Half-Life 1”
MIT Media Lab: Cognitive Machines Group
Dialogue Integration
Unified Planning for Actions and Speech
Unified Planning for Actions and Speech
Plan Recognition
Plan Recognition
Plan Recognition
Questions?
12.65M

gdc2006_orkin_jeff_fear

1. 3 States and a Plan: The AI of F.E.A.R.

Jeff Orkin
Monolith Productions/
MIT Media Lab

2.

3.

4. Survey Says!

5. Survey Says!

6.

7. FSM: 3 States

8. FSM: 3 States

Goto
Animate
Use
Smart
Object

9. FSM: 3 States

Goto
Animate
Animate

10. FSM: 2 States!

Goto
Animate

11. FSM: Transition Logic

void StateAttack::Update()
{
//...
if( Ammo == 0 )
{
pState = Reload(bCrouch);
return;
}
//...
}

12. Shogo, 1998

13. No One Lives Forever, 2000

14. F.E.A.R., 2005

15. Halo 2, 2004 Allowable behaviors for infantry, drivers, and passengers

16. FSM vs Planning

FSM
- How
Planning
- What

17. FSM vs Planning

FSM
- How
- Procedural
Planning
- What
- Declarative

18. Motivation

19. Here’s the Plan:

• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.

20. What is Planning?

• Planning is a formalized process of
searching for sequence of actions to
satisfy a goal.
• Process is called “Plan Formulation.”

21. STRIPS Planning

…in a nutshell

22. STRIPS Planning

STRIPS =
STanford Research Institute
Problem Solver

23. STRIPS Planning

• STRIPS Goal:
Desired state of the world to reach.
• STRIPS Actions:
– Preconditions
– Effects

24. States

25. States: FSM

Attack
Search

26. States: Planning

Represented as a logical sentence:
AtLocation(Home) ^ Wearing(Tie)
Represented as a vector:
(AtLocation, Wearing) = (Home, Tie)

27. States: Planning

Example: Lemonade Stand
( weather, #lemons, $$ )=
(
,
,
) or
(
,
,
)

28. STRIPS Planning Example

29. STRIPS Planning Example

30. STRIPS Planning Example

31. STRIPS Planning Example

32. STRIPS Planning Example

33. STRIPS Planning Example

State: (phone#, recipe, hungry?)
Goal: ( -- , -- , NO )
(
, -- ,YES)
( -- ,
,YES)
(
, -- ,NO )
( -- ,
,NO )

34. STRIPS Planning Example

State: (phone#, recipe, hungry?)
Goal: ( -- , -- , NO )
(
,
(
,
,NO )
(
,
,NO )
,YES)

35. STRIPS Planning Example

36. STRIPS Planning Example

State: (phone#, recipe, hungry?)
Action
Preconditions: (
, -- , -- )
Effects:
Delete List: Hungry( YES )
Add List:
Hungry( NO )

37. STRIPS Planning Example

State: (phone#, recipe, hungry?)
Action
Preconditions: (
Effects:
Hungry( NO )
, -- , -- )
Hungry( YES ) ^ Hungry( NO )

38. STRIPS Planning Example

Action: Buy( object )
Preconditions: ( -- )
Effects:
Delete List: -Add List:
Own( object )
Own(
) ^ Own(
) ^ Own(
)

39. STRIPS Planning Example

40. STRIPS Planning Example

41. STRIPS Planning Example

42. STRIPS Planning Example

43. STRIPS Planning Example

44. FPS Statistics

Average lifespan of FPS enemy AI:

45. FPS Statistics

Average lifespan of FPS enemy AI:
12.23 seconds

46. FPS Statistics

Average lifespan of FPS enemy AI:
12.23 seconds
Number of enemy AI who die in
FPS’s per day:

47. FPS Statistics

Average lifespan of FPS enemy AI:
12.23 seconds
Number of enemy AI who die in
FPS’s per day:
19,789,203

48. FPS Statistics

Sally Struthers says “Save the AI!”

49. Here’s the Plan:

• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.

50. Design Philosophy

Designer’s job is:
Create environments that allow AI to
showcase their behaviors.
Designer’s job is NOT:
Script behavior of individual AI.

51. Planning Video #1

52. Planning Video #1

53. Planning Video #1

54. Planning Video #1

55. Planning Video #1

56. Planning Video #1

57. Planning Video #2

58. Planning Video #3

59. Planning Video #4

60. Planning Video #4

Soldier
Assassin
Rat

61. Benefits of Planning

1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving

62. Decoupling Goals & Actions

Decoupling Goals & Actions

63. Decoupling Goals & Actions

Decoupling Goals & Actions
Patrol
Priority: 1.0
Sensors: PatrolNodes
t
t
t
AttackFromCover
Priority: 8.0
Sensors: Vision, CoverNodes
t
t
Search
Investigate
t
t
Priority: 6.0
Sensors: Vision, SearchNodes
Priority: 3.0
Sensors: HearDisturbance
t
t
t
t
t
t
t
t
t
t
t
t
t

64. Decoupling Goals & Actions

Decoupling Goals & Actions

65. Decoupling Goals & Actions

Decoupling Goals & Actions
+
=

66. Decoupling Goals & Actions

Decoupling Goals & Actions
Patrol
Priority: 1.0
Sensors: PatrolNodes
t
t
t
AttackFromCover
Priority: 8.0
Sensors: Vision, CoverNodes
t
t
Search
Investigate
t
t
Priority: 6.0
Sensors: Vision, SearchNodes
Priority: 3.0
Sensors: HearDisturbance
t
t
t
t
t
t
t
t
t
t
t
t
t

67. Decoupling Goals & Actions

Decoupling Goals & Actions

68. Decoupling Goals & Actions

Decoupling Goals & Actions

69. Decoupling Goals & Actions

Decoupling Goals & Actions
Work
Priority: 8.0
Death
t
t
Stunned
Priority: 100.0
Priority: 80.0
t
t
t
t
t
t
t
t
t
t
t
t
t

70. Decoupling Goals & Actions

Decoupling Goals & Actions
Work
Priority: 8.0
Death
Stunned
Priority: 100.0
Priority: 80.0
Working Memory
Goto
Sit
Down
Animate
Recoil
Stand
Up

71. Benefits of Planning

1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving

72. Layering Behaviors

73. Layering Behaviors

74. Layering Behaviors

75. Layering Behaviors

76. Layering Behaviors

77. Layering Behaviors

78. Layering Behaviors

79. Layering Behaviors

80. Layering Behaviors

81. Layering Behaviors

82. Layering Behaviors

83. Layering Behaviors

84. Layering Behaviors

85. Layering Behaviors

86. Layering Behaviors

87. Layering Behaviors

88. Layering Behaviors

Kill Enemy
Attack Ranged

89. Layering Behaviors

Kill Enemy
Attack Melee
Attack Ranged

90. Layering Behaviors

Dodge
Kill Enemy
Dodge Roll
Attack Melee
Attack Ranged
Dodge Shuffle

91. Layering Behaviors

Dodge
Kill Enemy
Cover
Dodge Roll
Dodge Covered
Attack Melee
Attack Ranged
Dodge Shuffle
Goto

92. Layering Behaviors

Dodge
Kill Enemy
Cover
Dodge Roll
Dodge Covered
Attack Melee
Attack Ranged
Blind Fire
Dodge Shuffle
Goto

93. Layering Behaviors

Dodge
Kill Enemy
Cover
Dodge Roll
Ambush
Dodge Covered
Attack Melee
Attack Ranged
Blind Fire
Dodge Shuffle
Goto

94. Layering Behaviors

Dodge
Kill Enemy
Cover
Dodge Roll
Ambush
Dodge Covered
Attack Melee
Attack Ranged
Blind Fire
Dodge Shuffle
Goto

95. Layering Behaviors

96. Benefits of Planning

1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving

97. Dynamic Problem Solving

98. Dynamic Problem Solving

99. Differences From STRIPS

1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects

100. Cost Per Action

101. Cost Per Action

2
8

102. Cost Per Action

2
8
A

103. Cost Per Action

A

104. Cost Per Action

A*
Navigation
Planning
Nodes:
NavMesh Polys
World States
Edges:
NavMesh Poly
Edges
Actions
Goal:
NavMesh Poly
World State

105. Cost Per Action

A

106. Differences From STRIPS

1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects

107. No Add/Delete Lists

State: (phone#, recipe, hungry?)
Action
Preconditions: (
, -- , -- )
Effects:
Delete List: Hungry( YES )
Add List:
Hungry( NO )

108. No Add/Delete Lists

State: [phone#, recipe, hungry?]
Action
Preconditions: [
Effects:
, -- , -- ]
[ -- , -- , NO ]

109. No Add/Delete Lists

Attack
Preconditions: [ -- , -- ,
]
Reload
Effects:
[ -- , -- ,
]

110. No Add/Delete Lists

[ -- , -- , -- ]
4-byte values
TargetDead
[bool]
WeaponLoaded
[bool]
OnVehicleType[enum]
AtNode
[HANDLE]
- or AtNode
[variable*]

111. No Add/Delete Lists

Goal Cover: [ -- , CoverNode63, -- ]
Action Goto
Effect:
[ -- , variable*, -- ]

112. No Add/Delete Lists

[ -- ,
-- ,
]

113. No Add/Delete Lists

[ -- ,
-- ,
]

114. Differences From STRIPS

1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects

115. Procedural Preconditions

class Action
{
// [ -- , -- , -- ]
WORLD_STATE m_Preconditions;
WORLD_STATE m_Effects;
bool CheckPreconditions();
};

116. Procedural Preconditions

117. Differences From STRIPS

1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects

118. Procedural Effects

class Action
{
// [ -- , -- , -- ]
WORLD_STATE m_Preconditions;
WORLD_STATE m_Effects;
bool CheckPreconditions();
void ActivateAction();
};

119. Procedural Effects

void ActionFlee::ActivateAction()
{
// ...
}
<x,y,z>
Goto
Animate

120. Squad Behaviors

121. Global Coordinator

122. Global Coordinator

Squad 1
Squad 2
Squad 3

123. Squad Behaviors

1. Simple
2. Complex

124. Squad Behaviors

1. Simple
– Suppression fire
– Goto positions
– Follow leader
2. Complex




Flanking
Coordinated strikes
Retreats
Reinforcements

125. Simple Squad Behaviors


Get-to-Cover
Advance-Cover
Orderly-Advance
Search

126. Simple Squad Behaviors

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.

127. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

128. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

129. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

130. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

131. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

132. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

133. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

134. Get-to-Cover

1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player

135. Squad Behaviors

1. Simple
– Suppression fire
– Goto positions
– Follow leader
2. Complex




Flanking
Coordinated strikes
Retreats
Reinforcements

136. Complex Squad Behaviors

• There are none!!

137. Complex Squad Behaviors

Player

138. Complex Squad Behaviors

Player

139. Complex Squad Behaviors

Player

140. Squad Behavior Implementation

• Key is separation between squad
layer and individual behaviors.
• Possibly formalize with
Hierarchical Task Network (HTN)
planning.

141. Squad Communication

“I need reinforcements!”

142. Squad Communication

Dialogue, instead of announcements:
– Announcement: “Ouch!”
– Dialogue:
AI1: “What’s your status?”
AI2: “I’m hit!” (or “I’m alright!”)

143. Squad Communication

Dialogue, instead of announcements:
– Announcement: “Where did he go?”
– Dialogue:
AI1: “Can you see anything?”
AI2: “No sir!” (or “He’s behind
the boxes!”)

144. Squad Communication

Dialogue also explains lack of action:
– Dialogue:
AI1: “He’s aiming at you!
Get out of there!”
AI2: “I’ve got nowhere to go!”

145. Squad Communication

“…Not only do they give each other
orders, but they actually DO what
they’re told!”

146. Squad Communication

“…Not only do they give each other
orders, but they actually DO what
they’re told!”
Reality:
• Squad behavior has bird’s eye view of the
situation.
• Find appropriate dialogue sequence after
the fact.

147. Here’s the Plan:

• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.

148. “… just like the marines in Half-Life 1”

149. “… just like the marines in Half-Life 1”

1998
2005

150. MIT Media Lab: Cognitive Machines Group

http://www.media.mit.edu/cogmac

151.

152. Dialogue Integration

“What’s your status?”

153. Unified Planning for Actions and Speech

“Look out! Grenade!”
Precondition:
Effect:

154. Unified Planning for Actions and Speech

OpenDoor -or- “Open the door!”
Precondition:
door is closed
Effect:
door is open

155. Plan Recognition

UnlockDoor
CreateKey
LightFire
GotoForge
GetGold

156. Plan Recognition

Plan as Context Free Grammar (CFG):
UnlockDoor : CreateKey
CreateKey : LightFire GotoForge
GotoForge : GetGold

157. Plan Recognition

PLAN RECOGNITION VIDEO

158. Questions?

Jeff Orkin
http://www.jorkin.com
Monolith is hiring!
•Engineers
•Artists
•Designers
http://www.lith.com/jobs
English     Русский Правила