Pranay Pasula, pasula@berkeley.edu
New Algorithm! Physical Intelligence’s FAST Path to Embodied AI
The following is an overview of Physical Intelligence and how their just announced algorithm FAST (Fast Adaptive Sensorimotor Transformations) integrates ideas from robotics, neuroscience, and machine learning. My aim is to explain key principles, illustrate how these ideas can be applied, and provide logical justification for claims made.
Table of Contents
1. Why Physical Intelligence Matters
2. How FAST Works: Core Principles
• 2.2 Morphological Computation
• 2.3 Hierarchical Control and Learning
• 2.4 Real-Time Adaptation (Within Practical Limits)
• 4.1 Manufacturing and Industrial Settings
• 4.3 Assistive and Medical Applications
5. Technical Foundations: Algorithms and Code
6. Episodic Future Thinking and Its Parallels
• 6.1 Relevance of Episodic Future Thinking
• 6.3 Cognitive Map Inspiration
7. Beyond the Robot: Key Milestones for Understanding Intelligence
• 7.1 Deeper Observation of Model Internals
• 7.3 Refining Our Understanding of Intelligence
8. Underlying Missions and Trends
9. Conclusion
10. References
1. Why Physical Intelligence Matters
Embodiment and Efficiency
Biological organisms rely on integrated physical structures—like muscles and tendons—to reduce cognitive loads when interacting with external forces (Pfeifer and Bongard, 2007). Robotics researchers have adopted a similar principle, creating hardware designs (e.g., compliant arms) that distribute shock absorption, reducing the complexity demanded from software-based controllers (Bongard, 2013; Miglino et al., 1995).
Robustness and Adaptation
Physical intelligence emphasizes relatively rapid adaptation to dynamic and uncertain settings (Sutton and Barto, 2018; Mnih et al., 2015). These adaptations rely on continuous streams of sensor data and incremental model updates, which are typically faster than completely manual reprogramming cycles—yet still limited by computational resources and the data available.
Influence from Neuroscience
Neurophysiological research shows that the cerebellum refines motor commands using error-driven learning (Wolpert et al., 1998). Similar mechanisms—predictive modeling, feedback control, and reward-driven updates—offer guiding structures for designing robot controllers to handle unexpected perturbations in real-world tasks (Kawato, 1999).
2. How FAST Works: Core Principles
2.1 Sensorimotor Loops
FAST builds on cyclic feedback mechanisms observed in animals (Bizzi et al., 1991). The controller estimates current and near-future states, compares them to real-world feedback, and refines its internal model incrementally. This approach echoes forward models in the cerebellum (Miall and Wolpert, 1996).
2.2 Morphological Computation
Rather than depending solely on complex control algorithms, FAST leverages hardware design to simplify control (Pfeifer, 2012). Flexible joints or soft material grippers, for example, can adapt to minor variations in object shape without requiring extensive algorithmic correction (Laschi et al., 2012).
2.3 Hierarchical Control and Learning
FAST employs a layered approach (Todorov, 2009; Silver et al., 2012):
1.Low-Level Layer: Handles rapid, reflex-like commands.
2.Intermediate Layer: Learns sensorimotor mappings (Levine et al., 2016).
3.High-Level Layer: Oversees planning, potentially using model predictive control or hierarchical reinforcement learning (Barto and Mahadevan, 2003).
2.4 Real-Time Adaptation (Within Practical Limits)
“Fast” here means faster than a full retraining cycle, not instantaneous (Deisenroth and Rasmussen, 2011). The pace is bounded by factors such as:
• Availability of computing resources (e.g., GPU, CPU power).
• The amount and quality of sensor data.
• Stable learning rates (Lillicrap et al., 2016).
3. Parallels to Neuroscience
Predictive Processing
Humans rely on ongoing predictions about the consequences of actions (Friston, 2019). FAST employs forward models that estimate outcomes of candidate actions, akin to a simplified form of predictive coding (Hassabis et al., 2007).
Reward-Based Learning
Studies of the basal ganglia show how dopamine encodes reward prediction errors (Schultz et al., 1997). FAST typically uses reinforcement signals in a similar way to adjust policies after each discrepancy between expected and realized outcomes (Schultz, 2016).
Hierarchical Integration
The central nervous system manages reflexes in the spinal cord, integrates local signals in the cerebellum, and handles strategic planning in the motor cortex (Kandel et al., 2013). FAST mirrors this arrangement by offloading routine tasks to lower layers, freeing higher layers for more abstract decisions.
4. Real-World Relevance
4.1 Manufacturing and Industrial Settings
Reduced Manual Intervention: Modern robotics with morphological components and sensorimotor loops require less frequent reprogramming (Tobin et al., 2017).
Robust to Variation: FAST-based systems can learn or update their policies when parts differ in shape or alignment, minimizing downtime (Kragic and Christensen, 2002).
4.2 Collaborative Robotics
Safer Human-Robot Interaction: Force feedback and soft actuation help reduce the risk of harmful collisions (De Santis et al., 2008).
Adaptive Assistance: Over iterative trials, a robot can learn user-specific motion profiles and refine its support for tasks like tool handover (Argall et al., 2009).
4.3 Assistive and Medical Applications
Precision in Surgery: Sensor-based feedback reduces surgical error when tracking tissue or tool movements (Taylor, 2006).
Rehabilitation: Adaptive exoskeletons refine motor support for recovering patients over repeated sessions (Marchal-Crespo and Reinkensmeyer, 2009; Dollar and Herr, 2008).
5. Technical Foundations: Algorithms, Code
This section presents a reference implementation in PyTorch, pairing a forward model with a replay buffer and real-time updates. The code is intentionally modular to simplify experimentation or integration with hierarchical reinforcement learning (RL) layers.
import torch
import torch.nn as nn
import torch.optim as optim
from collections import deque
import numpy as np
import random
class ForwardModel(nn.Module):
def __init__(self, state_dim, action_dim, hidden_dim=256):
super().__init__()
self.net = nn.Sequential(
nn.Linear(state_dim + action_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, state_dim) # Predict next state
)
def forward(self, state, action):
x = torch.cat([state, action], dim=-1)
return self.net(x)
class ReplayBuffer:
def __init__(self, max_size=100000):
self.buffer = deque(maxlen=max_size)
def push(self, s, a, s_next):
self.buffer.append((s, a, s_next))
def sample(self, batch_size=64):
batch = random.sample(self.buffer, batch_size)
s_list, a_list, sn_list = zip(*batch)
return (torch.tensor(np.array(s_list), dtype=torch.float32),
torch.tensor(np.array(a_list), dtype=torch.float32),
torch.tensor(np.array(sn_list), dtype=torch.float32))
class FASTController:
def __init__(self, state_dim, action_dim, hidden_dim=256, lr=1e-3):
self.model = ForwardModel(state_dim, action_dim, hidden_dim)
self.optim = optim.Adam(self.model.parameters(), lr=lr)
self.replay = ReplayBuffer()
self.batch_size = 64
def predict(self, state, action):
with torch.no_grad():
s = torch.FloatTensor(state).unsqueeze(0)
a = torch.FloatTensor(action).unsqueeze(0)
next_pred = self.model(s, a)
return next_pred.squeeze(0).numpy()
def store_transition(self, s, a, s_next):
self.replay.push(s, a, s_next)
def update_model(self):
if len(self.replay.buffer) < self.batch_size:
return
s_batch, a_batch, sn_batch=self.replay.sample(self.batch_size)
pred_sn = self.model(s_batch, a_batch)
loss = nn.MSELoss()(pred_sn, sn_batch)
self.optim.zero_grad()
loss.backward()
self.optim.step()
def select_action(self, current_state, candidate_actions, cost_fn):
best_act, best_cost = None, float('inf')
for act in candidate_actions:
pred_next = self.predict(current_state, act)
c = cost_fn(pred_next)
if c < best_cost:
best_cost = c
best_act = act
return best_act5.1 How it Works
- Replay: The buffer stores recent transitions.
- Model Update: Mini-batches from the replay buffer refine the forward model via MSE loss to predict next states.
- Action Selection: Out of a set of candidate actions, choose the one that yields the lowest predicted cost. More advanced methods could integrate short-horizon planning or sampling-based optimization.
6. Episodic Future Thinking and Its Parallels
6.1 Relevance of Episodic Future Thinking
Humans often simulate possible futures based on past experiences (Schacter et al., 2012). FAST-like systems do something parallel: they reference stored transitions, either via replay buffers or short-horizon rollouts, to virtually “test” candidate actions before committing them to the real environment.
6.2 Implementation Nuance
Episodic Caching: Some implementations store entire trajectories (episodes). From these, the system can retrieve relevant segments based on current context, analogous to memory retrieval in the human brain (Pritzel et al., 2017).
Goal Conditioning: If a high-level planner sets a target state, FAST can estimate cost by a user-defined function or domain-specific metric, leveraging the forward model to see how closely a predicted next state matches the goal.
6.3 Cognitive Map Inspiration
O’Keefe and Nadel (1978) introduced the idea of a cognitive map, suggesting that place cells in the hippocampus encode spatial layouts. In robotics, map-like representations similarly guide path planning or collision avoidance. A FAST-based forward model can unify local motion planning with these higher-level spatial maps, refining real-time decisions through predictive insights.
7. Beyond the Robot: Key Milestones for Understanding Intelligence
7.1 Deeper Observation of Model Internals
AI systems can be instrumented at high resolution to track changes in sensorimotor loops (Montavon et al., 2018). This detail is still elusive in non-invasive human studies, which lack the same granularity (Kriegeskorte and Douglas, 2018).
7.2 Scaling Up Studies
Simulation tools allow thousands of parallel trials, accelerating R&D (Tassa et al., 2018; OpenAI et al., 2021). This volume of data can rapidly test and refine design choices that may otherwise require months of manual experimentation.
7.3 Refining Our Understanding of Intelligence
Experiments in embodied AI reveal which properties—hierarchical learning, morphological computation, reward-driven updates—are essential for robust intelligence (Brooks, 1991; Schmidhuber, 2015). By mirroring some neurological structures, we gain clues about how intelligence emerges in both machines and biological systems.
8. Underlying Missions and Trends
Biology–Robotics Convergence
Ongoing work emulates naturally evolved strategies, such as efficient gaits or dexterous manipulation (Ijspeert, 2008; Kim et al., 2011).
Hardware–Software Co-Design
Researchers integrate custom actuators, soft robotics, and deep learning to push performance boundaries (Cutkosky et al., 2008; Majidi, 2014).
Ethical and Societal Dimensions
As machines become increasingly adaptive, questions around accountability in decision-making grow (Winfield and Jirotka, 2018). New safety standards and transparent design practices remain critical (Vogel et al., 2011).
9. Conclusion
Physical intelligence—exemplified by FAST—shows how robots can fuse advanced hardware design, continuous sensor feedback, and hierarchical learning structures to exhibit resilience in complex, changing environments. Although adaptation is subject to computational and data constraints, these systems improve upon rigid, manually configured robotic setups.
Bridging neuroscience and robotics helps us understand adaptive control, morphological computation, and error-driven learning. Each innovation in this domain also gives us a fresh angle on intelligence itself. By comparing these AI systems to biological cognition, we uncover key elements—such as hierarchical organization and predictive processing—that underpin robust, goal-directed behavior. These insights pave the way for next-generation machines capable of safer, more flexible collaboration with humans, ultimately broadening our knowledge of what intelligence can be in both artificial and natural systems.
10. References
1. Pfeifer, Rolf and Bongard, Josh (2007). How the Body Shapes the Way We Think. MIT Press.
2. Bongard, Josh (2013). Evolutionary robotics. Communications of the ACM, 56(8), 74-83.
3. Miglino, Orazio, Lund, Henrik, and Nolfi, Stefano (1995). Evolving mobile robots in simulated and real environments. Artificial Life, 2(4), 417-434.
4. Sutton, Richard S. and Barto, Andrew G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT Press.
5. Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Graves, Alex, Antonoglou, Ioannis, Wierstra, Daan, and Riedmiller, Martin (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533.
6. Wolpert, Daniel M., Miall, R. Chris, and Kawato, Mitsuo (1998). Internal models in the cerebellum. Philosophical Transactions of the Royal Society B, 353(1373), 537–547.
7. Kawato, Mitsuo (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.
8. Bizzi, Emilio, Mussa-Ivaldi, Ferdinando A., and Giszter, Simon (1991). Computations underlying the execution of movement: a biological perspective. Science, 253(5017), 287-291.
9. Miall, R. Chris and Wolpert, Daniel M. (1996). Forward models for physiological motor control. Neural Networks, 9(8), 1265-1279.
10. Pfeifer, Rolf (2012). Conceptual frameworks for intelligence: Co-evolution of body and brain. International Journal of Advanced Robotic Systems, 9(1), 1-8.
11. Laschi, Cecilia, Mazzolai, Barbara, and Cianchetti, Matteo (2012). Soft robotics: Technologies and systems pushing the boundaries of robot abilities. Science Robotics, 1(1), eaah3690.
12. Todorov, Emanuel (2009). Efficient computation of optimal actions. Proceedings of the National Academy of Sciences, 106(28), 11478-11483.
13. Silver, David, Sutton, Richard, and Müller, Martin (2012). Temporal-difference search in computer Go. Machine Learning, 87(2), 183–219.
14. Levine, Sergey, Finn, Chelsea, Darrell, Trevor, and Abbeel, Pieter (2016). End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17(39), 1–40.
15. Barto, Andrew G. and Mahadevan, Sridhar (2003). Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, 13(4), 341-379.
16. Deisenroth, Marc P. and Rasmussen, Carl E. (2011). PILCO: A model-based and data-efficient approach to policy search. Proceedings of the 28th ICML, 465-472.
17. Lillicrap, Timothy P., Hunt, Jonathan J., Pritzel, Alexander, Heess, Nicolas, Erez, Tom, Tassa, Yuval, Silver, David, and Wierstra, Daan (2016). Continuous control with deep reinforcement learning. ICLR.
18. Friston, Karl (2019). A free energy principle for a particular physics. Neural Computation, 31(6), 127–146.
19. Hassabis, Demis, Kumaran, Dharshan, Vann, Seralynne D., and Maguire, Eleanor A. (2007). Patients with hippocampal amnesia cannot imagine new experiences. Proceedings of the National Academy of Sciences, 104(5), 1726-1731.
20. Schultz, Wolfram, Dayan, Peter, and Montague, P. Read (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599.
21. Schultz, Wolfram (2016). Dopamine reward prediction error coding. Dialogues in Clinical Neuroscience, 18(1), 23-32.
22. Kandel, Eric R., Schwartz, James H., and Jessell, Thomas M. (2013). Principles of Neural Science (5th ed.). McGraw-Hill.
23. Tobin, Josh, Biewald, Lukas, Duan, Yan, et al. (2017). Domain randomization for transferring deep neural networks from simulation to the real world. IROS.
24. Kragic, Danica and Christensen, Henrik I. (2002). Survey on visual servoing for manipulation. Computational Vision and Active Perception Laboratory, 2(1), 1–11.
25. De Santis, Alessandro, Siciliano, Bruno, De Luca, Alessandro, and Bicchi, Antonio (2008). An atlas of physical human–robot interaction. Mechanism and Machine Theory, 43(3), 253-270.
26. Argall, Brenna D., Chernova, Sonia, Veloso, Manuela, and Browning, Brett (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469-483.
27. Taylor, Russell H. (2006). A perspective on medical robotics. Proceedings of the IEEE, 94(9), 1652–1664.
28. Marchal-Crespo, Laura and Reinkensmeyer, David J. (2009). Review of control strategies for robotic movement training after neurologic injury. Journal of NeuroEngineering and Rehabilitation, 6, 20.
29. Dollar, Aaron M. and Herr, Hugh (2008). Lower extremity exoskeletons and active orthoses: challenges and state-of-the-art. IEEE Transactions on Robotics, 24(1), 144-158.
30. Montavon, Grégoire, Samek, Wojciech, and Müller, Klaus-Robert (2018). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1–15.
31. Kriegeskorte, Nikolaus and Douglas, Pamela K. (2018). Cognitive computational neuroscience. Nature Neuroscience, 21(9), 1148–1160.
32. Tassa, Yuval, Doron, Yotam, Muldal, Alistair, Erez, Tom, Li, Yanwu, Casas, Diego de, Budden, David, Abdolhosseini, Robik, et al. (2018). DeepMind Control Suite. arXiv preprint arXiv:1801.00690.
33. OpenAI, Andrychowicz, Marcin, Baker, Bowen, Chociej, Maciek, et al. (2021). Advances in real-world robotic manipulation via reinforcement learning. arXiv preprint arXiv:2108.04837.
34. Brooks, Rodney A. (1991). Intelligence without representation. Artificial Intelligence, 47(1-3), 139-159.
35. Schmidhuber, Jürgen (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.
36. Ijspeert, Auke J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural Networks, 21(4), 642-653.
37. Kim, Sangbae, Clark, Jonathan E., and Cutkosky, Mark R. (2011). iSprawl: Design and tuning for high-speed autonomous open-loop running. The International Journal of Robotics Research, 25(9), 903-912.
38. Cutkosky, Mark, Kim, Sangbae, and Provancher, William (2008). Forces and tactile sensing for bio-inspired climbing robots. Mechanisms and Machine Science, 2(1), 225-242.
39. Majidi, Carmel (2014). Soft robotics: a perspective—current trends and prospects for the future. Soft Robotics, 1(1), 5-11.
40. Winfield, Alan F. T. and Jirotka, Marina (2018). Ethical governance is essential to building trust in robotics and AI systems. Philosophical Transactions of the Royal Society A, 376(2133), 20180085.
41. Vogel, Jacob, Argall, Brenna, and Srinivasa, Siddhartha (2011). Implicit human feedback in learning human–robot interaction. IROS, 1938-1945.
42. Schacter, Daniel L., Addis, Donna R., and Buckner, Randy L. (2012). Remembering the past to imagine the future: the prospective brain. Nature Reviews Neuroscience, 8(9), 657-661.
43. Pritzel, Alexander, Uria, Benigno, Srinivasan, Siddhant M., Blundell, Charles, Lillicrap, Timothy, and Wierstra, Daan (2017). Neural episodic control. ICML, 2827-2836.
44. O’Keefe, John and Nadel, Lynn (1978). The Hippocampus as a Cognitive Map. Oxford University Press.