MAPS- A Metacognitive Architecture for Improved Perceptual and Social Learning: from simple tasks to multi-agent reinforcement learning

Reinforcement Learning (RL) has made significant strides but struggles with social and continuous learning. Cognitive neuroscience highlights metacognition as key to human self-monitoring, knowledge retention, and adaptive behavior, yet its potential in AI remains underexplored. Metacognition could mitigate RL’s catastrophic forgetting and enhance social intelligence, but current implementations focus on basic perceptual tasks, overlooking broader applications. This study introduces the Metacognitive Architecture for Perceptual and Social Learning (MAPS), integrating a second-order (metacognitive) network into AI systems (AIS) to improve both social and continuous learning. We evaluate MAPS across four conditions: perceptual learning (Know Thyself), SARL (MinAtar), SARL with continuous learning (SARL+CL, MinAtar), and MARL (MeltingPot 2.0). To assess social learning, we compare a 2nd-order confidence network in perceptual vs. social tasks, analyzing its impact on decision-making and interaction dynamics. For continuous learning, a 2nd-order teacher network stabilizes new knowledge integration, preventing past knowledge loss. Results show that metacognitive mechanisms significantly enhance adaptability in AIS. In perceptual tasks, the cascade model improves structured learning and information flow. In SARL, combining a 2nd-order network with a cascade model enables complex behavior adaptation. In SARL+CL, it prevents catastrophic forgetting more effectively than DQN. In MARL, MAPS shows promise in high-variability environments, though further testing is needed. These findings suggest metacognition as a powerful tool for enhancing AI’s learning efficiency and social competence.

Submitted to RLC 2025