RL setting In the reinforcement learning framework, we can model human behavior as an agent interacting with the world. Let’s formalize this mathematically.
Markov Decision Process (MDP) Formulation The human experience can be modeled as a Markov Decision Process (MDP) defined by the tuple:
$$\mathcal{M} = (\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R}, \gamma)$$
where:
$\mathcal{S}$ is the state space (all possible states of the world and internal states) $\mathcal{A}$ is the action space (all possible actions the human can take) $\mathcal{P}: \mathcal{S} \times \mathcal{A} \times \mathcal{S} \rightarrow [0,1]$ is the transition probability function $\mathcal{R}: \mathcal{S} \times \mathcal{A} \times \mathcal{S} \rightarrow \mathbb{R}$ is the reward function $\gamma \in [0,1]$ is the discount factor Environment is the world The environment encompasses everything external to the agent (brain). Mathematically, the environment dynamics are governed by:
...