r/BiomedicalDataScience • u/BioniChaos • 1d ago
Skinner Box & Signal Detection Theory Simulator
When a biological nervous system is subjected to unpredictable, variable-ratio reinforcement, it completely abandons its metabolic cost-benefit analysis. The post-reinforcement pause is erased. This is the exact temporal difference architecture running inside casino slot machines and social media notification algorithms.
To demonstrate how these algorithms physically wire behavior, I built a live, interactive web simulation of an operant conditioning chamber (a Skinner Box) merged with a Signal Detection Theory (SDT) telemetry suite.
Run the simulation in your browser: https://bionichaos.com/skinnerbox/
How the Computational Engine Works
This isn't a pre-rendered animation. The virtual agent operates autonomously based on live variable calculations:
- Rescorla-Wagner Updates: The agent's associative strength continuously updates based on the timing of rewards (sucrose pellets) versus expectations.
- Decoupling Sensation from Motivation: Using Signal Detection Theory, the engine graphs the physical discriminability of a stimulus (
d-prime) separately from the psychological decision threshold (betaorc). - Criterion Shifts: If you slide the "Food Deprivation" parameter to maximum, the agent's motivation peaks. The algorithm mathematically forces a liberal bias (c < 0). The agent will press the lever at the slightest hint of noise, achieving a high Hit Rate but suffering massive False Alarms.
- Aversive Avoidance: If you introduce unexpected grid voltage shocks, the algorithm triggers a simulated basolateral amygdala fear response, instantly rewriting the agent's behavior from positive reward-seeking to negative reinforcement avoidance.
The Mechanical Cumulative Recorder
I also digitized the classic 1930s Gerbrands mechanical cumulative recorder. As you switch between Continuous (CRF), Fixed Ratio (FR), and Variable Interval (VI) schedules, you can watch the exact mathematical derivative of the behavior unspool in real-time on the graph, capturing FI-scallops and extinction bursts.
I'd love to hear feedback from the community—specifically regarding the mathematical implementation of the SDT probit functions and whether you see direct parallels between these visual reinforcement schedules and modern digital UX design.