Neural Cellular Automata
Every cell runs the same two-layer network on its 3×3 neighbourhood. That’s the whole model, and it can grow a picture from one cell and regrow it when you cut it. I spent most of my time on the 3D version, trying to get it to learn gastrulation from a real embryo recording.
Live model
This is the growing model from Mordvintsev et al. (2020), running on your GPU through WebGPU. Nothing has coordinates and nothing looks at the whole grid. The picture grows out of the centre cell.
- Grid.
- 48×48 cells, 16 numbers each. Four are the colour, the other twelve are whatever the cells decide to use them for.
- Rule.
- Each cell sees its 3×3 neighbourhood through three fixed filters, runs that through a two-layer network with 128 hidden units, and adds the result to its own state. Cells update in a random order.
- Damage.
- Click the picture once it’s grown. There’s no repair code, the rule just keeps running and the shape comes back.
Gastrulation
Gastrulation is the part of embryo development where a ball of cells folds itself into layers and the body plan appears. Nothing coordinates it, each cell reacts to its neighbours, which is the setup an NCA already has. So the project was whether the same kind of rule could learn it from real data.
The data is a light-sheet recording of a mouse embryo from McDole et al. (2018), with every cell tracked through time. We turned the tracked x, y, z positions into voxel density targets and trained the NCA on transitions between timepoints rather than one final shape. Here is what I built on it between March and May 2025.
- The data. A reduced version of the cell database so training didn’t need the full recording, and voxel volumes that can be x×y×z instead of cubes.
- Training. Rewrote the training loop around batches that pair a volume with a later one: start to target, random earlier to later, and previous to next, with a variable number of update steps per batch.
- Loss. A differentiable IoU loss alongside the MSE, and the MSE fixed to score only the RGB channels.
- Logging. Persistent loss tracking and plots so we could tell whether a run was learning, and float types that let it train on a CPU.
The rest of it
NCA is a research team under Monash DeepNeuron. I joined in February 2025 as a deep learning engineer.
- Models. The growing model (Mordvintsev et al., 2020) and the texture model (2021) in PyTorch. Growing trains on squared error against a target image. Texture trains against VGG-16 features, so it learns what a material looks like rather than copying one picture of it.
- Simulator. Everything on neuralca.org runs in the browser on the GPU, with the models rewritten as compute shaders. My part was the cost: smaller kernels, fewer passes, and a cost function that doesn’t need a training loop to evaluate, so it runs on a laptop that isn’t a gaming machine.
- The ladder. The simulator also has Life, Life-like rules, Larger than Life and continuous automata, so you can see the idea build up one step at a time before the neural version.
The team
- Afraz Gul · Project lead
- Chloe Koe · Deep learning and graphics
- Nathan Culshaw · Deep learning
- Angus Bosmans · High performance computing
- Luca Lowndes · Deep learning, gastrulation
- Alexander Mai · Web and UI
- Keren Collins, Joshua Riantoputra, Nyan Knaw · Advisors
Stack
PyTorch, 3D convolutions, WebGPU, TypeScript, React, Python