archives –––––––– @btnaughton

Convolutional Neural Net Playing a 3D Game

Convnetjs is a lovely javascript-based neural net / deep learning library from @karpathy. Of course, it's slow since it's javascript and doesn't use the GPU, but the demos are terrific.

Convnetjs implements deep reinforcement learning, which has famously been used to teach computers to play video games just from looking at the pixels. Google's DeepMind, the folks responsible for AlphaGo, did the original work.

The deep reinforcement learning demo is a simple game where an agent wanders around a room, trying to pick up red balls and avoid green balls. The agent has nine antennae that register touches with walls and balls. The input to the neural net is simply the output of the nine antennae.

convnet *Convnetjs deep reinforcement demo: the agent's antennae sense two green balls*

I wanted to see what the net would do if given some "raw" pixels instead of a sanitized input. How magical are deep nets anyway? I made a three.js canvas where I render a representation of what the agent is seeing. Then I pull the pixel color values out of the canvas and pass them into the neural net, replacing the antennae's sensor values.

convnet *3D view, as rendered by three.js, and below, the pixel values that get fed into the neural net.*
convnet *3D view of the play area, from above (agent at the top right)*

The new pixel-based version works too, but it does take forever to learn anything — at least compared to the original. I optimistically expected it to just work without much tweaking. I probably need to understand the parameters a lot better to make it work better.

The three.js version of the demo is here (and the source here).


Comments


Boolean Biotech © Brian Naughton Powered by Pelican and Twitter Bootstrap. Icons by Font Awesome and Font Awesome More