Skip to content

An attempt at replicating Deepmind's Neural Turing Machines in Theano

Notifications You must be signed in to change notification settings

chiggum/Neural-Turing-Machines

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural-Turing-Machines

An attempt at replicating Deepmind's Neural Turing Machines in Theano as a part of my bachelor's thesis.

Advisor: Prof. Amit Sethi

Here is the link to the paper: http://arxiv.org/abs/1410.5401

Results

  • Following are the results on COPY task of the NTM I implemented from scratch.
  • Training is done on sequences of length varying from 1 to 20 and width 8 (bits per element in a sequence).
  • With version 1 of NTM which has a single read-write head.

Alt ntm-v1-on-test-seq-of-len-66

  • Learning Curve corresponding to version 1 of NTM (Sorry I din't compute the sliding window average in version 1).

Alt ntm-v1-learning-curve

  • With version 2 of NTM which has separate heads for reading and writing.

Alt ntm-v2-on-test-seq-of-len-34

  • Learning Curve corresponding to version 2 of NTM.

Alt ntm-v2-learning-curve

Usage

For training: In ntm_v*.py set

to_test = False

To run your trained model or a pre-trained model from model directory,

In ntm_v*.py set

to_test = True
test_path = path_to_npz_model_parameters_file

Thesis Report

Please visit this link for my bachelor's thesis report.

Presentation

Please visit this link for a presentation with comments, of my thesis.

Reading material

Check out the reading material directory of this project on github for some relevant papers related to RAM based models.

Other NTM Implementations

Future works

  • Making NTM to work on other tasks described in the paper.
  • Using NTM to make agents for playing games (Deep reinforcement learning).

About

An attempt at replicating Deepmind's Neural Turing Machines in Theano

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published