mlpack  blog
Deep Learning Module in mlpack - Week 2

Deep Learning Module in mlpack - Week 2

Kris Singh, 22 June 2017

This week we pushed our existing implementations to mlpack. We are now done with Basic Wrapper Layer for RBM the base visible and hidden layer. We also are done with the Cd-k and and PCd-k algorithm. This week will be spent in writing test for the Binary RBM layer. I will also try to complete the ssRBM this would be easy since we have to only edit the visible layer for this hopefully. Currently we are facing some diffculties in the storage of the parameters that are shared by the visible and hidden layer. But we expect to finish that by this week.

Here are the links to the works

  1. CD-k algorithm
  2. Wrapper Layer and the Binary Visible Layer

Deep Learning Module in mlpack - Week 2

Week Three

This week I tried to finish the PR of the BinaryRBM implmentation. I expected t is would not take much time. But as the famous saying goes "We make plans and god laugh". Most of the time this week was spent in debuggin the existing implmentation of the RBM implementation.This code majority of my time this week though the code alse went through some major changes. Some of the major Changes it underwent were as follows:

  1. Change of the evaluation function
  2. Major Style Fixes
  3. Cd-k code addition.
  4. Addition of batch training to cd-k algorithm

The important thing I learnt this week how important is to intialise the variable.

We finally were able to solve the problem of training and we kind of get okay results now have a look here. Here is parmaeters list we got results by. cd-1, batch size: 20, learning rate:0.1

The samples are generated from 1-step gibbs sampling.

The last image uses mnist-binary dataset with threshold value of 0.2

Next Week

I had hoped to finish Binary RBM in the previous week but now it has to be extended this week. Major goals for this week include

  1. Writing test for Binary RBM(write now i am planning to add reconstruction loss and classification accuray as test)
  2. Merge Binary RBM PR
  3. Start with ssRBM

*Hopefull this week we would be able to achive are targets. :) AUTHORSTART Kris Singh AUTHORSTOP DATESTART 22 June 2017 DATESTOP PAGESTART KrisSinghPage PAGESTOP