[mlpack] GSoC - 2017

Tue Feb 14 10:02:25 EST 2017

Hello Prasanna,

> I was thinking of introducing couple of new module in mlpack myself. I have
> PReLU activation function and maxout layer in my mind. So should I proceed with
> this?

Sounds great especially maxout is interesting, let us know if you need any help
with the implementation.

> Yup I checked them out. Currently I am reading the "Back to the Future: Radial
> Basis Function Networks Revisited" paper. I am planning to go through all of
> them in coming month. Also one of the papers in RBF sections is paid so just
> wanted to check if you have some other source for learning methods of RBF
> networks.

Sorry, I didn't realize that one of the papers isn't accessible for free, here
are two more papers that are interesting:

"An improved radial basis function neural network for object image retrieval” by
G. A. Montazer et al. - https://pdfs.semanticscholar.org/2f62/ff139763c697674e447825bc214653b97192.pdf <https://pdfs.semanticscholar.org/2f62/ff139763c697674e447825bc214653b97192.pdf>

"A Comparison of RBF Neural Network Training Algorithms for Inertial Sensor Based Terrain Classification” by
T. Kurban et al. - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3312446/ <https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3312446/>

> So basically it is up to participant to decide which model he will be
> implementing during project? If that's the case then how much models you guys
> are expecting to be implemented during 3 months of project (implementing models
> + documenting them)? I don't have first hand experience with implementing models
> in actual open source library so I can't estimate days needed to implement a
> well tested model.

That highly depends on the models you like to implement e.g. implementing
Bidirectional Recurrent networks is straightforward since there is already an
RNN class that can be used to construct the model or if you like to implement
RBMs the step to DBN's is also fairly straightforward since you can stack RBM's
to build you DBN. On the other side the implementation of the main idea might be
straightforward but for each model, there are a bunch of interesting features
that can be explored like different training methods so that you could easily
work entirely on a single model.

I would recommend that when you write your proposal; define some main goals that
you think are realistic to achieve and probably add some other features that you
like to work on if there is time left.

> Also is reading research papers part of the project or something to be done
> before project period starts? This will help me to get my expected timeline.

I would recommend getting familiar with the project before the actual coding
starts that is especially helpful when you write your proposal. Besides that,
there is the community bonding time that will be used to get to know each other
and to work out some plan and project details.

> That's great. Familiarising myself with mlpack codebase is certainly hard
> without working with it and getting hand dirty. I am looking forward to issues
> related with ann. Until then I will try to understand mlpack coding style and
> read research papers referenced on project ideas page. I apologise if any of the
> questions I asked above are inappropriate.

Sounds good, let us know if you have any question.

Thanks,
Marcus

> On 10 Feb 2017, at 14:17, Marcus Edel <marcus.edel at fu-berlin.de> wrote:
> 
> Hello Prasanna,
> 
> Thanks for getting in touch.
> 
>> I have intermediate knowledge in deep learning field. I have also read Vivek's discussion on this topic and it gave me more information on the project idea. I have studied RBM, DBN and Hopfield network from book "Machine learning: an algorithmic perspective, Stephen Marsland". The book gave me idea of these networks and basic training algorithm for these models. I have implemented basic training algorithm but I am unaware of latest research going on in these models?
> 
> I've linked some recent papers on the ideas pages, that revisit some of the
> traditional models like "Spike and slab restricted boltzmann machine" by
> Courville et al or "Back to the Future: Radial Basis Function Networks
> Revisited" by Que and Belkin paper. There are a couple more references for the
> mentioned models, especially for GAN there are some really interesting papers
> that are definitely worth a look.
> 
>> The project page says that this project will implement RBM, DBN, BRNN and GAN networks. Are there any other modules expected in this project? Also is there anything you can help me get started with this project? (any issues / bugs related to this part of code)
> 
> 
> As stated in the project description, all models mentioned are just examples
> that could be implemented, we don't expect you to implement all models or any of
> the models. Meaning if you like to implement another network model that mlpack
> currently doesn't have, we are open to discuss the ideas with you.
> 
>> As of now I am reading DeepLearningBook to dive into latest research going on in deep learning fields. It has variety of models / layers (for e.g. batch norm layer, GRU cell) which are not part of mlpack as of now. So I just wanted to know which methods are exactly expected from deep learning project? Also can I come up with my own project that implements these other methods which are not part of current project idea ?
> 
> 
> If you like to implement machine learning algorithms that mlpack currently
> doesn't have, contributions like that are more than welcome. If you can provide
> a fast, well-tested implementation it would be great to include it in mlpack.
> Also, we are open for own project ideas, however, we have to make sure it's
> enough work and someone is able to mentor the project. The GRU layer and batch
> norm is definitely an interesting idea that when extended by some other methods
> could build up an interesting GSoC project.
> 
>> It would be much helpful if you can point me to some direction to get started with this project. Also it would be great if I can get my hands on issue related to ann module of mlpack.
> 
> I'll see if I can open some ann related issues, that could be a great start to
> dive into the codebase.
> 
> Thanks,
> Marcus
> 
> 
>> On 10 Feb 2017, at 01:52, Prasanna Patil <prasannapatil08 at gmail.com <mailto:prasannapatil08 at gmail.com>> wrote:
>> 
>> Hello everyone,
>> 
>> I am new to mlpack. I am 3rd year computer engineering undergraduate student from India. I am going to participate in GSoC this year and I would like to work with mlpack on "Essential Deep Learning Modules" project.
>> 
>> 
>> I have compiled mlpack locally and everything is set up. I have been exploring mlpack code base and I have created some basic ml programs using mlpack.
>> 
>> 
>> I have completed coursera machine learning course and udacity deep learning course. I have implemented a few machine learning algorithm such as neural net, decision tree, hopfield network, self organizing map, k means etc. myself. However they are implemented in Python. You can check them here (https://github.com/prasanna08/machinelearning <https://github.com/prasanna08/machinelearning>).
>> 
>> 
>> I have intermediate knowledge in deep learning field. I have also read Vivek's discussion on this topic and it gave me more information on the project idea. I have studied RBM, DBN and Hopfield network from book "Machine learning: an algorithmic perspective, Stephen Marsland". The book gave me idea of these networks and basic training algorithm for these models. I have implemented basic training algorithm but I am unaware of latest research going on in these models?
>> 
>> 
>> The project page says that this project will implement RBM, DBN, BRNN and GAN networks. Are there any other modules expected in this project? Also is there anything you can help me get started with this project? (any issues / bugs related to this part of code)
>> 
>> 
>> As of now I am reading DeepLearningBook to dive into latest research going on in deep learning fields. It has variety of models / layers (for e.g. batch norm layer, GRU cell) which are not part of mlpack as of now. So I just wanted to know which methods are exactly expected from deep learning project? Also can I come up with my own project that implements these other methods which are not part of current project idea ?
>> 
>> It would be much helpful if you can point me to some direction to get started with this project. Also it would be great if I can get my hands on issue related to ann module of mlpack.
>> 
>> Thanks,
>> Prasanna
>> _______________________________________________
>> mlpack mailing list
>> mlpack at lists.mlpack.org <mailto:mlpack at lists.mlpack.org>
>> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20170214/c5d5cedd/attachment-0001.html>