[mlpack] Augmented RNNs @ GSoC'17

[Marcus Edel]

Hello Konstantin,

> My name is Konstantin Sidorov, and I am an undergraduate student in Astrakhan
> State University (Russia). I’m glad to know that mlpack was accepted in GSoC’17
> – as a side note, congratulations :)

thanks and welcome!

> I’m already fairly familiar with deep learning. For example, recently I
> implemented optimality tightening from “Learning to play in a day”
> (https://arxiv.org/abs/1611.01606) for the AgentNet (“Deep Reinforcement
> Learning library for humans”, https://github.com/yandexdataschool/AgentNet).

Sounds really interesting, the "Learning to Play in a Day" paper is on my
reading list, looks like I should move it up.

> Of course, on such an early stage I have no detailed plan what (and how) to do –
> only some ideas. In the beginning, for example, I’m planning to implement NTMs
> as described in arXiv paper and implement *reusable* benchmarking code (e.g.,
> copy, repeat copy, n-grams). I would like to discuss this project more
> thoroughly if possible. In addition, this is my first participation in GSoC. So,
> excuse me in advance if I’ve done something inappropriate.

Implementing the NTM task from the paper, so that they can be used for other
models as well is a great idea. In fact, you see a lot of other papers that at
least reuse the copy task. There are a bunch of other interesting tasks that
could be implemented like the MNIST pen stroke classification task recently
introduced by Edwin D. de Jong in his "Incremental Sequence Learning" paper. The
Stanford Natural Language Inference task proposed by Samuel R. Bowman et al. in
"A large annotated corpus for learning natural language inference" can be also
transformed into a long-term dependency task, that might be interesting.

Regarding the project itself, take a look at other models as well, depending on
the model you choose, I think there is some time left for another model. Also,
about the implementation, mlpack's architecture is kinda different to Theano's
graph construction and compilation work, but if you managed to work with Theano
you shouldn't have a problem.

If you like we can discuss any details over the mailing list and brainstorm some
ideas, discuss an initial class design, etc.

I hope this is helpful, let us know if you have any more questions.

Thanks,
Marcus

> On 28 Feb 2017, at 07:06, Сидоров Константин <kost2906 at yandex.ru> wrote:
> 
> Hello Marcus,
> My name is Konstantin Sidorov, and I am an undergraduate student in Astrakhan State University (Russia). I’m glad to know that mlpack was accepted in GSoC’17 – as a side note, congratulations :)
> I’m interested to work on project “Augmented Recurrent Neural Networks”. I’m already fairly familiar with deep learning. For example, recently I implemented optimality tightening from “Learning to play in a day” (https://arxiv.org/abs/1611.01606 <https://arxiv.org/abs/1611.01606>) for the AgentNet (“Deep Reinforcement Learning library for humans”, https://github.com/yandexdataschool/AgentNet <https://github.com/yandexdataschool/AgentNet>). Here is the merged pull request: https://github.com/yandexdataschool/AgentNet/pull/88 <https://github.com/yandexdataschool/AgentNet/pull/88>.
> As you see, I’m quite familiar with deep learning and Theano. Even though my main field of interest is RL, I would be very interested in doing something new – that is why I’ve chosen “Augmented RNNs”.
> Of course, on such an early stage I have no detailed plan what (and how) to do – only some ideas. In the beginning, for example, I’m planning to implement NTMs as described in arXiv paper and implement *reusable* benchmarking code (e.g., copy, repeat copy, n-grams).
> I would like to discuss this project more thoroughly if possible. In addition, this is my first participation in GSoC. So, excuse me in advance if I’ve done something inappropriate.
> ---
> Best Regards,
> Konstantin.
> _______________________________________________
> mlpack mailing list
> mlpack at lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20170228/fb39cd2c/attachment-0001.html>