mlpack IRC logs, 2018-07-06

Logs for the day 2018-07-06 (starts at 0:00 UTC) are shown below.

July 2018
Sun
Mon
Tue
Wed
Thu
Fri
Sat
1
2
3
4
5
6
7
8
9
10
11
12
13
--- Log opened Fri Jul 06 00:00:34 2018
01:49 -!- manish7294 [8ba7a6b4@gateway/web/freenode/ip.139.167.166.180] has joined #mlpack
01:51 < manish7294> rcurtin: Finally I am able to reproduce those eval results. Earlier while doing those simulations I had commented out "if ( eval < -1 ) { bp = l; break; }" part, so that we do not loose any eval which is less than -1. But this time I wasn't doing that and that's the reason results were not same.
02:13 < rcurtin> right, have you found any difference between the theory and what you implemented?
02:14 < rcurtin> what I am understanding from your comment on github is that the pruning should be working much better
02:14 < rcurtin> but it seems like the problem is in the implementation of the bounds, did I understand that right?
02:17 < manish7294> rcurtin: Probably that could be the reason but I still have to find what's going wrong in there.
02:18 -!- cjlcarvalho [~caio@177-177-182-238.user.veloxzone.com.br] has joined #mlpack
02:30 < manish7294> rcurtin: The error is while updating transformationOld matrix, It should be done in the last part once the for loops ends.
02:32 < manish7294> But still runtime is not good.
03:20 < rcurtin> are you caching the previous calculation correctly for each bound?
03:27 < rcurtin> er, sorry, "for each point"
03:27 -!- manish7294 [8ba7a6b4@gateway/web/freenode/ip.139.167.166.180] has quit [Ping timeout: 252 seconds]
03:45 < jenkins-mlpack> Yippee, build fixed!
03:45 < jenkins-mlpack> Project docker mlpack weekly build build #49: FIXED in 2 hr 58 min: http://masterblaster.mlpack.org/job/docker%20mlpack%20weekly%20build/49/
03:45 < jenkins-mlpack> * manish887kr: lmnn
03:45 < jenkins-mlpack> * manish887kr: Small Changes
03:45 < jenkins-mlpack> * manish887kr: Reviewed Changes
03:45 < jenkins-mlpack> * manish887kr: Small Changes
03:45 < jenkins-mlpack> * manish887kr: Added some tests - inspired by NCA
03:45 < jenkins-mlpack> * manish887kr: Small update
03:45 < jenkins-mlpack> * manish887kr: Shuffle Update
03:46 < jenkins-mlpack> * manish887kr: Update
03:46 < jenkins-mlpack> * manish887kr: Rectify Gradient
03:46 < jenkins-mlpack> * manish887kr: resolve failing tests
03:46 < jenkins-mlpack> * manish887kr: Optimized some parts
03:46 < jenkins-mlpack> * manish887kr: Low rank conversion
03:46 < jenkins-mlpack> * manish887kr: update tests
03:46 < jenkins-mlpack> * manish887kr: Remove warnings and added some more tests
03:46 < jenkins-mlpack> * manish887kr: small fix
03:46 < jenkins-mlpack> * manish887kr: Low rank fix
03:46 < jenkins-mlpack> * manish887kr: Added binding tests
03:46 < jenkins-mlpack> * manish887kr: small update
03:46 < jenkins-mlpack> * haritha1313: subview batch test
03:46 < jenkins-mlpack> * manish887kr: make constraint member of the LMNN function class
03:46 < jenkins-mlpack> * manish887kr: reviewed changes
03:46 < jenkins-mlpack> * manish887kr: update
03:46 < jenkins-mlpack> * manish887kr: resolve failing tests
03:46 < jenkins-mlpack> * manish887kr: Use PRINT_CALL() macro
03:46 < jenkins-mlpack> * haritha1313: submat index error debug
03:46 < jenkins-mlpack> * haritha1313: end parameter condition
03:46 < jenkins-mlpack> * manish887kr: allow user to pass a initial learning point
03:46 < jenkins-mlpack> * manish887kr: added some simple dimensionality tests for initial point
03:46 < jenkins-mlpack> * manish887kr: Reviewed changes
03:46 < jenkins-mlpack> * manish887kr: style fix
03:46 < jenkins-mlpack> * manish887kr: comment out low rank accuracy test
03:46 < jenkins-mlpack> * Ryan Curtin: Specify namespace to handle confused MSVC compiler.
03:46 < jenkins-mlpack> * Ryan Curtin: Update history.
03:46 < jenkins-mlpack> * akhandait45: make changes suggested in sampling PR
04:13 -!- manish7294 [8ba7972d@gateway/web/freenode/ip.139.167.151.45] has joined #mlpack
04:18 < manish7294> rcurtin: Found a good speedup in the implementation of bounds and now things are way faster than current LMNN.
04:29 -!- manish7294 [8ba7972d@gateway/web/freenode/ip.139.167.151.45] has quit [Ping timeout: 252 seconds]
04:51 < rcurtin> manish7294: (I am up late today!) if you want to go ahead and push it, I'll take a look tomorrow. glad to hear that you were able to get some acceleration :)
05:53 < Atharva> Is there some utility function in mlpack or arma that will allow me to save matrices as images?
09:12 < zoq> Atharva: For the mnist pgm or ppm should work: A.save("A.pgm", pgm_binary);
09:13 < Atharva> zoq: I will try that, thanks!
10:05 < jenkins-mlpack> Project docker mlpack nightly build build #371: UNSTABLE in 2 hr 51 min: http://masterblaster.mlpack.org/job/docker%20mlpack%20nightly%20build/371/
12:24 -!- travis-ci [~travis-ci@ec2-54-205-139-46.compute-1.amazonaws.com] has joined #mlpack
12:24 < travis-ci> manish7294/mlpack#56 (evalBounds - d2c8124 : Manish): The build was broken.
12:24 < travis-ci> Change view : https://github.com/manish7294/mlpack/compare/772a7b4afd13...d2c812414523
12:24 < travis-ci> Build details : https://travis-ci.com/manish7294/mlpack/builds/78287172
12:24 -!- travis-ci [~travis-ci@ec2-54-205-139-46.compute-1.amazonaws.com] has left #mlpack []
14:11 -!- caiojcarvalho [~caio@200-165-27-17.user.veloxzone.com.br] has joined #mlpack
14:13 -!- cjlcarvalho [~caio@177-177-182-238.user.veloxzone.com.br] has quit [Ping timeout: 240 seconds]
14:13 -!- cjlcarvalho [~caio@200-165-4-199.user.veloxzone.com.br] has joined #mlpack
14:15 -!- caiocarvalho [~caio@189-105-111-62.user.veloxzone.com.br] has joined #mlpack
14:15 -!- caiojcarvalho [~caio@200-165-27-17.user.veloxzone.com.br] has quit [Ping timeout: 268 seconds]
14:18 -!- cjlcarvalho [~caio@200-165-4-199.user.veloxzone.com.br] has quit [Ping timeout: 245 seconds]
14:20 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has joined #mlpack
14:35 -!- caiocarvalho [~caio@189-105-111-62.user.veloxzone.com.br] has quit [Ping timeout: 268 seconds]
14:48 < sumedhghaisas> zoq, ryan: Hey guys, do you use any library to plot training curves?
14:49 < sumedhghaisas> I mean in MLPack :)
14:50 < rcurtin> sumedhghaisas: if I need to plot something I always end up just saving as CSV then importing into Python to use matplotlib...
14:50 < rcurtin> I never bothered to learn gnuplot
14:50 < zoq> same here :)
14:52 < zoq> You could also pipe/process the output and plot the results afterwards as well.
14:52 < sumedhghaisas> rcurtin: ahh Thats what I had in mind. :)
14:53 < sumedhghaisas> Its not ideal though, for example VAE samples need to be stored before loading them in python... too much work :P
15:10 < rcurtin> agreed---you could write a binding for VAE to Python (basically just write a _main.cpp file, then CMake it with 'add_python_binding(<name>)')
15:11 < rcurtin> then you could do all the work in Python
15:11 < rcurtin> but the needs of what you're doing might be too complex for that
15:20 < sumedhghaisas> rcurtin: ahh no, Atharva and I were just checking the VAE training and seems to be working so YEAAAHHHH!
15:20 < sumedhghaisas> we were thinking how to show the training curves
15:20 < sumedhghaisas> and samples
15:23 < rcurtin> great to hear it is working! :)
15:23 < Atharva> Yess!
15:23 < rcurtin> I guess the best idea I have is just write the points to be plotted to a CSV then use python... but you are right, it is not ideal
15:46 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has joined #mlpack
16:46 < Atharva> zoq: The current evaluate functions don't let us calculate loss of the network given custom predictors and targets
16:47 < Atharva> What do you think about an overload of Evaluate which takes in predictors and targets and returns the loss with the given parameters
16:47 < Atharva> I think it's useful in cases where you are using mini batches to train and want to see the loss over some other input.
16:48 < Atharva> The Evaluate functions we have now take the predictors matrix from the last forward pass.
16:48 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
16:51 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
16:51 < zoq> Atharva: What about using Predict(arma::mat predictors, arma::mat& results)?
16:51 < zoq> Atharva: We would have to return the loss as well.
16:52 < Atharva> Yeah, it doesn't return the loss
16:53 < zoq> I guess, we could provide another Evaluate method or return the loss, your choice.
16:54 < Atharva> To return the loss, the predict function will also need to take the targets as a parameter
16:55 < zoq> Atharva: Ahh, you are right.
16:55 < Atharva> So, is a Evaluate(predictors, targets) a good idea?
16:55 < Atharva> an*
16:56 < zoq> Atharva: Yeah sounds good to me.
16:56 < Atharva> I will add it then
16:57 < zoq> okay, great
16:57 < zoq> The PR is going to be full of really neat features :)
16:58 < Atharva> Yeah, seems like it. I am really glad to be of use. :)
17:03 < Atharva> zoq: Whenever you are free, please give a look at #1366.
17:18 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
17:21 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
17:36 < Atharva> can someone give me an example of how to use the serialize function to save models?
17:39 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
17:41 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
17:42 < zoq> Atharva: http://www.mlpack.org/docs/mlpack-git/doxygen/anntutorial.html#model_saving_loading_anntut
17:43 < Atharva> Thanks!
17:46 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
17:47 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
17:50 -!- vivekp [~vivek@unaffiliated/vivekp] has quit [Read error: Connection reset by peer]
17:51 -!- vivekp [~vivek@unaffiliated/vivekp] has joined #mlpack
19:28 < rcurtin> ok, just an update on the build systems:
19:28 < rcurtin> I've talked with my manager at Symantec and asked if he's willing to "donate" them for mlpack use, since they're already set up and honestly everything other than masterblaster is too obsolete to be useful to the company
19:29 < rcurtin> he said he would think about it, and probably have an answer in two weeks; but until that time, I think the systems will remain up and untouched and we can continue to use them
19:29 < ShikharJ> rcurtin: Yay!
19:29 < rcurtin> nonetheless, I'm working now on the transition to using ci.mlpack.org (which is actually just mlpack.org) as the Jenkins master node
19:30 < rcurtin> and I should have that done (or done enough) in the next couple of hours
19:30 < rcurtin> Jenkins doesn't make export/import easy, so I'm just manually copying the jobs over... there may be some minor issues to work out
19:30 < rcurtin> in the worst case, where Symantec decides not to donate the systems, I have two of the old benchmarking systems we used when I was at GT that I'll set up at my house
19:31 < rcurtin> and then I can add a few other systems I have floating around. so we wouldn't have the kind of "firepower" we have now, but it would certainly be enough to get by
19:31 < rcurtin> it's also not clear what my new company might be interested in providing for infrastructure, but I'll have to scope that out once I'm on the inside---that will probably be a month or two
19:31 < rcurtin> (I don't start until early August... I am going to try and relax in July :))
19:32 < rcurtin> I don't have any prior probability distribution on whether Symantec will let us keep using them, so I guess we will just have to wait and see
19:36 < zoq> Good news for now, also I can easily provide access to two more machines, just let me know what I should install.
19:54 < rcurtin> great, let me get Jenkins set up and then if you want to add them, we can do it then, or if you want to wait to see what Symantec will do, that's fine too
20:12 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has quit []
20:13 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has joined #mlpack
20:16 < ShikharJ> zoq: I just tmux'd a build on the latest code to check if changing the move semantics doesn't break our design.
20:16 < zoq> ShikharJ: For the standard GAN?
20:25 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has quit []
20:26 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has joined #mlpack
20:32 < rcurtin> ok, now ci.mlpack.org/ is working
20:32 < rcurtin> I think the PR builders should be working but it's not fully tested yet
20:32 < rcurtin> I'll leave Jenkins running on masterblaster
20:35 < zoq> rcurtin: So the PR builds are running on mlpack.org?
20:58 < rcurtin> or any free node
20:58 < rcurtin> but managed by Jenkins on ci.mlpack.org
20:58 < rcurtin> or, at least, the next PR build should be
20:58 < rcurtin> I think I'd like to set up those PR builds to run in an unprivileged docker container
20:59 < rcurtin> but that should not be too hard
20:59 < zoq> yeah, good idea
20:59 < zoq> so, jenkins runs on mlpack.org and masterblaster is another build node?
21:00 < rcurtin> right, that is how I have it for now
21:00 < rcurtin> Jenkins is still running on masterblaster for now
21:01 < rcurtin> but I transitioned all the plugins and all non-obsolete jobs
21:04 < rcurtin> looks like at least the git commit webhook works
21:05 < rcurtin> do you want to pish a simple change to a PR to test that? I don't think I have any open at the moment actually
21:05 < zoq> sure, hold on
21:06 -!- ImQ009 [~ImQ009@unaffiliated/imq009] has quit [Quit: Leaving]
21:08 < rcurtin> ha, you always have a new optimizer ready to go :)
21:09 < rcurtin> hmm, no success. let me look into it a bit...
21:10 < zoq> okay, let me know if I should trigger another build
21:10 < zoq> We can also retrigger the webhoock in the github settings
21:11 < rcurtin> ah, I see the jenkins credentials are not working right
21:11 < rcurtin> I am on the train now, let me look into it once I get home
21:12 < zoq> okay
21:37 < jenkins-mlpack2> Project mlpack - git commit test build #3: STILL FAILING in 33 min: http://ci.mlpack.org/job/mlpack%20-%20git%20commit%20test/3/
21:37 < jenkins-mlpack2> Ryan Curtin: Update to ci.mlpack.org. Also hopefully trigger a build.
22:16 < rcurtin> so, I think that I have copied over the credentials for the mlpack-jenkins github user wrong... but I have forgotten the credentials too!
22:16 < rcurtin> so I am working with github to try and get access to that account again (since I'm actually not even sure what email was used to create it!)
22:16 < zoq> hm, let me check
22:17 < zoq> I created the account :)
22:17 < rcurtin> oh! hm
22:17 < rcurtin> well so clearly I have no idea what is going on :)
22:38 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has quit [Ping timeout: 256 seconds]
22:39 -!- jenkins-mlpack2 [~PircBotx@knife.lugatgt.org] has joined #mlpack
23:11 -!- witness_ [uid10044@gateway/web/irccloud.com/x-xkthjmtcqugjjfzc] has joined #mlpack
23:12 < jenkins-mlpack2> Yippee, build fixed!
23:12 < jenkins-mlpack2> Project mlpack - git commit test build #4: FIXED in 33 min: http://ci.mlpack.org/job/mlpack%20-%20git%20commit%20test/4/
--- Log closed Sat Jul 07 00:00:35 2018