Parallalizing any non trivial ML learning algorithm is hard. I am evaluating a product called Alpine Data Labs and got to know some interesting thoughts on MapReducing a Stochaic Graident Descent algorithm.
Here is the paper I am referring to. Thanks to Alpine Data Labs for sharing the information.
http://www.research.rutgers.edu/~lihong/pub/Zinkevich11Parallelized.pdf. Basically it takes the average of the optimized values after running SGD on each individual nodes.
Here is the paper I am referring to. Thanks to Alpine Data Labs for sharing the information.
http://www.research.rutgers.edu/~lihong/pub/Zinkevich11Parallelized.pdf. Basically it takes the average of the optimized values after running SGD on each individual nodes.
No comments:
Post a Comment