Machine_learning | Democratizing Data

The first conference of Operational Machine Learning: OpML ‘19

Tue, 04 Jun 2019 13:50:07 +0900

I attended OpML ’19 is a conference for “Operational Machine Learning” held at Santa Clara on May 20th.

OpML ‘19
_The 2019 USENIX Conference on Operational Machine Learning (OpML ‘19) will take place on Monday, May 20, 2019, at the…_www.usenix.org

The scope of this conference is varied and seems not to be specified yet, even if I attended it. I’ll borrow the description from the OpML website.

The 2019 USENIX Conference on Operational Machine Learning (OpML ’19) provides a forum for both researchers and industry practitioners to develop and bring impactful research advances and cutting edge solutions to the pervasive challenges of ML production lifecycle management. ML production lifecycle is a necessity for wide-scale adoption and deployment of machine learning and deep learning across industries and for businesses to benefit from the core ML algorithms and research advances.

Overview of the conference

The number of attendees was 210, they came from LinkedIn, Microsoft, Google, Airbnb, Facebook, etc.
The target of “Operational Machine Learning” is diverse. I thought it focuses on MLOps things such as reproducibility, ML DSL for productionization, visualization, stakeholder management, but there are many talks about ML for system, system utilization optimization, SRE for ML, hardware accelerator, etc.
There is a contrast between tech giants, e.g. Google, Uber, Facebook, Airbnb, Microsoft, and LinkedIn, and other followers. While ML lead companies are talking about their OSSs or ML infrastructures, following companies tend to talk about their specific use case or their solutions (those speakers seems to be small ML ventures).

Some interesting talks

Keynote: Ray: A Distributed Framework for Emerging AI Applications

https://www.usenix.org/conference/opml19/presentation/jordan
Current target of Machine Learning is pattern recognition, but Jordan said decision-making will be the future of ML/AI
Creating a “recommendation market” is the key

MLOp Lifecycle Scheme for Vision-based Inspection Process in Manufacturing

https://www.usenix.org/conference/opml19/presentation/lim
https://www.usenix.org/sites/default/files/conference/protected-files/opml19_slides_lim.pdf
A challenge for defeat recognition by an image in edge applied for Samsung smartphone.
They need to inference for 3000 GB images/day.
The team structure which involves product inspectors and product managers is interesting

From [https://www.usenix.org/sites/default/files/conference/protected-files/opml19\_slides\_lim.pdf](https://www.usenix.org/sites/default/files/conference/protected-files/opml19_slides_lim.pdf)

From https://www.usenix.org/sites/default/files/conference/protected-files/opml19_slides_lim.pdf

AIOps: Challenges and Experiences in Azure

https://www.usenix.org/conference/opml19/presentation/li-ze
Anomaly detection and diagnosis with lambda architecture for Azure
Disk failure prediction for Azure which introduces proactively live to migrate the workloads to a healthy disk

How the Experts Do It: Production ML at Scale

A panel discussion for ML infrastructures

Lead and moderator: Joel Young, LinkedIn

Panelists:

Sandhya Ramu, Director, AI SRE, LinkedIn
Andrew Hoh, Product Manager, ML Infra and Applied ML, AirBNB
Aditya Kalro, Engineering Manager, AI Infra Services and Platform, Facebook
Faisal Siddiqi, Engineering Manager, Personalization Infrastructure, Netflix
Pranav Khaitan, Engineering Manager, Personalization and Dialog ML Infra, Google

The important thing to keep top level is

the lead time from experiment to production
Flows build for production with involving different team
Not everything is the highest priority. Metrics, dashboards are important

Cost of run/train vs Agility

It’s hard to find down streaming use cases. (Airbnb)
Monitor model resource usage
Keep ML infrastructure extremely flexible
Hard to force using a single framework

What are the important things for your ML platform?

Facebook

Reliability
Scalability
Developer productivity

Agility (Available libraries etc)
Enabling the latest technology
Cost and impact of Machine Learning

Netflix

How quickly/many A/B test we can do
How rapid new researcher can do?

Airbnb

Business impact
# of users for the infrastructures
How many inferences/scoring is done?
Availability, scalability, cost, and long-term decision making

Google

Innovation aspect
How can the ML infrastructure system will empower the next 5 yrs products?

Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform

https://www.usenix.org/conference/opml19/presentation/baylor
TFX provides a library for recording and retrieving metadata for ML: ML Metadata https://www.tensorflow.org/tfx/guide/mlmd

From [https://www.usenix.org/system/files/opml19papers-baylor.pdf](https://www.usenix.org/system/files/opml19papers-baylor.pdf)

From https://www.usenix.org/system/files/opml19papers-baylor.pdf

Disdat: Bundle Data Management for Machine Learning Pipelines

https://www.usenix.org/conference/opml19/presentation/yocum
https://github.com/kyocum/disdat
Talk about OSS for ML pipeline and data versioning.

Predictive Caching@Scale

https://www.usenix.org/conference/opml19/presentation/janardhan
Traffic prediction for CDN (Akamai)
Interesting cache strategy with covering prediction error

Ruby for Data Science and Machine Learning

Wed, 24 Apr 2019 12:10:28 +0900

I attended RubyKaigi 2019 held at Fukuoka from Apr 18 to Apr 21. This year’s RubyKaigi was a really great opportunity for me to know the possibility of Data Science and Machine Learning for Ruby.

Data Science and Ruby

As many of you may know, Ruby is widely known for web application with such as Ruby on Rails, but there is another momentum of Ruby or non-Python language. Here is the list of the sessions about Data Science.

Ruby for NLP
A Deep Learning Adventure \[[repo](https://github.com/nusco/deep_learning_adventure)\] (talked by Paolo Perrotta, the author of Metaprogramming Ruby!)
Ruby Data Workshop
Reducing ActiveRecord memory consumption using Apache Arrow
Red Chainer and Cumo: Practical Deep Learning in Ruby
Make Ruby Differentiable

Center of data science with Ruby

There is three core software supporting these movements:

Apache Arrow
Numo/Cumo
Red Chainer (Deep Learning framework ported from Chainer, implemented in Python)

Apache Arrow is a cross-language data structure for in-memory data. Kohei Sutou, the creator of Red Arrow, Ruby binding of Apache Arrow, who is a Japanese PMC of Apache Arrow. He has also been organizing an initiative called Red Data tools, monthly developer meet-ups for Ruby data tools. The meetup drives Ruby data ecosystem, especially for beginners. I heard from mrkn, a Ruby committer, that Arrow is trying to implement data manipulations those pandas does as C++ code. That means, calculations of tabula style data, a.k.a. DataFrame can be done in Apache Arrow’s Table format so that Ruby would be able to be suitable for data manipulation.

Another essential thing is Numo, which enables to handle numeric array like Numpy and is the fundamental part of DS/ML execution. Cumo is the GPU version of Numo and 75 times faster than Numo for the hello world for Deep Learning, a.k.a. MNIST. The talk about Cumo suggested that many Deep Learning related executions depend on CUDA so that scripting languages can be just a wrapper of them.

Red Chainer enables Deep Learning tasks, but it seems still young. Rather than that, Menoh-Ruby can be a great tool, which allows to inference/predict with pre-trained models with PyTorch, Chainer, or any other frameworks which can export ONNX, the intermediate format of DL.

So, how will be the Ruby data science going on?

Looking at those momenta of Apache Arrow and Cumo, I feel the data science on Ruby would become much easier since the core problems which are related to execution speed can be hidden into C++/GPU layer. And using Menoh-Ruby can be a good opportunity for Ruby on Rails applications to serve prediction results on Ruby!

Red Data tools also create opportunities for many software engineers to jump into ML/DS world. One of my friends told me why he started working on Red Data tools that he wanted to change his field, and it’s an excellent area to join.

If you interested in this movement, let’s join Red Data tools!

Why OSS based machine learning is good?

Thu, 03 Aug 2017 12:56:59 +0900

This article is translation of Japanese version.

After releasing of TensorFlow, the movement of OSS-based machine learning is accelerating. François Chollet, the creator of Keras, says the essential point of this change. I think his phrase is enough, but in this article, I would like to organize why open source machine learning is great, and what recent trends are.

tl;dr

Machine learning and deep learning frameworks have become standard things for software engineers
Since arXiv becomes very famous, many papers are published before peer review of international conferences. This change made easier for other companies to validate the algorithm.
Many researchers have been started to study machine learning, machine learning researches in academia become Red Oceanic.
The strategy, “Make a great algorithm, but the implementation is secret” becomes a thing of the past.

Halcyon days

Five or ten years ago, almost all players working on advanced machine learning were in laboratories such as universities or large enterprises, or some advanced companies. In particular, the amount of data with a label was smaller than the present, and many researchers had been improving the performance by researching algorithms, by feature engineering.

Many researchers from academia studied state-of-the-art machine learning, posted to international conferences. Most of the insights were shared after peer review. Implementation was not shared as much as now, and each researcher had to reimplement the preceding research from scratch. A typical cycle for releasing new algorithms was a half year, in some cases more than a year.

There were few open source machine learning libraries/frameworks like Weka. scikit-learn, released in 2010, was not famous among software engineers. Many of us used libraries with single/few algorithms such as libsvm and liblinear.

Fast moving era

As of 2017, people who work in machine learning have significantly increased compared with 10 years ago. The center of machine learning has been moved from academia to companies with large data. In particular, software engineers, who have never worked on machine learning, entering deep learning world. I was surprised to hear that my friend of the community who had never worked on machine learning in business had started working on Deep Learning. The reasons for this movement are 1) it became general for companies to store large data that can be used for machine learning,
2) excellent machine learning frameworks have been increased, and 3) the GPU power leverage Deep Learning for efficient calculation.

Many open source libraries became popular not only in the frameworks of Deep Learning such as TensorFlow, Chainer, MXNet, Caffe 2, PyTorch but also by XGBoost, Lightgbm, which are famouse among kaggler. scikit-learn is also common tool as a framework to experiment with multiple algorithms.

The rise of “open papers”

This movement is supported by machine learning competition site “kaggle”, and by a place to post open papers called “arXiv”. (There is discussion arxiv does not have a peer review process and quality is not assured. So can we call the document as a research paper? But, in this post, I will call the research paper style report as “paper”)

The following article describes the number of paper submissions related to machine learning (especially Deep Learning) submitted to arXiv. According to this article, it is pointed out that the number of papers related to machine learning has more than quadrupled in 2017 compared to five years ago.

A Peek at Trends in Machine Learning
_Have you looked at Google Trends? It’s pretty cool — you enter some keywords and see how Google Searches of that term…_medium.com

Papers of arXiv are posted every day. It means, state-of-the-art results from such as Google, Facebook, Microsoft, etc. are published more and more before peer review. This is a challenge for the central laboratories of the traditional large enterprises to research and develop cutting edge algorithms of machine learning itself. Those companies usually set targets for a year or half a year. There is also criticism of “just adding parts”, but it is clear that the speed of developing machine learning algorithms is significantly fast.

In the field of machine translation, the breakthrough in deep learning was encoder-decoder and attention. The subsequent papers are not interesting, “I just put existing parts here.” I can’t understand why these papers come to the top conference.

Recently, for those who read new arXiv’s paper day and night, there is an system called “ariXiv Times” to better check new arrival documents.

arXivTimes Indicator
_Edit description_arxivtimes.herokuapp.com

Open papers accelerates Open source machine learning

This March, a paper about “Deep Forest” was published at arXiv, and it became a hot topic with the author claims that “performance is better than Deep Learning”.

\[1702.08835\] Deep Forest: Towards An Alternative to Deep Neural Networks
_Abstract: In this paper, we propose gcForest, a decision tree ensemble approach with performance highly competitive to…_arxiv.org

This method proposed in this paper, about one week (2017/3/5) after the publication (2017/2 / 28), R implementation came up and Python implementation came out after R one. A discussion was made with the following LightGBM issue on GitHub, and it came out that there was not reproducibility of the article, they can’t confirm the performance.

Support gcForest · Issue #331 · Microsoft/LightGBM
_LightGBM - A fast, distributed, high performance gradient boosting (GBDT, GBRT, GBM or MART) framework based on…_github.com

It is a symbolic event where the OSS implementation of the paper
appeared within a week after published in arXiv and the discussion of the community began.

I hear that it is increasing that the number of international conferences that require disclosing the implementation when a paper is submitted.

Conclusion

It is an essential task to develop the machine learning algorithm. Thanks to open papers, ML competition web site, and fast implementation of new algorithms as an OSS, we can adopt state-of-the-art knowledge into the business rapidly.

IMHO, it is becoming fun to focus on where we can make use of ML in business rather than developing the algorithm itself.

In other words, now, it is too hard to say “special machine learning algorithms that only our company can do”. Of course, people in academia will push these cutting-edge initiatives if they can prepare data. What is the evidence that one company invents a better algorithm quickly than most state-of-the-art people from tech giants like Google, Facebook, Microsoft, etc.? That is the reason for the strength of open source based machine learning.

Among academia, there is a famous phrase, “Standing on the shoulders of giants”, it means that we should thank previous research then we can go on to the next step. Even in machine learning based on open source, we can not ignore this phrase. We cannot ignore giants.