I created a simple sparklyr extension to handle Apache Avro file. It is just a simple wrapper of DataBrick’s spark-avro. It is listed in the official document of sparklyr extensions. chezou/sparkavro _sparkavro - Load Avro data into Spark with sparklyr_github.com Installation Use {devtools} to install sparkavro.
2017-03-26
Impala is very fast SQL-on-Hadoop, and it will enhance your R experience with implyr, a dplyr based interface for Apache Impala (incubating) created by Ian Cook. I will show you how to setup connection to Kerberized Impala cluster with implyr from local macOS. You can find my GitHub repo as follows:
2017-03-26
Redash is a famous OSS visualization tool, which enables to visualize your data with SQL. It supports Apache Impala (incubating), fast SQL-on-Hadoop suitable for BI tools and exploratory analysis. With Impala, you can query SQLs to tables on Amazon S3. In this post, we connect to Impala from Redash and visualize data.
2017-02-11
(Oct 7th, 2019) As of Oct. 2019, I launched a documentation site and Google Colab notebook for tabula-py. The FAQ would be good place to execute accurate extraction.
2017-01-09
livy is a REST server of Spark. You can see the talk of the Spark Summit 2016, Microsoft uses livy for HDInsight with Jupyter notebook and sparkmagic. Jupyter notebook is one of the most popular notebook OSS within data scientists. Using sparkmagic + Jupyter notebook, data scientists can execute ad-hoc Spark job easily.
2016-12-30
Amazon Polly, Text-to-speech service from AWS was announced at today ‘s re:Invent. Amazon Polly is speech synthesize system based on deep learning. Amazon Polly — Text to Speech in 47 Voices and 24 Languages \[updated\] I added generated speech of this article.
2016-12-01
tl;dr visualizing MovieLens 20M data (famous movie rating data) with Ibis build predictive model for movie favor with scikit-learn repo / notebook What is Ibis? Ibis is a bridge between Python and Big Data. Ibis enables pandas handling Big Data. architecture of Ibis architecture of Ibis
2016-10-15