Machine Learning Project and Scrum
I’ve worked on several machine learning projects, and intuitively, I’ve felt that Scrum doesn’t seem well-suited for machine learning. However, during an internal discussion, a colleague said, “If we use Technical Stories, we should be able to break down tasks to fit within two weeks for any tasks. And if we do that, we should be able to deliver value in two weeks for ML products.” I couldn’t properly counter this, so I’m writing this article to articulate my thoughts, including how others in the world are approaching this.
2025-05-02
Migrated From Netlify to Cloudflare Pages
Netlify is a great service, but it is also known as slowness in Japan. I have been using Netlify for my blog hosting for a long time, but I decided to migrate to Cloudflare Pages to improve the speed of access to my blog from Japan.
2024-02-02
Scrape Notion and convert into PDF
I love VanGohan, who is a Japanese meal kits provider in Vancouver. Their meal kits are really tasty and authentic Japanese foods. I can’t live without them. When I visited Japan last year, I wasn’t too eager to find nice Japanese restaurants because of them.
2024-01-26
tabula-py 2.8.0 now uses jpype to launch JVM
Recently, I released tabula-py 2.8.0. It is a major release because it uses jpype to launch JVM. This means that it reduces JVM launch time since jpype reuse JVM via JNI. How fast is it? I measured read_pdf_with_template function execution time, which repeatedly launches Java process in the previous version.
2023-09-09
4 Steps to Release a CLI in Python
This is what I learned from creating a Python CLI (digdaglog2sql) in a day. In just 4 steps, you can release a CLI written in Python easily. Create a project by using poetry Poetry is a modern Python packaging and dependency management tool. Poetry is becoming popular and defacto rapidly.
2022-05-20
Create data lineage from Trino/Hive queries in digdag log with Python
What’s data lineage? Data lineage is something to describe “Where this data comes from and where it goes?” I learned this term in my previous job. They provided “Cloudera Navigator” which includes data lineage from execution logs of Hive/Spark etc.
2022-05-05
3 configs add recommend articles into your Hugo blog by GitHub Actions
Hugo has a feature to show keyword based related articles. Yeah, keyword based articles might be useful, for people who can manage keyword, category, etc, constantly. I’d love to add content based recommendation that doesn’t require to write explicit keywords by myself. Then, I found an open source named “Prelims” which is developed by takuti.
2022-01-25
py> operator development guide for Python users
This article show how to develop a digdag Python workflow task efficiently.
2020-03-05
How to release Python package from GitHub Actions
Photo by Hitesh Choudhary on Unsplash Recently, I changed my CI from Travis to GitHub Actions. GitHub Actions is handy and useful for testing, publishing Python packages. Testing Python code on GitHub Actions Migration from Travis is super easy, just writing a simple workflow like:
2019-11-26
How to test a new Docker image for digdag workflow on CircleCI?
Photo by [Campaign Creators](https://unsplash.com/@campaign_creators?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com?utm_source=medium&utm_medium=referral) Photo by Campaign Creators on Unsplash Testing workflow runnability would be important when we build a complex workflow. digdag is a workflow engine which syntax is simple and is able to run tasks with SQL, Python, Ruby, shell script, etc. digdag has Docker executor and it works like a charm with py>, rb>, and sh> operators.
2019-10-06