Data is Nutritious

Data Engineer's Memo

Entries from 2019-09-01 to 1 month

AWS Glue's GetPartition API is slow for table with much Partitions.

Introduction AWS Glue is very useful Hive Metastore service for people using Hive on EMR / Spark on EMR / Presto on Athena. I felt that fetching partitions is very slow, especially tables with much partitions. Technically users need to cal…