hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wong (Jira)" <>
Subject [jira] [Updated] (HUDI-614) EMR Presto cannot read Hudi tables
Date Tue, 25 Feb 2020 22:28:00 GMT


Andrew Wong updated HUDI-614:
    Summary: EMR Presto cannot read Hudi tables  (was: .hoodie_partition_metadata created
for non-partitioned table)

> EMR Presto cannot read Hudi tables
> ----------------------------------
>                 Key: HUDI-614
>                 URL:
>             Project: Apache Hudi (incubating)
>          Issue Type: Bug
>    Affects Versions: 0.5.0, 0.5.1
>            Reporter: Andrew Wong
>            Priority: Major
> Original issue: []
> I made a non-partitioned Hudi table using Spark. I was able to query it with Spark &
Hive, but when I tried querying it with Presto, I received the error {{Could not find partitionDepth
in partition metafile}}.
> I attempted this task using emr-5.28.0 in AWS. I tried using the built-in spark-shell
with both Amazon's /usr/lib/hudi/hudi-spark-bundle.jar (following [] and
the org.apache.hudi:hudi-spark-bundle_2.11:0.5.1-incubating jar (following []).
> I used NonpartitionedKeyGenerator & NonPartitionedExtractor in my write options,
according to [].
You can see my code in the github issue linked above.
> In both cases I see the .hoodie_partition_metadata file was created in the table path
in S3. Querying the table worked in spark-shell & hive-cli, but attempting to query the
table in presto-cli resulted in the error, "Could not find partitionDepth in partition metafile".
> Please look into the bug or check the documentation. If there is a problem with the EMR
install I can contact the AWS team responsible.
> cc: [~bhasudha]

This message was sent by Atlassian Jira

View raw message