nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Append to Parquet
Date Thu, 30 Nov 2017 22:44:11 GMT
Hello,

As far as I know there is not an option in Parquet to append due to
the way it's internal format works.

The ParquetFileWriter has a mode which only has CREATE and OVERWRITE:

https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java#L105-L107

-Bryan


On Thu, Nov 30, 2017 at 5:12 PM, VinShar <VINAYSHARMAHK@gmail.com> wrote:
> Hi,
>
> Is there any way to use PutParquet to append to an existing parquet file? i
> know that i can create a Kite DataSet and write parques to it but i am
> looking for an alternate to Spark's DataFrame.write.parquet (destination,
> mode="overwrite")
>
> Regards,
> Vinay
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Mime
View raw message