arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anders johansson <anders.johans...@tickup.se>
Subject Re: [C++] error when writing Timestamps in NANOS resolution using StreamWriter to parquet files
Date Wed, 09 Dec 2020 11:27:13 GMT
Hi,

Thanks for pointing that out

On Wed, Dec 9, 2020 at 11:20 AM Uwe L. Korn <uwelk@xhochy.com> wrote:

> Hello Anders,
>
> you have twice the same time_type in your mail. I guess one of them should
> be different?
>
> Cheers
> Uwe
>
> On Wed, Dec 9, 2020, at 11:00 AM, anders johansson wrote:
>
> Hi,
>
> I am trying to write time stamps in int64_t format representing time in
> UTC normalized nanoseconds to a parquet file.
>
> I'm using the following code:
>
> auto time_type = LogicalType::Timestamp(true,
> LogicalType::TimeUnit::NANOS, false, false);
> NodeVector nv;
>
> nv.push_back(PrimitiveNode::Make("Time", Repetition::REQUIRED,  time_type,
> Type::INT64));
>
> but when I try to write to the output stream
>
> std::shared_ptr<parquet::StreamWriter> parquet_os_;
> *parquet_os_ << se.time; /* time is uint64_t */
>
> I get the following runtime error:"Column converted type mismatch.  Column
> 'Time' has converted type[NONE] not 'INT_64'"
>
> Everything works fine if I set:
>
> auto time_type = LogicalType::Timestamp(true,
> LogicalType::TimeUnit::NANOS, false, false);
>
> but I want it as Time or Timestamp so that I get it in the proper format
> when I read the file using pandas in python.
>
> Thanks,
> Anders
>
>
>

Mime
View raw message