arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe L. Korn" <uw...@xhochy.com>
Subject Re: [C++] error when writing Timestamps in NANOS resolution using StreamWriter to parquet files
Date Wed, 09 Dec 2020 10:20:26 GMT
Hello Anders,

you have twice the same time_type in your mail. I guess one of them should be different?

Cheers
Uwe

On Wed, Dec 9, 2020, at 11:00 AM, anders johansson wrote:
> Hi,
> 
> I am trying to write time stamps in int64_t format representing time in UTC normalized
nanoseconds to a parquet file.
> 
> I'm using the following code:
> 
> auto time_type = LogicalType::Timestamp(true, LogicalType::TimeUnit::NANOS, false, false);
> NodeVector nv;
> 
> nv.push_back(PrimitiveNode::Make("Time", Repetition::REQUIRED,  time_type, Type::INT64));
> 
> but when I try to write to the output stream
> 
> std::shared_ptr<parquet::StreamWriter> parquet_os_;
> *parquet_os_ << se.time; /* time is uint64_t */
> 
> I get the following runtime error:"Column converted type mismatch.  Column 'Time' has
converted type[NONE] not 'INT_64'"
> 
> Everything works fine if I set:
> 
> auto time_type = LogicalType::Timestamp(true, LogicalType::TimeUnit::NANOS, false, false);
> 
> but I want it as Time or Timestamp so that I get it in the proper format when I read
the file using pandas in python. 
> 
> Thanks, 
> Anders

Mime
View raw message