reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shouheng Yi <sho...@microsoft.com.INVALID>
Subject RE: Updating Avro from 1.7.7 to 1.8.1
Date Wed, 05 Apr 2017 19:40:25 GMT
I ran builds and tests locally both for Java and .Net (with filter -Trait:Environment). All
tests passed.

-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de] 
Sent: Wednesday, April 5, 2017 10:45 AM
To: REEF Developers Mailinglist <dev@reef.apache.org>
Subject: Re: Updating Avro from 1.7.7 to 1.8.1

Can you update the version locally and run all the tests, both Java and .NET? We need to be
sure that updated Avro on the Java side does not break the end-to end tests for .NET.

Markus

On Tue, Apr 4, 2017 at 4:24 PM, Shouheng Yi <shouyi@microsoft.com.invalid>
wrote:

> * Made some changes in the original email to comply with the email format.
>
> Hi Reef devs,
>
> Right now we have an interest of adding a parquet reader in REEF in 
> REEF-1765[1]. The reason is that we want to consume parquet file 
> format which is widely used in Hadoop and Spark.
>
> To do that, my current solution requires Avro 1.8.1 for it to work. I 
> understand that there are a lot of moving parts in Avro. Therefore I'm 
> asking if updating 1.7.7 is viable. Thank you for your feedback.
>
> Best,
> Shouheng
>
> [1]: 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> s.apache.org%2Fjira%2Fbrowse%2FREEF-1765&data=02%7C01%7Cshouyi%40micro
> soft.com%7C8497eb8fe01f4acaa48208d47c4b80a2%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C636270111098622631&sdata=f9KcR%2BjtIfUMQ2tJxiXXmMOD
> NsZfzfOBGKv3vvoLsnU%3D&reserved=0 < 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> s.apache.org%2Fjira%2Fbrowse%2FREEF-1765&data=02%7C01%7Cshouyi%40micro
> soft.com%7C8497eb8fe01f4acaa48208d47c4b80a2%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C636270111098622631&sdata=f9KcR%2BjtIfUMQ2tJxiXXmMOD
> NsZfzfOBGKv3vvoLsnU%3D&reserved=0>
>
>
>
> -----Original Message-----
> From: Shouheng Yi [mailto:shouyi@microsoft.com.INVALID]
> Sent: Tuesday, April 4, 2017 4:20 PM
> To: dev@reef.apache.org
> Subject: Updating Avro from 1.7.7 to 1.8.1
>
> Hi Reef devs,
>
> Right now we have an interest of adding a parquet reader in REEF ( 
> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FREEF-
> 1765&data=02%7C01%7Cshouyi%40microsoft.com%7C6bc252f85a9e45a3d5ef08d47
> bb1 246d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> 7C636269448124808079&sdata=IZITo0K8aDXPhfbF9771YMcbbeGMvN
> Q2RePQO3u0JnE%3D&reserved=0). The reason is that we want to consume 
> parquet file format which is widely used in Hadoop and Spark.
>
> To do that, my current solution requires Avro 1.8.1 for it to work. I 
> understand that there are a lot of moving parts in Avro. Therefore I'm 
> asking if updating 1.7.7 is viable. Thank you for your feedback.
>
> Best,
> Shouheng
>
Mime
View raw message