drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: create a huge size of parquet file
Date Fri, 20 Dec 2013 08:04:28 GMT
Oops.

I forgot to include a link:

https://github.com/tdunning/log-synth



On Thu, Dec 19, 2013 at 11:32 PM, YouPeng Yang <yypvsxf19870706@gmail.com>wrote:

> Hi Ted
>
>   Thank you very much for your reply.
> I am considering to build a map-reduce version.
>
>
> Regards
>
>
> 2013/12/20 Ted Dunning <ted.dunning@gmail.com>
>
> > You could adapt the log-synth program to produce files in parquet.  At
> the
> > same time, I would recommend that you build a map-reduce version as well
> so
> > that you can generate much bigger files.
> >
> >
> > On Thu, Dec 19, 2013 at 10:55 PM, YouPeng Yang <
> yypvsxf19870706@gmail.com
> > >wrote:
> >
> > > Hi
> > >    I have complied the drill M1 with hadoop-2.2. And I have tested the
> > > sample-data/region.parquet in HDFS.It goes well.
> > >
> > >   I want to make a deep test on the performance of dirll.
> > >   First thing it need to create a huge size of parquet file. Is there
> any
> > > tools to convert my flat file or other format file to parquet file?
> > >
> > >
> > >
> > > Regards.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message