hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Nanda <saurabhna...@gmail.com>
Subject Re: Output compression not working on hive-trunk (r802989)
Date Tue, 18 Aug 2009 10:21:35 GMT
Is this what you're talking about -- http://pastebin.ca/1533627 ? Seems like
compression is on.

Is there any difference in how CLI queries and JDBC queries are treated?

Saurabh.

On Tue, Aug 18, 2009 at 11:19 AM, Zheng Shao <zshao9@gmail.com> wrote:

> Hi Saurabh,
>
> So the compression flag is correct when the plan is generated.
> When you run the query, you should see "plan = xxx.xml" in the log
> file. Can you open that file (in HDFS) and see whether the compression
> flag is on or not?
>
> Zheng
>
> On Mon, Aug 17, 2009 at 5:17 AM, Saurabh Nanda<saurabhnanda@gmail.com>
> wrote:
> > Hey Zheng, any clues as to what the bug is? Or what I'm doing wrong? I
> can
> > do some more digging and logging if required.
> >
> > Saurabh.
> >
> > On Mon, Aug 17, 2009 at 1:28 PM, Saurabh Nanda <saurabhnanda@gmail.com>
> > wrote:
> >>
> >> Here's the log output:
> >>
> >> 2009-08-17 13:26:42,183 INFO  parse.SemanticAnalyzer
> >> (SemanticAnalyzer.java:genFileSinkPlan(2575)) - Created FileSink Plan
> for
> >> clause: insclause-0dest_path:
> >> hdfs://master-hadoop/user/hive/warehouse/raw/dt=2009-04-07 row schema:
> >> {(_col0,_col0: string)(_col1,_col1: string)(_col2,_col2:
> >> string)(_col3,_col3: string)(_col4,_col4: string)(_col5,_col5:
> >> string)(_col6,_col6: string)(_col7,_col7: string)(_col8,_col8:
> >> string)(_col9,_col9: string)(_col10,_col10: int)} .
> >> HiveConf.ConfVars.COMPRESSRESULT=true
> >>
> >> Is the SemanticAnalyszer run more than once in the lifetime of a job?
> >> Should I be looking for another log entry like this one?
> >>
> >> Saurabh.
> >>
> >> On Mon, Aug 17, 2009 at 1:26 PM, Saurabh Nanda <saurabhnanda@gmail.com>
> >> wrote:
> >>>
> >>> Strange. The compression configuration log entry was also info but I
> >>> could see it in the task logs:
> >>>
> >>>       LOG.info("Compression configuration is:" + isCompressed);
> >>>
> >>> Saurabh.
> >>>
> >>> On Mon, Aug 17, 2009 at 12:56 PM, Zheng Shao <zshao9@gmail.com> wrote:
> >>>>
> >>>> The default log level is WARN. Please change it to INFO.
> >>>>
> >>>> hive.root.logger=INFO,DRFA
> >>>>
> >>>> Of course you can also use LOG.warn() in your test code.
> >>>>
> >>>> Zheng
> >>>>
> >>> --
> >>> http://nandz.blogspot.com
> >>> http://foodieforlife.blogspot.com
> >>
> >>
> >>
> >> --
> >> http://nandz.blogspot.com
> >> http://foodieforlife.blogspot.com
> >
> >
> >
> > --
> > http://nandz.blogspot.com
> > http://foodieforlife.blogspot.com
> >
>
>
>
> --
> Yours,
> Zheng
>



-- 
http://nandz.blogspot.com
http://foodieforlife.blogspot.com

Mime
View raw message