hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "Hive/GettingStarted" by JoydeepSensarma
Date Thu, 12 Aug 2010 08:59:29 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/GettingStarted" page has been changed by JoydeepSensarma.
http://wiki.apache.org/hadoop/Hive/GettingStarted?action=diff&rev1=35&rev2=36

--------------------------------------------------

    * The total number of map-tasks is less than: ''hive.exec.mode.local.auto.tasks.max''
(4 by default)
    * The total number of reduce tasks required is 1 or 0.
  
- So for queries over small data sets, or for queries with multiple map-reduce jobs where
the input to subsequent jobs is substantially smaller (because of reduction/filtering in the
prior job), jobs may be run locally. Note that there may be differences in the runtime environment
of hadoop server nodes and the machine running the hive client (because of different jvm versions
or different software libraries). This can cause unexpected behavior/errors while running
in local mode.
+ So for queries over small data sets, or for queries with multiple map-reduce jobs where
the input to subsequent jobs is substantially smaller (because of reduction/filtering in the
prior job), jobs may be run locally. 
+ 
+ Note that there may be differences in the runtime environment of hadoop server nodes and
the machine running the hive client (because of different jvm versions or different software
libraries). This can cause unexpected behavior/errors while running in local mode. Also note
that local mode execution is done in a separate, child jvm (of the hive client). If the user
so wishes, the maximum amount of memory for this child jvm can be controlled via the option
''hive.mapred.local.mem''. By default, it's set to zero, in which case Hive lets Hadoop determine
the default memory limits of the child jvm.
  
  === Error Logs ===
  Hive uses log4j for logging. By default logs are not emitted to the 

Mime
View raw message