hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bing Jiang <jiangbinglo...@gmail.com>
Subject Re: Tez and MapReduce
Date Tue, 02 Sep 2014 02:27:21 GMT
By the way, mapreduce.framework.name can be set yarn or yarn-tez. It will
make differences.


2014-09-02 8:24 GMT+08:00 jay vyas <jayunit100.apache@gmail.com>:

> Yes as an example of running a mapreduce job followed by a tez you can see
> our last post on this
> https://blogs.apache.org/bigtop/entry/testing_apache_tez_with_apache .
> You can see in the bigtop/tez testing
> blogpost that you can confirm that Tez is being used easily on the web ui.
>
> From TezClent.java:
>
>
> /**
>  * TezClient is used to submit Tez DAGs for execution. DAG's are executed
> via a
>  * Tez App Master. TezClient can run the App Master in session or
> non-session
>  * mode. <br>
>  * In non-session mode, each DAG is executed in a different App Master that
>  * exits after the DAG execution completes. <br>
>  * In session mode, the TezClient creates a single instance of the App
> Master
>  * and all DAG's are submitted to the same App Master.<br>
>  * Session mode may give better performance when a series of DAGs need to
>  * executed because it enables resource re-use across those DAGs.
> Non-session
>  * mode should be used when the user wants to submit a single DAG or wants
> to
>  * disconnect from the cluster after submitting a set of unrelated DAGs.
> <br>
>  * If API recommendations are followed, then the choice of running in
> session or
>  * non-session mode is transparent to writing the application. By changing
> the
>  * session mode configuration, the same application can be running in
> session or
>  * non-session mode.
>  */
>
>
>
> On Mon, Sep 1, 2014 at 12:43 PM, Alexander Pivovarov <apivovarov@gmail.com
> > wrote:
>
>> e.g. in hive to switch engines
>> set hive.execution.engine=mr;
>> or
>> set hive.execution.engine=tez;
>>
>> tez is faster especially on complex queries.
>> On Aug 31, 2014 10:33 PM, "Adaryl "Bob" Wakefield, MBA" <
>> adaryl.wakefield@hotmail.com> wrote:
>>
>>>   Can Tez and MapReduce live together and get along in the same cluster?
>>> B.
>>>
>>
>
>
> --
> jay vyas
>

Mime
View raw message