hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhijie Shen <zs...@hortonworks.com>
Subject Re: Differences between HistoryServer and Yarn TimeLine server?
Date Thu, 24 Apr 2014 05:15:12 GMT
Sam,

You're right. We can definitely integrate MapReduce to use the timeline
server to store and serve its specific data, and this is actually our plan.

However, it's a big move, and we still need time to get it done. In
addition, not to disturb the users that are currently relying on JHS for MR
job information, we cannot simply remove JHS from Hadoop.


On Wed, Apr 23, 2014 at 8:15 PM, sam liu <samliuhadoop@gmail.com> wrote:

> Zhijie,
>
> I am much clear now. Thanks a lot!
>
> As my understanding, besides previous Job History Server, hadoop now has a
> new timeline server which could restore both the generic YARN application
> history and the framework specific information. However, I think the
> timeline server also include the functions of Job History Server, because
> it can store the framework specific information(of course, include
> mapreduce framework). In another words, Job History Server is not necessary
> any more.* If that's the case, why hadoop still include Job History
> Server?*
>
>
> 2014-04-23 12:56 GMT+08:00 Zhijie Shen <zshen@hortonworks.com>:
>
>> In Hadoop 2.4, we have delivered the timeline server at a preview stage,
>> which actually can serve some generic YARN application history as well as
>> the framework specific information. Due to the development logistics, we
>> have created the two concepts: History Server and Timeline Server. To be
>> simple, you can consider the history server of the service of the generic
>> YARN application information, while consider the timeline server of the
>> service of the framework specific information. Importantly, we just have
>> one daemon, which includes both services, and which we'd like to call
>> timeline server (unfortunately, the confusing thing is that the command to
>> start the daemon is "historyserver"). We're going on working on the
>> timeline server to integrate these two parts, including refactoring the
>> names.
>>
>> BTW, if you mean MapReduce JobHistoryServer by HistoryServer, it's a
>> different daemon, which serves the historic information of MapReduce jobs
>> only.
>>
>>
>> On Tue, Apr 22, 2014 at 8:44 PM, sam liu <samliuhadoop@gmail.com> wrote:
>>
>>> Hi Experts,
>>>
>>> I am confusing on these two concepts. Could you help explain the
>>> differences?
>>>
>>> Thanks!
>>>
>>
>>
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>


-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message