hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Shankar <ashwinshanka...@gmail.com>
Subject Re: Differences between HistoryServer and Yarn TimeLine server?
Date Thu, 24 Apr 2014 06:25:21 GMT
Hi Zhijie,
There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you
please let me know what is the
difference ? Is timeline server finally going to be YARN321+YARN1530 ?

You mentioned that MR is going to integrated with timeline server,is there
a jira I can watch ?

Thanks,
Ashwin


On Wed, Apr 23, 2014 at 10:15 PM, Zhijie Shen <zshen@hortonworks.com> wrote:

> Sam,
>
> You're right. We can definitely integrate MapReduce to use the timeline
> server to store and serve its specific data, and this is actually our plan.
>
> However, it's a big move, and we still need time to get it done. In
> addition, not to disturb the users that are currently relying on JHS for MR
> job information, we cannot simply remove JHS from Hadoop.
>
>
> On Wed, Apr 23, 2014 at 8:15 PM, sam liu <samliuhadoop@gmail.com> wrote:
>
>> Zhijie,
>>
>> I am much clear now. Thanks a lot!
>>
>> As my understanding, besides previous Job History Server, hadoop now has
>> a new timeline server which could restore both the generic YARN application
>> history and the framework specific information. However, I think the
>> timeline server also include the functions of Job History Server, because
>> it can store the framework specific information(of course, include
>> mapreduce framework). In another words, Job History Server is not necessary
>> any more.* If that's the case, why hadoop still include Job History
>> Server?*
>>
>>
>> 2014-04-23 12:56 GMT+08:00 Zhijie Shen <zshen@hortonworks.com>:
>>
>>>  In Hadoop 2.4, we have delivered the timeline server at a preview
>>> stage, which actually can serve some generic YARN application history as
>>> well as the framework specific information. Due to the development
>>> logistics, we have created the two concepts: History Server and Timeline
>>> Server. To be simple, you can consider the history server of the service of
>>> the generic YARN application information, while consider the timeline
>>> server of the service of the framework specific information. Importantly,
>>> we just have one daemon, which includes both services, and which we'd like
>>> to call timeline server (unfortunately, the confusing thing is that the
>>> command to start the daemon is "historyserver"). We're going on working on
>>> the timeline server to integrate these two parts, including refactoring the
>>> names.
>>>
>>> BTW, if you mean MapReduce JobHistoryServer by HistoryServer, it's a
>>> different daemon, which serves the historic information of MapReduce jobs
>>> only.
>>>
>>>
>>> On Tue, Apr 22, 2014 at 8:44 PM, sam liu <samliuhadoop@gmail.com> wrote:
>>>
>>>> Hi Experts,
>>>>
>>>> I am confusing on these two concepts. Could you help explain the
>>>> differences?
>>>>
>>>> Thanks!
>>>>
>>>
>>>
>>>
>>> --
>>> Zhijie Shen
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>



-- 
Thanks,
Ashwin

Mime
View raw message