hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Kelkar <rohitkel...@gmail.com>
Subject Re: Query Regarding design MR job for Billing
Date Mon, 27 Feb 2012 10:36:29 GMT
Just trying to understand your use case
you need an hour job to run on data between 6:40 AM and 7:40 AM. Would
it be like a moving window? For ex. run hour jobs on
6:41 AM to 7:41 AM
6:42 AM to 7:42 AM
and so on...


On Mon, Feb 27, 2012 at 1:01 PM, Stuti Awasthi <stutiawasthi@hcl.com> wrote:
> Hi all,
>
> I have to implement BillingEngine using MR jobs. My usecase is like this:
> I will be having data files of format <TimeStamp> <Information for Billing>.
> Now these datafiles will be containing timestamp either at minute interval, hour inverval,
day interval, month interval, year interval. Every type of interval will be having different
type of calculation for billing so basically different jobs for every type of interval.
>
> Suppose I have a data file which contain minute interval timestamp. I have a scenario
that if data is present for hours , then it should be processed by hourly job and remaining
will be processed by minutejob.
>
> Example :
>
> 2/10/12 6:40 AM <data for billing>
> 2/10/12 6:40 AM <data for billing>
> .
> 2/10/12 6:45 AM <data for billing>
> 2/10/12 6:45 AM <data for billing>
> .
> .
> 2/10/12 7:40 AM <data for billing>
> 2/10/12 7:40 AM <data for billing>
> .
> .
> 2/10/12 7:45 AM <data for billing>
> 2/10/12 7:45 AM <data for billing>
> .
>
> Now I want data between 2/10/12 6:40 AM to 2/10/12 7:40 AM is processed by Hourjob and
2/10/12 7:45 AM is processed by MinuteJob.
> Please suggest how to design my MR to achieve this.
>
> Thanks
> Stuti
>
> ::DISCLAIMER::
> -----------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and intended for the
named recipient(s) only.
> It shall not attach any liability on the originator or HCL or its affiliates. Any views
or opinions presented in
> this email are solely those of the author and may not necessarily reflect the opinions
of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure, modification, distribution
and / or publication of
> this message without the prior written consent of the author of this e-mail is strictly
prohibited. If you have
> received this email in error please delete it and notify the sender immediately. Before
opening any mail and
> attachments please check them for viruses and defect.
>
> -----------------------------------------------------------------------------------------------------------------------

Mime
View raw message