hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karthik Srivasthava <karthiksrivasth...@gmail.com>
Subject Re: List of dates as arguments
Date Sun, 31 Aug 2014 17:18:16 GMT
dt Between $startdate and $enddate  is working..

But we want only list_dates  to use as dt in ($listofdates) as this is
faster.

Usingpowershell i passed list of dates and invoke-hive -file   .hql .. but
it works only when the datelist has only one date..

I think there is a syntax error.?


On Sun, Aug 24, 2014 at 2:56 PM, karthik Srivasthava <
karthiksrivasthava@gmail.com> wrote:

> Yes.. Its working
>
>
> On Sun, Aug 24, 2014 at 2:31 PM, Bala Krishna Gangisetty <
> bala@altiscale.com> wrote:
>
>> Yes, it does. Did you enclose the currentdate value in ' ', or " "?
>>
>>
>>
>> On Sun, Aug 24, 2014 at 12:22 PM, karthik Srivasthava <
>> karthiksrivasthava@gmail.com> wrote:
>>
>>> Bala Krishna Gangisetty and Nithin,
>>>
>>> Thanks for your response.. Your response was very helpful..
>>>
>>> But date  datatype being STRING, will it encourage this query ??
>>>
>>> SELECT * FROM *table_name* WHERE *date_column* BETWEEN date_sub(*${hiveconf:currentdate},
>>> 15)* AND  *${hiveconf:currentdate}*
>>>
>>> *I am getting zero records back...*
>>>
>>>
>>> On Sun, Aug 24, 2014 at 2:04 PM, Bala Krishna Gangisetty <
>>> bala@altiscale.com> wrote:
>>>
>>>> Nitin: Given the requirement, I don't think it's required to pass array
>>>> of dates to HIVE query.
>>>>
>>>>
>>>> On Sun, Aug 24, 2014 at 11:56 AM, Bala Krishna Gangisetty <
>>>> bala@altiscale.com> wrote:
>>>>
>>>>> The query can be modified further with one variable with the of
>>>>> date_sub() UDF function, available by default.
>>>>>
>>>>> Suppose currentdate is the argument whose value is current date, the
>>>>> query will look like:
>>>>>
>>>>> SELECT * FROM *table_name* WHERE *date_column* BETWEEN date_sub(*${hiveconf:currentdate},
>>>>> 15)* AND  *${hiveconf:currentdate}*
>>>>>
>>>>> However, the query comes with an overhead of UDF execution.
>>>>>
>>>>> --Bala G.
>>>>>
>>>>>
>>>>> On Sun, Aug 24, 2014 at 11:25 AM, Bala Krishna Gangisetty <
>>>>> bala@altiscale.com> wrote:
>>>>>
>>>>>> Here is my understanding on your requirements. Let me know if I am
>>>>>> missing something. You,
>>>>>>
>>>>>> a) would like to run a query daily to find top 10 products in the
>>>>>> past 15 days
>>>>>> b) would like to pass dates dynamically as arguments to HIVE query
>>>>>>
>>>>>> Given the requirement a), passing just two variables(startdate and
>>>>>> enddate) to HIVE query will suffice to achieve the requirement b).
>>>>>>
>>>>>> Assuming startdate and enddate variables are passed to HIVE query,
>>>>>> the query will look like below.
>>>>>>
>>>>>> SELECT * FROM *table_name* WHERE *date_column* BETWEEN
>>>>>> *${hiveconf:startdate}* AND  *${hiveconf:enddate}*
>>>>>>
>>>>>> Note, values for startdate and enddate must be enclosed in ' '.
>>>>>>
>>>>>> Hope this helps.
>>>>>>
>>>>>> --Bala G.
>>>>>>
>>>>>>
>>>>>> On Sun, Aug 24, 2014 at 12:57 AM, Nitin Pawar <
>>>>>> nitinpawar432@gmail.com> wrote:
>>>>>>
>>>>>>> I am not sure if you can transform array from shell to java,
you may
>>>>>>> want to write your own custom UDF for that
>>>>>>>
>>>>>>> if these are continuous dates, then you can have less than greater
>>>>>>> than comparison
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Aug 24, 2014 at 12:39 PM, karthik Srivasthava <
>>>>>>> karthiksrivasthava@gmail.com> wrote:
>>>>>>>
>>>>>>>> Nitin,
>>>>>>>> Teja
>>>>>>>>
>>>>>>>> Thank you.. I exactly need what Teja suggested... i need
list of
>>>>>>>> dates between start date and end date....
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Aug 24, 2014 at 2:05 AM, Teja Kunapareddy <
>>>>>>>> tejakunapareddy@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks Nithin For your reply.. I can get start date and
end date,.
>>>>>>>>> But can i get all the dates with in START DATE AND END
DATE.??? . so that
>>>>>>>>> my query looks something like this
>>>>>>>>>
>>>>>>>>> "Select  a, b, c from table_x where date in  (${hiveconf:LIST_OF
>>>>>>>>> DATES})"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 24 August 2014 01:18, Nitin Pawar <nitinpawar432@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> with your shell script calculate your start date
and end date
>>>>>>>>>> hive $HIVEPARAMS -hiveconf startdate="$var1"  -hiveconf
>>>>>>>>>> enddate="$var2"
>>>>>>>>>>
>>>>>>>>>> also set in ..hiverc
>>>>>>>>>> set hive.variable.substitute=true;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Aug 24, 2014 at 10:19 AM, karthik Srivasthava
<
>>>>>>>>>> karthiksrivasthava@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> As my raw-data table is partitioned by date..
i want to get data
>>>>>>>>>>> to run a query every days to find top 10 products
in last 15 days .
>>>>>>>>>>>
>>>>>>>>>>> How to pass list of dates dynamically as arguments
in hive query
>>>>>>>>>>> using hiveconf?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Nitin Pawar
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Nitin Pawar
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message