db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <kristian.waa...@oracle.com>
Subject Re: Fwd: [jira] [Created] (INFRA-4026) Jira no longer allows export of more than 1000 issues. Gives HTTP Status 403 - You are not allowed to get a result set of more than 1000 results.
Date Fri, 14 Oct 2011 08:38:15 GMT
On 10/13/11 02:47 PM, Rick Hillegas wrote:
> On 10/13/11 3:12 AM, Kristian Waagan wrote:
>> On 10/13/11 12:48 AM, Kathey Marsden wrote:
>>> FYI, I filed this IFRA issue which has some impact on the Derby Jira 
>>> VTI in our demos.
>>> I suppose the demo will still work but now Jira export is limited to 
>>> 1000 issues.  Is a DERBY issue in order as well?
>>
>> Working around that limit is possible in this case [1] - the question 
>> is if we want to do that?
>>
>> If people are using this demo extensively as their own private "Derby 
>> JIRA cache", I think some effort should be invested into writing 
>> something that imposes less load on the ASF JIRA instance.
>>
>> Writing something solid here may require quite a bit of effort.
>> Without having given this extensive thought, I think parsing emails 
>> as suggested on infra is the kindest option. This would probably suit 
>> committers well.
>> Maybe the activity stream (as an RSS feed) could be used too, but in 
>> that case it may be harder to make sure no updates are missed.
>> A third option is to use the SOAP API. One has to be able to define a 
>> suitable search (for instance 'give me all updates on Derby from date 
>> X to date Y'), but I suspect you'd have to download all the issue 
>> information instead of just what changed in that period. The upside 
>> with this approach compared to the RSS one, is that you decide when 
>> you want to update your copy.
> Now that we (by which I mean Kristian) know how to use the SOAP API, 
> it may be possible to write a restricted vti that wraps the SOAP API. 
> I believe that the existing vti is used to dramatically reduce the set 
> of issues you have to look at by applying restrictions which cannot be 
> expressed in the JIRA gui. 

What are examples of such restrictions?

> If I understand correctly, those restrictions can be expressed in the 
> SOAP API. By pushing those restrictions into the SOAP query, we would 
> dramatically reduce the data set being requested from JIRA in the 
> first place. That should satisfy the objections of the INFRA group.

I'm still a little puzzled by how this demo/tool is being used. There's 
a big difference if we go from parsing a local XML file multiple times, 
to accessing the JIRA instance through the SOAP API on every invocation 
(for instance as you are refining your SQL query).

FYI, pulling in SOAP requires some extra libraries. A tool as described 
above would probably be a Maven project like the SOAP client used to 
generate parts of the release notes.


Regards,
-- 
Kristian

>
> Thanks,
> -Rick
>>
>> All solutions require some kind of initial database population 
>> functionality, which can be "abused" in the same way as the current 
>> approach (before the JIRA limit was in place).
>>
>> The requirements for the "permanent Derby JIRA cache" and the demo 
>> differ, and it may be unwise to include something that will reach out 
>> to the ASF JIRA instance out of the box. If I understand this 
>> correctly, you currently have to manually obtain the XML file to be 
>> parsed by the VTI.
>>
>>
>> -- 
>> Kristian
>>
>> [1] This would be a two-step task, it requires changes both on how we 
>> interact with JIRA and how the VTI works.
>>
>>>
>>> Thanks
>>>
>>>
>>> Kathey
>>>
>>> -------- Original Message --------
>>> Subject:     [jira] [Created] (INFRA-4026) Jira no longer allows 
>>> export of more than 1000 issues. Gives HTTP Status 403 - You are not 
>>> allowed to get a result set of more than 1000 results.
>>> Date:     Wed, 12 Oct 2011 22:45:12 +0000 (UTC)
>>> From:     Kathey Marsden (Created) (JIRA) <jira@apache.org> 
>>> <mailto:jira@apache.org>
>>> To:     kmarsdenderby@sbcglobal.net 
>>> <mailto:kmarsdenderby@sbcglobal.net>
>>>
>>>
>>>
>>> Jira no longer allows export of more than 1000 issues. Gives HTTP 
>>> Status 403 - You are not allowed to get a result set of more than 
>>> 1000 results.
>>> --------------------------------------------------------------------------------------------------------------------------------------------------

>>>
>>>
>>>                   Key: INFRA-4026
>>>                   URL:https://issues.apache.org/jira/browse/INFRA-4026
>>>               Project: Infrastructure
>>>            Issue Type: Bug
>>>        Security Level: public (Regular issues)
>>>            Components: JIRA
>>>              Reporter: Kathey Marsden
>>>
>>>
>>> Occasionally I will export  the DERBY Jira issues so I can perform 
>>> queries on them with Derby Jira VTI provided in the Derby Demos. 
>>> Seehttp://wiki.apache.org/db-derby/HowToRunJiraVtiReports.  Once the 
>>> issues are dumped I can run lots of analytic queries with the VTI 
>>> without taxing  Jira and without manual intervention.
>>>
>>> I have noticed that recently though, that the maximum rows returned 
>>> from Jira has been reduced to 1000.
>>>
>>> e.g.
>>> https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none

>>> <https://issues.apache.org/jira/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?pid=10594&sorter/field=issuekey&sorter/order=DESC&tempMax=6000&reset=true&decorator=none>

>>>
>>>
>>> now yields an error:
>>> HTTP Status 403 - You are not allowed to get a result set of more 
>>> than 1000 results. Current search returns 5450 results
>>>
>>> type Status report
>>>
>>> message You are not allowed to get a result set of more than 1000 
>>> results. Current search returns 5450 results
>>>
>>> description Access to the specified resource (You are not allowed to 
>>> get a result set of more than 1000 results. Current search returns 
>>> 5450 results) has been forbidden.
>>>
>>> Can you please restore the previous defaults so the data dump can 
>>> occur ?
>>>
>>>
>>>
>>>
>>> .
>>>
>>>
>>> -- 
>>> This message is automatically generated by JIRA.
>>> If you think it was sent incorrectly, please contact your JIRA 
>>> administrators:https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa

>>> <https://issues.apache.org/jira/secure/ContactAdministrators%21default.jspa>

>>>
>>> For more information on JIRA, 
>>> see:http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>
>


Mime
View raw message