asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ildar Absalyamov <>
Subject Re: New Asterix REST API design
Date Sat, 16 Apr 2016 00:22:46 GMT
I agree that the example where CSV is embedded into return JSON looks quirky (and I am not
the big fan of it either).
I believe the tradeoff here is following: do we want to keep number of API calls just to get
the data minimum, or logically separate metadata (like plans, execution time metrics, etc)
from the data on the endpoint level.
I have tried to address the former case, however left an option to make this logical separation
if the user is wiling to do that (via include-results parameter). There is no real way to
do it other way around, since the plans, etc are generated before query is scheduled and any
results could be returned.

> On Apr 15, 2016, at 17:13, Till Westmann <> wrote:
> Yes, this API is not ideal for "just getting the data". However, Ildar’s
> goal was to separate the data from the HTML and to build an API that can be
> the basis for the Web-interface - and I think that the API looks good for
> that :)
> I'm wondering if an endpoint to get the data should be an option on this one
> or a different endpoint. The reason is, that all of the additional request
> metadata that we can ask for (plan, metrics, warnings, ..) cannot easily be
> returned with such an API. An API that play well with curl might even put
> the format into the URI, e.g.:
> curl http://host:19100/query/csv?statment=select+element+1+as+one; > one.csv
> Thoughts? Trade-offs?
> Cheers,
> Till
> On 15 Apr 2016, at 16:48, Cameron Samak wrote:
>> That hop is exactly what I think should be (optionally) avoidable though
>> because
>>   1. The user still needs to parse both JSON (to get the URL) along with
>>   the other format (i.e. CSV)
>>   Consider curl {myquery} > myoutput.csv. That's harder with the proposed
>>   API.
>>   2. It's an unnecessary round trip back to the server (which, depending
>>   on the environment, can be significant esp. with quick queries).
>> Understood for the result distribution + serialization.
>> Cameron
>> On Fri, Apr 15, 2016 at 4:24 PM, Till Westmann <> wrote:
>>> I had a misunderstanding that I think I clarified now. I believed that we
>>> don’t have the separation into tuples anymore after result distribution and
>>> that we only have bytes that we pass to the client. In that case limiting
>>> in
>>> the HTTP server would have had to choose between
>>> a) limiting based on the number of bytes or
>>> b) re-establishing tuple boundaries.
>>> However, even though result distribution has serialized the tuples to
>>> whatever format (ADM, JSON, CSV), we still send frames and so we should be
>>> able to separate the tuples (and limit the number that we return).
>>> So I think that it should be feasible to add that (feature creep is coming
>>> ... :) )
>>> Cheers,
>>> Till
>>> On 15 Apr 2016, at 14:55, Mike Carey wrote:
>>> I read this much more simply:  Can we enhance the API, in the case where
>>>> you start with a handle and know that the results are ready now, to fetch
>>>> the results in blocks instead of as one giant result?  So still computing
>>>> the giant result - just not pushing it all back at once - seems like it
>>>> might help?
>>>> On 4/15/16 2:48 PM, Till Westmann wrote:
>>>>> Hi Wail,
>>>>> I’m not completely sure that I understand how to implement the idea.
>>>>> we
>>>>> do this only in the API, it might be tricky to get the boundaries between
>>>>> records right (e.g. if we do indentation on the server). However, if
>>>>> want
>>>>> to push this into the query engine, we need to understand enough of the
>>>>> query/statements to put the limit clause in.
>>>>> Both approaches don't look great to me.
>>>>> What did you have in mind?
>>>>> Cheers,
>>>>> Till
>>>>> On 15 Apr 2016, at 13:19, Wail Alkowaileet wrote:
>>>>> Hi Ildar,
>>>>>> I think if there's something I would love to have is getting partial
>>>>>> result
>>>>>> instead of all result at once. This can be beneficial for result
>>>>>> pagination. When I use AsterixDB UI, 50% of the time my tab crashes
>>>>>> forget to limit the result).
>>>>>> Thanks...
>>>>>> On Fri, Apr 15, 2016 at 1:23 AM, Ildar Absalyamov <
>>>>>>> wrote:
>>>>>> Hi Devs,
>>>>>>> Recently there have been a number of conversations about the
future of
>>>>>>> our
>>>>>>> REST (aka HTTP) API. I summarized these discussions in an outline
>>>>>>> the
>>>>>>> new API design:
>>>>>>> <
>>>>>>>> .
>>>>>>> The need to refactor existing API came from different directions
>>>>>>> from
>>>>>>> different people), and is explained in motivation section. Thus
>>>>>>> believe
>>>>>>> it’s about the time to take an effort and improve existing
API, so
>>>>>>> that it
>>>>>>> will not drag us down in the future. However during the transition
>>>>>>> step I
>>>>>>> believe it would be better to keep exiting API endpoints, so
that we
>>>>>>> would
>>>>>>> not break people’s current experimental setup.
>>>>>>> It would be good to know feedback from the folks, who have been
>>>>>>> contributing to that part of the systems recently.
>>>>>>> Best regards,
>>>>>>> Ildar
>>>>>> --
>>>>>> *Regards,*
>>>>>> Wail Alkowaileet

Best regards,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message