asterixdb-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raman Grover <ramangrove...@gmail.com>
Subject Re: Socket feed questions
Date Fri, 13 Nov 2015 00:52:46 GMT
Not yet Chen.

I was outside of U.S and resumed work past Monday. I have some catching up
to do.
I think, I can get to steal some cycles tomorrow or early next week.

I will keep you posted.

Regards,
Raman

On Thu, Nov 12, 2015 at 4:38 PM, Chen Li <chenli@gmail.com> wrote:

> @Raman: did you have a chance to look into test cases of using AQL UDF
> in data feed?
>
> Chen
>
> On Mon, Nov 2, 2015 at 9:25 AM, Jianfeng Jia <jianfeng.jia@gmail.com>
> wrote:
> > Great, here is the cc log:
> >
> > Thank you!
> >
> > On Nov 2, 2015, at 5:54 AM, Raman Grover <ramangrover29@gmail.com>
> wrote:
> >
> > Hi,
> >
> > I should be able to look at this one in a few days when I return and
> resume
> > work (I am currently on a vacation till 5th Nov).
> >
> > However, the exception suggests that building the secondary feed pipeline
> > encountered an exception. Can you share the logs so that I can get a
> better
> > understanding of the sequence of steps that happened as your statement
> > executed.
> >
> > I have intermittent access to net, but should be able to revert with some
> > delays.
> > On Nov 2, 2015 2:32 AM, "Jianfeng Jia" <jianfeng.jia@gmail.com> wrote:
> >
> > I’ve tried the “apply AQLfunction” idea, but got an NullPointException:
> > java.lang.NullPointerException
> >        at
> >
> org.apache.asterix.aql.translator.AqlTranslator.getFeedJointKey(AqlTranslator.java:2268)
> >        at
> >
> org.apache.asterix.aql.translator.AqlTranslator.getFeedConnectionRequest(AqlTranslator.java:2214)
> >        at
> >
> org.apache.asterix.aql.translator.AqlTranslator.handleConnectFeedStatement(AqlTranslator.java:2130)
> >        at
> >
> org.apache.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:362)
> >        at
> >
> org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:114)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
> >        at
> > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
> >        at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
> >        at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> >        at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
> >        at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
> >        at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> >        at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
> >        at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> >        at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
> >        at org.eclipse.jetty.server.Server.handle(Server.java:347)
> >        at
> >
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
> >        at
> >
> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
> >        at
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
> >        at
> > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
> >        at
> >
> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
> >        at
> >
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
> >        at
> >
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
> >        at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
> >        at java.lang.Thread.run(Thread.java:745)
> >
> > The ddl that I was using is here:
> >
> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
> > <
> >
> https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
> >
> >
> > Anyone has any idea? Thank you!
> >
> > On Oct 30, 2015, at 12:30 AM, Heri Ramampiaro <heriram@gmail.com> wrote:
> >
> > Although I haven’t tested this (I have mostly used & created java-based
> >
> > UDFs),
> >
> > one can execute AQL calls from within a UDF. Feeds allows functions to
> > execute arbitrary AQL statements (DDL, DMLs etc). I.e., I believe what
> > you are trying to do is possible.
> >
> > For example you could do:
> > (Given that you have a rec. type called Tweet, and a dataset
> >
> > ProcessedTweets
> >
> >
> > create feed CoordTwitterFeed if not exists
> > using “push_twitter" (("type-name"="Tweet”))
> >      apply function find-intersection;
> >
> > (Here "find-intersection” is an AQL function that does the step
> >
> > specified under 2)).
> >
> >
> > To do nr. 3 the easiest way is to have a modified version of the
> >
> > “hashTag”
> >
> > (let’s call this “tweetlocator”).  Java-based UDF (see my previous
> >
> > message with the
> >
> > TweetLib example). You can then connect this  as a secondary feed,
> >
> > connected to
> >
> > “CoordTwitterFeed”.
> >
> > E.g.:
> > create secondary feed ProcessedTwitterFeed from feed CoordTwitterFeed
> > apply function “tweelib#tweetlocator";
> >
> > Thereafter you can connect the feeds to appropriate datasets.
> >
> > (Since the find-intersection is an AQL function, you can also call
> >
> > “tweelib#tweetlocator”
> >
> > inside this function).
> >
> > Best,
> > -heri
> >
> > On Oct 30, 2015, at 12:18 AM, Young-Seok Kim <kisskys@gmail.com> wrote:
> >
> > I'm not sure whether the following UDF is possible or not, but
> >
> > hopefully it
> >
> > is.
> > What we're trying to do is to have the following UDF.
> >
> > The UDF
> > 1) accepts an incoming tweet record through a feed job as an input and
> >
> > then
> >
> > 2) takes a field, more specifically, coordinate field value from the
> >
> > tweet
> >
> > record and sends a spatial-intersect query using the coordinate in
> >
> > order to
> >
> > find out the corresponding county of the coordinates (we have created
> > AsterixDB instance which stores US county shapes records into a dataset
> >
> > and
> >
> > created R-tree index on the polygon field of it, so the query will
> >
> > return
> >
> > the county effectively using the R-tree index)
> > 3) creates a new tweet record consisting of the original record's
> >
> > fields +
> >
> > the returned county value
> > 4) ingests to a tweet dataset in the AsterixDB instance.
> >
> > Can we have such an UDF?
> >
> > Best,
> > Young-Seok
> >
> >
> > On Thu, Oct 29, 2015 at 3:53 PM, Heri Ramampiaro <heriram@gmail.com>
> >
> > wrote:
> >
> >
> >
> > 2. Can we use AQL function in those kind of feed UDFs?
> >
> >
> > Can you give an example of what you are trying to do?
> > I.e do you want to run an AQL inside a UDF or use an AQL function as a
> > UDF connected to a running feed?
> >
> > -heri
> >
> >
> >
> >
> >
> > Best,
> >
> > Jianfeng Jia
> > PhD Candidate of Computer Science
> > University of California, Irvine
> >
> >
> >
> >
> > Best,
> >
> > Jianfeng Jia
> > PhD Candidate of Computer Science
> > University of California, Irvine
> >
> >
>



-- 
Raman

Mime
View raw message