asterixdb-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pääkkönen Pekka <Pekka.Paakko...@vtt.fi>
Subject Re: Re: Socket feed questions
Date Wed, 21 Oct 2015 11:45:33 GMT
Hi,

Thanks, now I am able to create a socket feed, and save items to the dataset from the feed.

It seems that data items are written to the dataset after I close the socket at the client.
Is there some way to indicate to AsterixDB feed (with a newline or other indicator) that data
can be written to the database, when the connection is open?

After I close the socket at the client, the feed seems to close down. Or is it only paused,
until it is resumed?

-Pekka



Hi Pekka,
That's interesting, I'm not sure why the CC would appear as being down
to Managix. However if you can access the web console, it that
evidently isn't the case.
As for data ingestion via sockets, yes it is possible, but it kind of
depends on what's meant by sockets. There's no tutorial for it, but
take a look at SocketBasedFeedAdapter in the source, as well as
https://github.com/kisskys/incubator-asterixdb/blob/kisskys/indexonlyhilbertbtree/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/SocketTweetGenerator.java
for some examples of how it works.
Hope that helps!
Thanks,
-Ian


On Mon, Oct 19, 2015 at 10:15 PM, Pääkkönen Pekka

<Pekka.Paakkonen@vtt.fi> wrote:

> Hi Ian,

>

>

>

> Thanks for the reply.

>

> I compiled AsterixDB v0.87 and started it.

>

>

>

> However, I get the following warnings:

>

> INFO: Name:my_asterix

>

> Created:Mon Oct 19 08:37:16 UTC 2015

>

> Web-Url:http://192.168.101.144:19001

>

> State:UNUSABLE

>

>

>

> WARNING!:Cluster Controller not running at master

>

>

>

> Also, I see the following warnings in my_asterixdb1.log. there are no

> warnings or errors in cc.log

>

> "

>

> Oct 19, 2015 8:37:39 AM

> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager configure

>

> SEVERE: LifecycleComponentManager configured

> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@7559ec47

>

> ..

>

> INFO: Completed sharp checkpoint.

>

> Oct 19, 2015 8:37:40 AM org.apache.asterix.om.util.AsterixClusterProperties

> getIODevices

>

> WARNING: Configuration parameters for nodeId my_asterix_node1 not found. The

> node has not joined yet or has left.

>

> Oct 19, 2015 8:37:40 AM org.apache.asterix.om.util.AsterixClusterProperties

> getIODevices

>

> WARNING: Configuration parameters for nodeId my_asterix_node1 not found. The

> node has not joined yet or has left.

>

> Oct 19, 2015 8:38:38 AM

> org.apache.hyracks.control.common.dataset.ResultStateSweeper sweep

>

> INFO: Result state cleanup instance successfully completed."

>

>

>

> I seems that AsterixDB is running, and I can access it at port 19001.

>

>

>

> The documentation shows ingestion of tweets, but I would be interested in

> using sockets.

>

> Is it possible to ingest data from sockets?

>

>

>

> Regards,

>

> -Pekka

>

>

>

>

>

>

>

> Hey there Pekka,

>

> Your intuition is correct, most of the newer feeds features are in the

>

> current master branch and not in the (very) old 0.8.6 release. If you'd

>

> like to experiment with them you'll have to build from source. The details

>

> about that are here:

>

> https://asterixdb.incubator.apache.org/dev-setup.html#setting-up-an-asterix-development-environment-in-eclipse

>

> , but they're probably a bit overkill for just trying to get the compiled

>

> binaries. For that all you really need to do is :

>

> - Clone Hyracks from git

>

> - 'mvn clean install -DskipTests'

>

> - Clone AsterixDB

>

> - 'mvn clean package -DskipTests'

>

> Then, the binaries will sit in asterix-installer/target

>

>

>

>

>

> For an example, the documentation shows how to set up a feed that's

>

> ingesting Tweets:

>

> https://asterix-jenkins.ics.uci.edu/job/asterix-test-full/site/asterix-doc/feeds/tutorial.html

>

>

>

>

>

> Thanks,

>

> -Ian

>

>

>

>

>

> On Wed, Oct 7, 2015 at 9:48 PM, Pääkkönen Pekka <Pekka.Paakkonen@vtt.fi>

>

> wrote:

>

>

>

>> Hi,

>

>>

>

>>

>

>>

>

>> I would like to experiment with a socket-based feed.

>

>>

>

>> Can you point me to an example on how to utilize them?

>

>>

>

>> Do I need to install 0.8.7-snapshot version of AsterixDB in order to

>

>> experiment with feeds?

>

>>

>

>>

>

>>

>

>> Regards,

>

>>

>

>> -Pekka Pääkkönen

>

>>

>

>


Mime
View raw message