incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lefteris Soulas <lefterissou...@gmail.com>
Subject Re: Top_n_hashtags were not appearing. Correct way to stop cluster/adapter?
Date Mon, 12 Dec 2011 10:17:41 GMT
Hello,
I wanted to ask a question on the new version of S4 you mentioned.
When will it be released approximately? And also what are the new features
we should wait for?
My regards,
Lefteris

On Mon, Dec 12, 2011 at 11:36 AM, Matthieu Morel <mmorel@apache.org> wrote:

> Hi,
>
> I'm answering on the s4-user@incubator.apache.org mailing list, since
> this is the mailing list we are now using.
>
> On 12/9/11 9:05 PM, scotavoosh wrote:
>
>> While running the sample twittertopiccount app, I ran into the problem
>> that the top_n_hashtags file was not appearing. First issue was
>> configuring the app to pass through a proxy. That can be done by
>> adding these lines of code to the connectAndRead method in
>> TwitterFeedListener before opening the URLConnection:
>>
>> System.setProperty("https.**proxyHost","your.proxy.com");
>> System.setProperty("https.**proxyPort","8080");
>>
>
> I assume this was specific to your network configuration. But that will be
> useful to others with similar network configs, thanks!
>
>
>> The problem still persisted. The log files showed that events were
>> being dispatched but were not being consumed. Also in the log file was
>> a line that said:
>>
>> 2011-12-08 15:07:29,544 io.s4.comm.file.**StaticTaskManager INFO
>> (StaticTaskManager.java:185) Process taken up by another process
>> lockFile:/Users/user/**Development/s4/build/s4-image/**
>> scripts/../s4-core/
>> lock/s4s4node-0
>> 2011-12-08 15:07:29,544 io.s4.comm.file.**StaticTaskManager INFO
>> (StaticTaskManager.java:102) processAvailable:false
>>
>> I found the solution was to remove the lock files in /Users/user/
>> Development/s4/build/s4-image/**scripts/../s4-core/lock/**s4s4node-0
>>
>> These need to be removed after killing the running processes for the
>> cluster, client-adapter, and feed listener.
>>
>> What is the correct way to stop the cluster and adapter so that these
>> lock files are released?
>>
>
> There are currently 2 ways to control assignation of logical tasks to S4
> nodes:
> - through the file system, by creating "lock" files
> - through Zookeeper, by using ephemeral nodes
>
> S4 nodes delete owned lock files upon exit using the File.deleteOnExit
> method ( http://docs.oracle.com/javase/**6/docs/api/java/io/File.html#**
> deleteOnExit()<http://docs.oracle.com/javase/6/docs/api/java/io/File.html#deleteOnExit%28%29>).
Unfortunately, this won't work well if you kill the S4 node process:
> lock files will remain.
>
> Therefore you should either:
> - clean-up lock files at start-up through a script or an initialization
> routine
> - use the Zookeeper task assignation mechanism (
> http://docs.s4.io/tutorials/**zookeeper.html<http://docs.s4.io/tutorials/zookeeper.html>)
>
> By the way, future versions of S4 (from 0.5) will only use Zookeeper for
> task assignation.
>
>
> Hope this helps,
>
> Matthieu
>
>

Mime
View raw message