hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: Zookeeper stops
Date Thu, 19 Aug 2010 18:24:01 GMT
+1 on that Ted. I frequently see this issue crop up as "I just rebooted 
my server and lost all my data ..." -- many os's will cleanup tmp on 
reboot. :-)

Patrick

On 08/19/2010 07:43 AM, Ted Dunning wrote:
> Also, /tmp is not a great place to keep things that are intended for
> persistence.
>
> On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar<mahadev@yahoo-inc.com>wrote:
>
>> Hi Wim,
>>   It mostly looks like that zookeeper is not able to create files on the
>> /tmp filesystem. Is there is a space shortage or is it possible the file is
>> being deleted as its being written to?
>>
>> Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem.
>>
>> Thanks
>> mahadev
>>
>>
>> On 8/19/10 1:15 AM, "Wim Jongman"<wim.jongman@gmail.com>  wrote:
>>
>> Hi,
>>
>> I have a zookeeper server running that can sometimes run for days and then
>> quits:
>>
>> Is there somebody with a clue to the problem?
>>
>> I am running 64 bit Ubuntu with
>>
>> java version "1.6.0_18"
>> OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
>> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
>>
>> Zookeeper 3.3.0
>>
>> The log below has some context before it shows the fatal error. Our
>> component.id=40676 indicates that it is the 40676th time that I ask ZK to
>> publish this information. It has been seen to go up to half a million
>> before
>> stopping.
>>
>> Regards,
>>
>> Wim
>>
>> ZooDiscovery>  Service Unpublished: Aug 18, 2010 11:17:28 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@68a1e081,
>> component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server,
>> component.id=40676,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5b9a6ad1
>> }]]
>> ZooDiscovery>  Service Published: Aug 18, 2010 11:17:29 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>> component.id=40677,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>> }]]
>> [log;+0200 2010.08.18
>>
>> 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No
>> async remote service interface found with
>> name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service
>>
>> class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]]
>> 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:ZooKeeperServer@262] -
>> Severe unrecoverable error, exiting
>> java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e
>> (No such file or directory)
>>         at java.io.FileOutputStream.open(Native Method)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:209)
>>         at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
>>         at
>>
>> org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224)
>>         at
>>
>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211)
>>         at
>>
>> org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260)
>>         at
>>
>> org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120)
>> ZooDiscovery>  Service Unpublished: Aug 18, 2010 11:17:37 PM.
>> ServiceInfo[uri=osgiservices://
>>
>> 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._iana@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice
>> ,
>>
>> osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService,
>> ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
>> =org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@71bfa0a4,
>> component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server,
>> component.id=40677,
>>
>> ecf.sp.cid=org.eclipse.ecf.discovery.ServiceProperties$ByteArrayWrapper@5bcba953
>> }]]
>>
>>
>

Mime
View raw message