Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 872C217F88 for ; Sat, 28 Feb 2015 09:21:10 +0000 (UTC) Received: (qmail 55209 invoked by uid 500); 28 Feb 2015 09:21:05 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 55147 invoked by uid 500); 28 Feb 2015 09:21:05 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 55136 invoked by uid 99); 28 Feb 2015 09:21:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 28 Feb 2015 09:21:05 +0000 Date: Sat, 28 Feb 2015 09:21:05 +0000 (UTC) From: "linzhao (JIRA)" To: dev@directory.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (DIRSERVER-2047) Some data can be lost when using ldapadd command to insert data into apacheds MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DIRSERVER-2047?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D= 14341437#comment-14341437 ]=20 linzhao commented on DIRSERVER-2047: ------------------------------------ Hi Emmanuel, I can reproduce the issue with your code: Below is the debug info for the issue: The array stack is 3 length long. The stack info list below: [<3,Node[r2693, nbElems:3, offset:10426368] -> {r2691|[nbC:0, nbD:0], 374]>|r2633|[nbC:0, nbD:0], 383]>|r2683|[nbC:0, nbD:0], 383]>|r2689}>, <10,Node[r26= 89, nbElems:11, offset:8612352] -> {r2266|[nbC:0, nbD:0], 465]>|r2561|[nbC:0, nbD:0], 465]>|r2501|[nbC:0, nbD:0], 465]>|r2421|[nbC:0, nbD:0], 465]>|r2621|[nbC:0, nbD:0], 465]>|r1886|[nbC:0, nbD:0], 465]>|r1686|[nbC:0, nbD:0], 465]>|r2566|[nbC:0, nbD:0], 465]>|r2616|[nbC:0, nbD:0], 465]>|r2651|[nbC:0, nbD:0], 468]>|r2206|[nbC:1, nbD:1], 236]>|r2686}>, <11,Leaf[r2206, nbElems:11= , offset:9030144] -> {[n= bC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 386],ValueHolder[Stri= ngSerializer, isRaw[44]]>, [nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 386],ValueHolder= [StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 371],ValueHolder[Stri= ngSerializer, isRaw[44]]>, [n= bC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 371],ValueHolder[StringSer= ializer, isRaw[44]]>, [n= bC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, [nbC:0, nbD:0], 345],ValueHolder[Str= ingSerializer, isRaw[44]]>, [nbC:0, nbD:0], 389],ValueHolder[StringSerializer, isRaw[44]]>}>, null= , null, null, null, null, null, null, null, null, null, null, null, null, n= ull, null, null, null, null, null, null, null, null, null, null, null, null= , null, null, null] Currently, the depth is 2. So, you can see the detail of the second Node in= the stack array. The second node's pos is 10 and nbElems is 11. So if you = change the code to {color:red}for ( int i =3D depth - 1; i >=3D 0; i-- ){co= lor}. The value isLast will be false, so the TupleCursor will not move to t= he last. So what is your thought? Is it correct? > Some data can be lost when using ldapadd command to insert data into apac= heds > -------------------------------------------------------------------------= ---- > > Key: DIRSERVER-2047 > URL: https://issues.apache.org/jira/browse/DIRSERVER-2047 > Project: Directory ApacheDS > Issue Type: Bug > Components: ldap > Affects Versions: 2.0.0-M19 > Environment: centOS 6.5 with openldap and apacheds installed > Reporter: linzhao > Priority: Blocker > Labels: LDAP > Fix For: 2.0.0-M20 > > Attachments: data.ldif, ordered-data.ldif, site.ldif, site_new.ld= if, site_simple.ldif, site_topology_schema.ldif, site_topology_schema_simpl= e.ldif, supercluster_partition.ldif, test.jpg > > > In our system, we need to do data backup and restore for apacheds. For no= w, we use the ldapsearch and ldapadd command to do BR function. We use ldap= search to backup apacheds data to be a ldif file and use ldapadd to restore= the data. But when the ldif is a little big, I always found that the data = can't be restored successfully, but the ladpadd command showed that the dat= a can be added successfully. No exceptions for ldapadd command. But the res= tored data didn't exist in the node. This bug only happened when the ldif f= ile is a little big. I mean if the data entry greater than 500 entries. But= the node for backup and restore is using mavibot partition. Because I foun= d so many problems for jdbm. So I change it to mavibot partition. Another q= uestion is that do you know is there some good way to do the data backup an= d restore for apacheds? > May be my mavibot partition has some problems, so someone can tell me how= to create a mavibot partition in APACHEDS? Because the mavibot parttion I = created can't be viewed when click the "Open Configuration" in apacheds stu= dio. Also I used unboundid JDK to insert many entries apacheds and the same= problem happened. So someone can tell me how to config the mavibot partiti= on on APACHEDS? That's may be very helpful. > Below is the exceptions from apacheds when problems happened: > [16:21:41] INFO [org.apache.directory.server.ldap.LdapServer] - Ldap serv= ice stopped. > [16:21:41] WARN [org.apache.directory.server.core.shared.partition.Defaul= tPartitionNexus] - Failed to flush partition data out. > org.apache.directory.api.ldap.model.exception.LdapOperationErrorException > at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePar= tition.saveContextCsn(AbstractBTreePartition.java:3364) > at org.apache.directory.server.core.shared.partition.DefaultPartitionNexu= s.sync(DefaultPartitionNexus.java:319) > at org.apache.directory.server.core.DefaultDirectoryService.shutdown(Defa= ultDirectoryService.java:1283) > at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:= 600) > at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheD= sTanukiWrapper.java:97) > at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3= 134) > Caused by: java.lang.NullPointerException > at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePar= tition.saveContextCsn(AbstractBTreePartition.java:3350) > ... 5 more > [16:21:41] ERROR [org.apache.directory.server.wrapper.ApacheDsTanukiWrapp= er] - Failed to stop the service. > org.apache.directory.api.util.exception.MultiException: ERR_265 Grouping = many exceptions on root nexus sync() > at org.apache.directory.server.core.shared.partition.DefaultPartitionNexu= s.sync(DefaultPartitionNexus.java:328) > at org.apache.directory.server.core.DefaultDirectoryService.shutdown(Defa= ultDirectoryService.java:1283) > at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:= 600) > at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheD= sTanukiWrapper.java:97) > at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3= 134) > Nested exceptions to follow: > INFO | jvm 1 | 2015/02/02 17:11:06 | [17:11:06] WARN [org.apache.director= y.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session t= o close: sending disconnect notice to client. > INFO | jvm 1 | 2015/02/02 17:11:06 | java.io.IOException: Connection rese= t by peer > INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.FileDispatcherImpl.rea= d0(Native Method) > INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.SocketDispatcher.read(= SocketDispatcher.java:39) > INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.IOUtil.readIntoNativeB= uffer(IOUtil.java:223) > INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.IOUtil.read(IOUtil.jav= a:197) > INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.SocketChannelImpl.read= (SocketChannelImpl.java:379) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.transport.socket.= nio.NioProcessor.read(NioProcessor.java:311) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.transport.socket.= nio.NioProcessor.read(NioProcessor.java:45) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.Abst= ractPollingIoProcessor.read(AbstractPollingIoProcessor.java:694) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.Abst= ractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.Abst= ractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.Abst= ractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.Abst= ractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1121) > INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.util.NamePreservi= ngRunnable.run(NamePreservingRunnable.java:64) > INFO | jvm 1 | 2015/02/02 17:11:06 | at java.util.concurrent.ThreadPoolEx= ecutor.runWorker(ThreadPoolExecutor.java:1142) > INFO | jvm 1 | 2015/02/02 17:11:06 | at java.util.concurrent.ThreadPoolEx= ecutor$Worker.run(ThreadPoolExecutor.java:617) > INFO | jvm 1 | 2015/02/02 17:11:06 | at java.lang.Thread.run(Thread.java:= 745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)