Return-Path: X-Original-To: apmail-accumulo-user-archive@www.apache.org Delivered-To: apmail-accumulo-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 937F39974 for ; Thu, 29 Nov 2012 16:11:37 +0000 (UTC) Received: (qmail 14936 invoked by uid 500); 29 Nov 2012 16:11:37 -0000 Delivered-To: apmail-accumulo-user-archive@accumulo.apache.org Received: (qmail 14770 invoked by uid 500); 29 Nov 2012 16:11:36 -0000 Mailing-List: contact user-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@accumulo.apache.org Delivered-To: mailing list user@accumulo.apache.org Received: (qmail 14677 invoked by uid 99); 29 Nov 2012 16:11:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Nov 2012 16:11:34 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of josh.accumulo@gmail.com designates 209.85.213.193 as permitted sender) Received: from [209.85.213.193] (HELO mail-ye0-f193.google.com) (209.85.213.193) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Nov 2012 16:11:02 +0000 Received: by mail-ye0-f193.google.com with SMTP id q13so679377yen.0 for ; Thu, 29 Nov 2012 08:10:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=AKX+ivzk+/7fZlWY2dSRQHCUkhMAJ+lmlRkleh4rgK4=; b=0ONqNKR+dMXC+BTAN0U0hgXR2eUWTU4QOBDts6hQrBsX5314PtammqVi6RCYSVuxxi LNuP3wWa2muqN0DiFTK0C5VbuLkFCaf5XgVMXJh3N8aD5UXICga/XjF4XCGDEGp8MmKS yJOnEPg3EFOf+ECG7j4843nzuNHIyfAvtZ+/7WOODrLvKK2SoTPm+lTDeKXduFl5/fN4 OAtU9QdKtUOsxSLV11bwvrixhR6aSXand6LUOVelY6yf+VrQAz1sxapOG5E4wf7SBqbW JUtvzTCJh2YFOHfAtNPwVWBlxU+fvC4rDs9Vvu8c9ucVEnfPTFsM8G+A2M+J4m9Zg2S0 JiyQ== MIME-Version: 1.0 Received: by 10.101.3.17 with SMTP id f17mr7023297ani.87.1354205439143; Thu, 29 Nov 2012 08:10:39 -0800 (PST) Received: by 10.146.160.9 with HTTP; Thu, 29 Nov 2012 08:10:39 -0800 (PST) In-Reply-To: References: Date: Thu, 29 Nov 2012 11:10:39 -0500 Message-ID: Subject: Re: Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 ) From: Josh Berk To: user@accumulo.apache.org Content-Type: multipart/alternative; boundary=001636c5952b27365604cfa48659 X-Virus-Checked: Checked by ClamAV on apache.org --001636c5952b27365604cfa48659 Content-Type: text/plain; charset=ISO-8859-1 Sorry Adam, I can't give my source code :/. and Eric, I'm positive that the timestamp is not the issue. When I said I use the same key, i meant only the key and not the entire same Entry. The "timestamp" and "visibility" are associated with the entry and not the key. so, the timestamp is auto-updated. Because, if it weren't auto-updated then the problem i'm experiencing would occur for the same method outside of JUnit. I think the problem I'm experiencing has something to do with JUnit running it's test methods concurrently.. Each of my JUnit methods are independent from each other. So, no other method could influence the values of the objects that i'm storing/accessing with Accumulo in any particular method. -All JUnit methods are independent of each other. -The problem I'm experiencing is that when i update a value & retrieve it, I can intermittently receive "stale" data, which causes my test to fail. -The same method ran in a normal class (non-JUnit) succeeds every single time. I know that JUnit concurrently runs its test methods . I'm wondering if more than one thread maybe executes for a single test method and some parts are getting ahead of others in the same method. I won't say more about that theory in fear of being labeled a crazy person. I only know that the problem is JUnit-Accumulo specific & am wondering if anyone else has experienced the same issues? -Josh On Thu, Nov 29, 2012 at 10:51 AM, Eric Newton wrote: > "I am definitely using the same key to update and retrieve the data." > > At least update the timestamp to the current time (or old timestamp + 1). > > -Eric > > > On Thu, Nov 29, 2012 at 10:38 AM, Adam Fuchs wrote: > >> Josh, >> >> Can you share your junit test code so I can replicate this behavior? >> >> Adam >> >> >> >> On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk wrote: >> >>> Good morning all, >>> >>> I'm experiencing some "weirdness" when executing JUnit tests for my >>> classes that operate with Accumulo. I can best describe it as latency. >>> Basically, when I write my object to Accumulo & then immediately >>> retrieve it to inspect the values, the values are not always updated to >>> what I just saved them as. >>> >>> Problem: >>> part 1: >>> - I create an object that has some primitive types. >>> - I set the primitive variables to acceptable values. >>> - I serialize the object (the "Value") >>> - I write the Value to Accumulo ( Entry ) >>> - I retrieve the Object from Accumulo & inspect. The primitive >>> values are equal to what they were set to. >>> >>> part 2: >>> - I retrieve the object from Accumulo >>> - I set the primitive variables to different values >>> - I serialize the object >>> - I write the Value to Accumlo ( Entry ) >>> - I retrieve the Object from Accumulo & inspect. The primitive >>> values are *not equal* to what they were just set to >>> >>> This only seems to be happening during the JUnit. >>> >>> I have a method that performs the above task, in a JUnit test, and when >>> I repeatedly run the JUnit test, it will intermittently fail. >>> I have the same exact method, but it is in a regular class, and I can >>> run it as much as I want, with no failure. >>> >>> for the non-JUnit test, MockInstances and "real" instances succeed >>> every time >>> for the JUnit test, MockInstances and "real" instances both fail >>> intermittently. >>> >>> sidenotes: >>> - I am definitely using the same key to update and retrieve the data. I >>> also inspected the entries that I was writing to Accumulo, every time, and >>> can confirm that they are being "sent"/"written" to Accumulo as I intend >>> them to be. In summary, I am positive that I am sending the correct data to >>> be written. This is doubly verified by my ability to intermittently succeed >>> when JUnit and 100% succeed in a "normal" class. >>> >>> Any assistance would be greatly appreciated. >>> >>> Best Regards, >>> >>> Josh >>> >>> >>> >>> >>> >>> >>> >>> >>> >> > --001636c5952b27365604cfa48659 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sorry Adam, I can't give my source code :/.=A0

and E= ric, I'm positive that the timestamp is not the issue. When I said I us= e the same key, i meant only the key and not the entire same Entry. =A0The = "timestamp" and "visibility" are associated with the en= try and not the key. so, the timestamp is auto-updated. Because, if it were= n't auto-updated then the problem i'm experiencing would occur for = the same method outside of JUnit.

I think the problem I'm experiencing has something = to do with JUnit running it's test methods concurrently.. =A0Each of my= JUnit methods are independent from each other. So, no other method could i= nfluence the values of the objects that i'm storing/accessing with Accu= mulo in any particular method.

-All JUnit methods are independent of each other.
=
-The problem I'm experiencing is that when i update a value & = retrieve it, I can intermittently receive "stale" data, which cau= ses my test to fail.
-The same method ran in a normal class (non-JUnit) succeeds every sing= le time.=A0

I know that JUnit concurrently runs it= s test methods . I'm wondering if more than one thread maybe executes f= or a single test method and some parts are getting ahead of others in the s= ame method. I won't say more about that theory in fear of being labeled= a crazy person. I only know that the problem is JUnit-Accumulo specific &a= mp; am wondering if anyone else has experienced the same issues?

-Josh






On Thu, Nov 29, 201= 2 at 10:51 AM, Eric Newton <eric.newton@gmail.com> wrote= :
"I am definitely usin= g the same key to update and retrieve the data."

<= div> At least update the timestamp to the current time (or old timestamp + 1).

-Eri= c


On Thu, Nov 29, 2012 at 10:38 AM, Adam F= uchs <afuchs@apache.org> wrote:
Josh,

Can you share your junit test code so I can replic= ate this behavior?

A= dam



On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk <josh.accumulo@gmail.c= om> wrote:
Good morning all,

I'm= experiencing some "weirdness" when executing JUnit tests for my = classes that operate with Accumulo. I can best describe it as latency.
Basically, when I write my object to Accumulo & then immediately r= etrieve it to inspect the values, the values are not always updated to what= I just saved them as.

Problem:
=A0 part 1:
=A0 =A0 - I cr= eate an object that has some primitive types.
=A0 =A0 - I set the= primitive variables to acceptable values.
=A0 =A0 - I serialize = the object (the "Value")
=A0 =A0 - I write the Value to Accumulo ( Entry )
=A0 =A0 - = I retrieve the Object from Accumulo & inspect. The primitive values are= equal to what they were set to.

=A0 part 2:
=
=A0 =A0 - I retrieve the object from Accumulo
=A0 =A0 - I set the primitive variables to different values
= =A0 =A0 - I serialize the object
=A0 =A0 - I write the Value to A= ccumlo ( Entry )=A0
=A0 =A0 - I retrieve the Object from Accumulo= & inspect. The primitive values are not equal=A0to what they we= re just set to

This only seems to be happening during the JUnit.=A0

I have a method that performs the above task, in a J= Unit test, and when I repeatedly run the JUnit test, it will intermittently= fail.
I have the same exact method, but it is in a regular class, and I can = run it as much as I want, with no failure.=A0

for = the non-JUnit test, MockInstances and =A0"real" instances succeed= every time
for the JUnit test, MockInstances and "real" instances both = fail intermittently.

sidenotes:
- I am d= efinitely using the same key to update and retrieve the data. I also inspec= ted the entries that I was writing to Accumulo, every time, and can confirm= that they are being "sent"/"written" to Accumulo as I = intend them to be. In summary, I am positive that I am sending the correct = data to be written. This is doubly verified by my ability to intermittently= succeed when JUnit and 100% succeed in a "normal" class.

Any assistance would be greatly appreciated.
=
Best Regards,

Josh=A0

=A0 =A0


=





--001636c5952b27365604cfa48659--