accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Fuchs <afu...@apache.org>
Subject Re: Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )
Date Thu, 29 Nov 2012 18:45:34 GMT
Sounds like you might need to introduce some synchronization to serialize
your JUnit tests. If you can recreate these symptoms in a small test case
that is representative of your code, maybe you can share that?

Adam



On Thu, Nov 29, 2012 at 11:10 AM, Josh Berk <josh.accumulo@gmail.com> wrote:

> Sorry Adam, I can't give my source code :/.
>
> and Eric, I'm positive that the timestamp is not the issue. When I said I
> use the same key, i meant only the key and not the entire same Entry.  The
> "timestamp" and "visibility" are associated with the entry and not the key.
> so, the timestamp is auto-updated. Because, if it weren't auto-updated then
> the problem i'm experiencing would occur for the same method outside of
> JUnit.
>
> I think the problem I'm experiencing has something to do with JUnit
> running it's test methods concurrently..  Each of my JUnit methods are
> independent from each other. So, no other method could influence the values
> of the objects that i'm storing/accessing with Accumulo in any particular
> method.
>
> -All JUnit methods are independent of each other.
> -The problem I'm experiencing is that when i update a value & retrieve it,
> I can intermittently receive "stale" data, which causes my test to fail.
> -The same method ran in a normal class (non-JUnit) succeeds every single
> time.
>
> I know that JUnit concurrently runs its test methods . I'm wondering if
> more than one thread maybe executes for a single test method and some parts
> are getting ahead of others in the same method. I won't say more about that
> theory in fear of being labeled a crazy person. I only know that the
> problem is JUnit-Accumulo specific & am wondering if anyone else has
> experienced the same issues?
>
> -Josh
>
>
>
>
>
>
> On Thu, Nov 29, 2012 at 10:51 AM, Eric Newton <eric.newton@gmail.com>wrote:
>
>> "I am definitely using the same key to update and retrieve the data."
>>
>> At least update the timestamp to the current time (or old timestamp + 1).
>>
>> -Eric
>>
>>
>> On Thu, Nov 29, 2012 at 10:38 AM, Adam Fuchs <afuchs@apache.org> wrote:
>>
>>> Josh,
>>>
>>> Can you share your junit test code so I can replicate this behavior?
>>>
>>> Adam
>>>
>>>
>>>
>>> On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk <josh.accumulo@gmail.com>wrote:
>>>
>>>> Good morning all,
>>>>
>>>> I'm experiencing some "weirdness" when executing JUnit tests for my
>>>> classes that operate with Accumulo. I can best describe it as latency.
>>>> Basically, when I write my object to Accumulo & then immediately
>>>> retrieve it to inspect the values, the values are not always updated to
>>>> what I just saved them as.
>>>>
>>>> Problem:
>>>>   part 1:
>>>>     - I create an object that has some primitive types.
>>>>     - I set the primitive variables to acceptable values.
>>>>     - I serialize the object (the "Value")
>>>>     - I write the Value to Accumulo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are equal to what they were set to.
>>>>
>>>>   part 2:
>>>>     - I retrieve the object from Accumulo
>>>>     - I set the primitive variables to different values
>>>>     - I serialize the object
>>>>     - I write the Value to Accumlo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are *not equal* to what they were just set to
>>>>
>>>> This only seems to be happening during the JUnit.
>>>>
>>>> I have a method that performs the above task, in a JUnit test, and when
>>>> I repeatedly run the JUnit test, it will intermittently fail.
>>>> I have the same exact method, but it is in a regular class, and I can
>>>> run it as much as I want, with no failure.
>>>>
>>>> for the non-JUnit test, MockInstances and  "real" instances succeed
>>>> every time
>>>> for the JUnit test, MockInstances and "real" instances both fail
>>>> intermittently.
>>>>
>>>> sidenotes:
>>>> - I am definitely using the same key to update and retrieve the data. I
>>>> also inspected the entries that I was writing to Accumulo, every time, and
>>>> can confirm that they are being "sent"/"written" to Accumulo as I intend
>>>> them to be. In summary, I am positive that I am sending the correct data
to
>>>> be written. This is doubly verified by my ability to intermittently succeed
>>>> when JUnit and 100% succeed in a "normal" class.
>>>>
>>>> Any assistance would be greatly appreciated.
>>>>
>>>> Best Regards,
>>>>
>>>> Josh
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message