tephra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Poorna Chandra <poornachan...@gmail.com>
Subject Re: Tephra unreliable in CI environments
Date Tue, 11 Oct 2016 04:09:08 GMT
Hi Francis,

(Apology for the delayed response, I was out of office for the past few
weeks.)

Do you see a guava-13 jar in the classpath of Tephra? It should be in
<tephra-home>/lib directory. You can define environment variable CLASSPATH
pointing to path of guava-13 jar before starting Tephra. This will put
guava-13 jar earlier in Tephra's classpath.

Thanks,
Poorna.


On Tue, Sep 27, 2016 at 8:57 PM, F21 <f21.groups@gmail.com> wrote:

> Hi Poorna,
>
> Thanks, that narrows down the problem. I was spinning up a few VMs with
> various versions of Ubuntu and their kernels, but that didn't shed any
> light on the problem.
>
> You mentioned that adding the guava-13 jar to the classpath before the
> guava-12 jar would be a workaround. I am using Phoenix 4.8.1-rc0, so what
> would be the best way to do this?
>
> Cheers,
> Francis
>
> On 28/09/2016 1:43 PM, Poorna Chandra wrote:
>
>> Hi Francis,
>>
>> This is due to guava-12 vs guava-13 incompatibility [1]. Tephra depends on
>> guava-13 and HBase depends on guava-12. Depending on how the OS orders the
>> jars in the classpath, sometimes guava-12 may appear earlier in the
>> classpath. This leads to the NoSuchMethodError exception. We are planning
>> on removing Tephra's dependency on guava-13 in the next release. Until
>> then
>> a workaround is to add guava-13 jar into the classpath before guava-12
>> jar.
>>
>> Thanks,
>> Poorna.
>>
>> [1] - https://issues.apache.org/jira/browse/TEPHRA-181
>>
>>
>> On Tue, Sep 27, 2016 at 10:09 PM, F21 <f21.groups@gmail.com> wrote:
>>
>> Hi Poorna,
>>>
>>> That would be very helpful! Unfortunately, I ran into the same issue
>>> where
>>> the image no longer works correct on my dev environment, but works
>>> properly
>>> on travis.
>>>
>>> I am not receiving this error:
>>>
>>> java.lang.NoSuchMethodError:
>>> co.cask.tephra.TransactionManager.addListener(Lcom/google/co
>>> mmon/util/concurrent/Service$Listener;Ljava/util/concurrent/Executor;)V
>>>           at
>>> co.cask.tephra.distributed.TransactionService$1.leader(Trans
>>> actionService.java:83)
>>>           at
>>> org.apache.twill.internal.zookeeper.LeaderElection.becomeLea
>>> der(LeaderElection.java:229)
>>>           at
>>> org.apache.twill.internal.zookeeper.LeaderElection.access$
>>> 1800(LeaderElection.java:53)
>>>           at
>>> org.apache.twill.internal.zookeeper.LeaderElection$5.onSucce
>>> ss(LeaderElection.java:207)
>>>           at
>>> org.apache.twill.internal.zookeeper.LeaderElection$5.onSucce
>>> ss(LeaderElection.java:186)
>>>           at
>>> com.google.common.util.concurrent.Futures$5.run(Futures.java:768)
>>>           at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>>           at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>>           at java.lang.Thread.run(Thread.java:745)
>>>
>>> This is the exact same problem I encountered and posted on the list about
>>> previously: https://mail-archives.apache.org/mod_mbox/phoenix-user/20160
>>> 3.mbox/%3C56FC727E.30906@gmail.com%3E
>>>
>>> It's puzzling that an identical image will behave differently on
>>> different
>>> systems. Does tephra use any kernel functionalities directly?
>>>
>>> Cheers,
>>> Francis
>>>
>>>
>>> On 28/09/2016 12:04 PM, Poorna Chandra wrote:
>>>
>>> Hi Francis,
>>>>
>>>> Tephra startup script redirects all logs to a file by default. To help
>>>> debug such issues, you could update the Tephra startup script [1] to log
>>>> everything to stdout instead of a file when running inside docker. We
>>>> are
>>>> also planning on adding a --foreground option to Tephra startup script,
>>>> in
>>>> which case the logs will be written to stdout directly. This will help
>>>> in
>>>> debugging in future.
>>>>
>>>> Thanks,
>>>> Poorna.
>>>>
>>>> [1] - https://github.com/apache/incubator-tephra/blob/master/bin/
>>>> tephra#L175
>>>>
>>>>
>>>> On Tue, Sep 27, 2016 at 12:59 AM, F21 <f21.groups@gmail.com> wrote:
>>>>
>>>> I was able to get it to run reliably with Phoenix 4.8.1-rc0. Another
>>>> part
>>>>
>>>>> of the equation was forcing travis to use their Ubuntu 14.04
>>>>> environment
>>>>> rather than the default 12.04 environment. I am assuming 12.04 had an
>>>>> older
>>>>> kernel which prevent docker images from working correctly.
>>>>>
>>>>> On 27/09/2016 10:15 AM, F21 wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>>> I have created a docker image containing HBase 1.2.3 and Phoenix
>>>>>> 4.8.0.
>>>>>> See: https://github.com/Boostport/hbase-phoenix-all-in-one
>>>>>>
>>>>>> When running tests against the image on my machine, tephra works
>>>>>> perfectly.
>>>>>>
>>>>>> However, tephra seems to be unreliable in CI environments. It seems
>>>>>> that
>>>>>> the tx service is not discovered:
>>>>>>
>>>>>> RuntimeException: java.lang.Exception: Thrift error for
>>>>>> org.apache.tephra.distributed.TransactionServiceClient$2@3e9e291e:
>>>>>> Unable to discover tx service. -> Exception: Thrift error for
>>>>>> org.apache.tephra.distributed.TransactionServiceClient$2@3e9e291e:
>>>>>> Unable to discover tx service. -> TException: Unable to discover
tx
>>>>>> service.
>>>>>>
>>>>>> Here's a build on wercker which shows tephra failing:
>>>>>> https://app.wercker.com/boostport/avatica/runs/build/57e9b5d
>>>>>> 170a35501008402b4?step=57e9b5f72c15ad000127a534
>>>>>>
>>>>>> I also tried travis, but the same issue occurs:
>>>>>> https://travis-ci.org/Boostport/avatica/builds/162952367
>>>>>>
>>>>>> Since I am unable to ssh into those docker container on wercker or
>>>>>> travis, it is hard to debug what's causing tephra to fail. I am hoping
>>>>>> that
>>>>>> the issue is related to TEPHRA-179 (and a few other JIRAs related
to
>>>>>> it)
>>>>>> which I reported a few weeks ago and has since been fixed.
>>>>>>
>>>>>> Has anyone else ran into similar problems? I would love to hear your
>>>>>> thoughts.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Francis
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message