accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Accumulo 1.7.1 on Docker
Date Tue, 22 Mar 2016 18:41:31 GMT
Every time I use Docker, I seem to run into odd situations like this. 
Very frustrating.

I would love to know what's happening and get this working reliably. 
Having an "Apache-owned" docker image for Accumulo would be nice (and 
would help us internally as well as externally, IMO).

Andrew Hulbert wrote:
> +1 for Josh's suggestion.
>
> Not sure if its the same problem, but I too had to add some sleeps for
> single node accumulo startup scripts directly before and after init'ing
> accumulo. I dug in once and it seemed that the datanode needed more time
> to let other processes know the files in HDFS existed at /accumulo.
> Perhaps it was awaiting the exit from safe mode? Three seconds did it
> for me:
> https://github.com/jahhulbert-ccri/cloud-local/blob/master/bin/cloud-local.sh#L109
>
>
>
> On 03/14/2016 01:49 PM, Josh Elser wrote:
>> Well, I'm not sure wtf is going on, but I added some sleeps to your
>> init.sh script (30seconds before starting ZooKeeper and before
>> init'ing Accumulo) and now things worked just fine.
>>
>> I'm not seeing any Accumulo problems here, but rather the same Docker
>> absurdities I run into every time I try to use it :). I also just
>> upgrade to Docker 1.10.3 if that helps.
>>
>> Josh Elser wrote:
>>> So, I got it running, and ran into the same problem you described.
>>>
>>> The odd thing is that both the Accumulo Master and TabletServer seemed
>>> to be... stuck. They both claimed to be running, but they were both hung
>>> (unable to communicate). This ultimately caused the TabletServer to lose
>>> its ZooKeeper lock (and why the Shell didn't think it was running).
>>>
>>> I'm running into all sorts of docker issues ATM (trying to upgrade now,
>>> hoping it will help), but I'd recommend trying to increase the amount of
>>> heap for both the master and tserver from 128M to 256M in
>>> /accumulo-1.7.1/conf/accumulo-env.sh (the -Xmx value).
>>>
>>> Josh Elser wrote:
>>>> Ok! Thanks, I'll look for that init.sh script.
>>>>
>>>> We're not too... docker-savvy here yet :), but we can definitely help
>>>> out if we can find an automation script. Let me get back to you.
>>>>
>>>> Sven Hodapp wrote:
>>>>> Hi Josh,
>>>>>
>>>>> thanks for your answer.
>>>>> Currently I haven't uploaded the Dockerfile... If you want I'll upload
>>>>> it!
>>>>> (But currently it only is an debian with ssh, rsync, jdk7, the
>>>>> unzipped distributions of hadoop, zookeeper and accumulo, and the set
>>>>> of environment variables)
>>>>>
>>>>> Currently, and for testing, I start all things manually.
>>>>> The steps I do is documented in /init.sh within the docker container.
>>>>> I've enshured that dfs and zookeeper are up before starting accumulo.
>>>>>
>>>>> Regards,
>>>>> Sven
>>>>>
>>>>> ----- Ursprüngliche Mail -----
>>>>>> Von: "Josh Elser"<josh.elser@gmail.com>
>>>>>> An: "user"<user@accumulo.apache.org>
>>>>>> Gesendet: Montag, 14. März 2016 15:41:16
>>>>>> Betreff: Re: Accumulo 1.7.1 on Docker
>>>>>
>>>>>> Hi Sven,
>>>>>>
>>>>>> I can't seem to find the source for your Docker image (nor am smart
>>>>>> enough to figure out how to extract it). Any chance you can point
>>>>>> us to
>>>>>> that?
>>>>>>
>>>>>> That said, I did run your image, and it looks like you didn't
>>>>>> actually
>>>>>> start Hadoop, ZooKeeper, or Accumulo.
>>>>>>
>>>>>> Our documentation treats Hadoop and ZooKeeper primarily as
>>>>>> prerequisites, so check their docs for good instructions.
>>>>>>
>>>>>> For Accumulo -
>>>>>>
>>>>>> Configuration:
>>>>>> http://accumulo.apache.org/1.7/accumulo_user_manual.html#_installation
>>>>>>
>>>>>> Initialization:
>>>>>> http://accumulo.apache.org/1.7/accumulo_user_manual.html#_initialization
>>>>>>
>>>>>>
>>>>>> Starting:
>>>>>> http://accumulo.apache.org/1.7/accumulo_user_manual.html#_starting_accumulo
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Sven Hodapp wrote:
>>>>>>> Dear reader,
>>>>>>>
>>>>>>> I'd like to create a docker image with accumulo 1.7.1.
>>>>>>> I've install it from scratch with hadoop-2.7.2, zookeeper-3.4.8
and
>>>>>>> accumulo-1.7.1.
>>>>>>> I'm going though the installation and every time I'll end up
like
>>>>>>> that:
>>>>>>>
>>>>>>> root@deebd8e29683:/# accumulo shell -u root
>>>>>>> Password: ******
>>>>>>> 2016-03-13 13:18:31,627 [trace.DistributedTrace] INFO : SpanReceiver
>>>>>>> org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
>>>>>>> 2016-03-13 13:20:31,955 [impl.ServerClient] WARN : There are
no
>>>>>>> tablet servers:
>>>>>>> check that zookeeper and accumulo are running.
>>>>>>>
>>>>>>> Anybody got an idea whats wrong? I have no idea anymore...
>>>>>>>
>>>>>>> I'll currently share this docker image on docker hub.
>>>>>>> If you want to try it you can simply start:
>>>>>>>
>>>>>>> docker run -it themerius/accumulo /bin/bash
>>>>>>>
>>>>>>> All binaries and configs are in there. In ./init.sh are my setup
>>>>>>> steps.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Sven
>

Mime
View raw message