bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Starina <david.star...@gmail.com>
Subject Re: User guide?
Date Thu, 17 Mar 2016 18:52:42 GMT
Some more problems ... Tried this on CentOS 6.6 now ...

1) this line in site.yaml: "bigtop::jdk_package_name:
"openjdk-7-jre-headless"" - not OK, you get the error:



*Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list
openjdk-7-jre-headless' returned 1: Error: No matching Packages to
listError: /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure: change
from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list
openjdk-7-jre-headless' returned 1: Error: No matching Packages to list*
2) so, I installed OpenJDK manually ... Got the following error:






*Error: Could not start Service[ignite-hadoop-service]: Execution of
'/sbin/service ignite-hadoop-service start' returned 1:
ignite-hadoop-service: unrecognized serviceError:
/Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/Service[ignite-hadoop-service]/ensure:
change from stopped to running failed: Could not start
Service[ignite-hadoop-service]: Execution of '/sbin/service
ignite-hadoop-service start' returned 1: ignite-hadoop-service:
unrecognized serviceDebug: Executing '/bin/rpm -q jdk --nosignature
--nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE}
%{ARCH}\n'Debug: Executing '/usr/bin/yum -d 0 -e 0 -y list jdk'Error:
Execution of '/usr/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No
matching Packages to listError: /Stage[main]/Main/Package[jdk]/ensure:
change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0
-y list jdk' returned 1: Error: No matching Packages to listNotice:
/Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency
Package[jdk] has failures: true*
.....


So, '*/usr/bin/yum -d 0 -e 0 -y list jdk*' doesn't return anything ... this
is how Java is listed when I run 'yum list installed':
java-1.8.0-openjdk.x86_64
java-1.8.0-openjdk-headless.x86_64


So, still can't install Ignite for now ...

--David

On Thu, Mar 17, 2016 at 12:46 PM, David Starina <david.starina@gmail.com>
wrote:

> Konstantin,
>
> So I should be safer using CentOS 6?
>
> Also - was I right to include "hadoop" in site.yaml components? It's not
> listed in the basic site.yaml, but I am guessing it is not installed
> automatically?
>
> -David
>
> On Thu, Mar 17, 2016 at 7:17 AM, David Starina <david.starina@gmail.com>
> wrote:
>
>> That's right, I'm using CentOS 7.
>>
>> --David
>>
>>
>> On Thursday, 17 March 2016, Konstantin Boudnik <cos@apache.org> wrote:
>>
>>> Hmm... that's plain nuts... Looks like latest ignite is pulling in some
>>> crap
>>> like osgi... OMG, WTF?
>>>
>>> Thanks for discovering this David! Somehow it never got stepped upon in
>>> my
>>> tests. Are you on centos7?
>>>
>>> Cos
>>>
>>> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote:
>>> > Hi,
>>> >
>>> > Roman, Evans, thank you for the tips - that helped a lot. Still have
>>> some
>>> > problems though, an error when running puppet apply ...
>>> >
>>> > This is a sample of errors I got:
>>> >
>>> >
>>> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install
>>> > ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum
>>> -d 0
>>> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package:
>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>  Requires:
>>> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package:
>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>  Requires:
>>> > osgi(org.apache.servicemix.bundles.spring-context-support)Error:
>>> Package:
>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>  Requires:
>>> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package:
>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>  Requires:
>>> > osgi(org.apache.servicemix.bundles.spring-core)*
>>> > .....
>>> >
>>> > Also, after that:
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > *Notice:
>>> >
>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
>>> > Dependency Package[ignite-hadoop] has failures: trueWarning:
>>> >
>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
>>> > Skipping because of failed dependenciesNotice:
>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency
>>> > Package[openjdk-7-jre-headless] has failures: trueWarning:
>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping
>>> > because of failed dependenciesNotice:
>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]:
>>> > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning:
>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]:
>>> Skipping
>>> > because of failed dependenciesNotice:
>>> >
>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]:
>>> > Dependency Package[ignite-hadoop] has failures: true*
>>> > .....
>>> >
>>> >
>>> > This is the "site.yaml" I used ("hadoop" wasn't listed in components,
>>> but I
>>> > think it should be there?)
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs:  -
>>> > /data/1hadoop_cluster_node::cluster_components:  - hadoop  -
>>> ignite_hadoop
>>> > - hue  - mahout  - yarn  - zookeeperbigtop::jdk_package_name:
>>> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri:
>>> > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64
>>> > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64
>>> >"*
>>> >
>>> >
>>> >
>>> >
>>> > Any idea what I am doing wrong?
>>> >
>>> > --David
>>> >
>>> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <evansye@apache.org> wrote:
>>> >
>>> > > David
>>> > >
>>> > > Most of the time you need to run same command on all nodes. "pdsh"
>>> would
>>> > > be a good tool to use.
>>> > >
>>> > > 1. You need to have puppet agent installed on every node. yum -y
>>> install
>>> > > puppet will do the trick, or you need to download repo, or use
>>> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no deploy
>>> effort
>>> > > other than just install the package.
>>> > >
>>> > > 2. Masterless means each puppet agent takes commands and runs
>>> > > independently.  We DONT use puppet master, which is a centralized
>>> master
>>> > > daemon to dispatch works to all the registered puppet agent.
>>> > >
>>> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera
>>> > > configuration to all nodes, or at least make them available for all
>>> nodes
>>> > > via nfs.
>>> > >
>>> > > 4. Finally, execute the puppet apply command on all nodes and the
>>> puppet
>>> > > recipes will automatically figure out it's role. There no need to
>>> deploy
>>> > > head node first and then slaves.
>>> > >
>>> > > Ask if you still have questions.
>>> > >
>>> > >
>>> > > Thanks
>>> > >
>>> > > Evans
>>> > > 2016年3月15日 下午4:10於 "David Starina" <david.starina@gmail.com>寫道:
>>> > >
>>> > >> Hi guys,
>>> > >>
>>> > >> me again. I haven't set up Puppet before, can you just point me
in
>>> the
>>> > >> right direction? I know this is not a Puppet mailing list, but
I
>>> bet you
>>> > >> have some tips for me :-)
>>> > >>
>>> > >> So what do I need to do? Install Puppet on every node? What does
it
>>> mean
>>> > >> that the mode of Puppet is "*masterless"* - do I just have to do
a
>>> basic
>>> > >> installation of Puppet and not care much about setting it up, and
>>> then
>>> > >> deploy bigtop on every node separately? And when setting the Hadoop
>>> head
>>> > >> node - does that mean that I need to deploy Bigtop on this node
>>> first, and
>>> > >> Bigtop will know this is the master node by the url or IP I provide
>>> to it?
>>> > >> And then deploy Bigtop via Puppet on all the other nodes and they
>>> will
>>> > >> connect to the head node (which is the NameNode, I suppose?).
>>> > >>
>>> > >> I can do this via trial and error, but if you guys can help me
>>> reduce the
>>> > >> "error" part, that will be great :) Sorry for such beginner
>>> questions, I
>>> > >> haven't used Puppet before ... The reason I'm asking is, I got
>>> access to a
>>> > >> "physical" (actually virtual, but it's the same) environment for
a
>>> limited
>>> > >> amount of time (two weeks), and I need to start testing on it asap
>>> - and
>>> > >> since my day job takes a lot of my time, if I wanted to do this
>>> without
>>> > >> help it could take me two weeks just to set up the environment
-
>>> but I need
>>> > >> to install it, which is not the central part of my project, and
>>> then run
>>> > >> some ML algorithms on a text corpus, which will be the main part
of
>>> the
>>> > >> project ...
>>> > >>
>>> > >> --David
>>> > >>
>>> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina <
>>> david.starina@gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >>> Thanks for the info!
>>> > >>>
>>> > >>> --David
>>> > >>>
>>> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas <
>>> jayunit100.apache@gmail.com>
>>> > >>> wrote:
>>> > >>>
>>> > >>>> I think there is enough info on youtube that you can cobble
>>> together a
>>> > >>>> syllabus if you need structured learning.
>>> > >>>>
>>> > >>>> See cos/Romans Bigtop talks which are on YouTube and then
my
>>> Bigtop
>>> > >>>> internals hackathon video.   That gives you both the high
and low
>>> level
>>> > >>>> details to do whatever you want.
>>> > >>>>
>>> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik <cos@apache.org>
>>> > >>>> wrote:
>>> > >>>> >
>>> > >>>> > We also had this discussion to start producing "The
Big Book of
>>> Apache
>>> > >>>> > Bigtop" like HBase does, but never got into the materialization
>>> part
>>> > >>>> of that
>>> > >>>> > project.
>>> > >>>> >
>>> > >>>> > Cos
>>> > >>>> >
>>> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote:
>>> > >>>> >> Hi David,
>>> > >>>> >>
>>> > >>>> >> If you just want to take Bigtop binaries to deploy
a cluster,
>>> you
>>> > >>>> can refer
>>> > >>>> >> to our puppet management recipes:
>>> > >>>> >>
>>> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet
>>> > >>>> >>
>>> > >>>> >> BTW bigtop provisioner is the best source to reference
for how
>>> bigtop
>>> > >>>> >> provision a cluster:
>>> > >>>> >>
>>> > >>>>
>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide
>>> > >>>> >>
>>> > >>>> >> Please ask us if you find anything confusing.
>>> > >>>> >>
>>> > >>>> >>
>>> > >>>> >> Evans
>>> > >>>> >>
>>> > >>>> >>
>>> > >>>> >>
>>> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina <
>>> david.starina@gmail.com>:
>>> > >>>> >>
>>> > >>>> >>> Hi,
>>> > >>>> >>>
>>> > >>>> >>> Is there some version of Bigtop User Guide
available? I know
>>> it
>>> > >>>> used to be
>>> > >>>> >>> on the be on http://bigtop.apache.org/ under
the "User Guide"
>>> > >>>> link, but
>>> > >>>> >>> there's nothing there anymore ...
>>> > >>>> >>>
>>> > >>>> >>> I need some guide on how to install Bigtop
on a physical
>>> cluster
>>> > >>>> ... It
>>> > >>>> >>> was easy to create a virtual cluster on my
machine using
>>> Vagrant,
>>> > >>>> but I
>>> > >>>> >>> cannot find a manual to install on a physical
cluster ...
>>> > >>>> >>>
>>> > >>>> >>> Cheers,
>>> > >>>> >>> David
>>> > >>>> >>>
>>> > >>>>
>>> > >>>
>>> > >>>
>>> > >>
>>>
>>
>

Mime
View raw message