bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: User guide?
Date Thu, 17 Mar 2016 00:03:44 GMT
Hmm... that's plain nuts... Looks like latest ignite is pulling in some crap
like osgi... OMG, WTF? 

Thanks for discovering this David! Somehow it never got stepped upon in my
tests. Are you on centos7?

Cos

On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote:
> Hi,
> 
> Roman, Evans, thank you for the tips - that helped a lot. Still have some
> problems though, an error when running puppet apply ...
> 
> This is a sample of errors I got:
> 
> 
> *Debug: Executing '/bin/yum -d 0 -e 0 -y install
> ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum -d 0
> -e 0 -y install ignite-hadoop-service' returned 1: Error: Package:
> ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)           Requires:
> osgi(org.apache.servicemix.bundles.spring-beans)Error: Package:
> ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)           Requires:
> osgi(org.apache.servicemix.bundles.spring-context-support)Error: Package:
> ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)           Requires:
> osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package:
> ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)           Requires:
> osgi(org.apache.servicemix.bundles.spring-core)*
> .....
> 
> Also, after that:
> 
> 
> 
> 
> 
> 
> *Notice:
> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
> Dependency Package[ignite-hadoop] has failures: trueWarning:
> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
> Skipping because of failed dependenciesNotice:
> /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency
> Package[openjdk-7-jre-headless] has failures: trueWarning:
> /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping
> because of failed dependenciesNotice:
> /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]:
> Dependency Package[openjdk-7-jre-headless] has failures: trueWarning:
> /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: Skipping
> because of failed dependenciesNotice:
> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]:
> Dependency Package[ignite-hadoop] has failures: true*
> .....
> 
> 
> This is the "site.yaml" I used ("hadoop" wasn't listed in components, but I
> think it should be there?)
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs:  -
> /data/1hadoop_cluster_node::cluster_components:  - hadoop  - ignite_hadoop
> - hue  - mahout  - yarn  - zookeeperbigtop::jdk_package_name:
> "openjdk-7-jre-headless"bigtop::bigtop_repo_uri:
> "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64
> <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64>"*
> 
> 
> 
> 
> Any idea what I am doing wrong?
> 
> --David
> 
> On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <evansye@apache.org> wrote:
> 
> > David
> >
> > Most of the time you need to run same command on all nodes. "pdsh" would
> > be a good tool to use.
> >
> > 1. You need to have puppet agent installed on every node. yum -y install
> > puppet will do the trick, or you need to download repo, or use
> > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no deploy effort
> > other than just install the package.
> >
> > 2. Masterless means each puppet agent takes commands and runs
> > independently.  We DONT use puppet master, which is a centralized master
> > daemon to dispatch works to all the registered puppet agent.
> >
> > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera
> > configuration to all nodes, or at least make them available for all nodes
> > via nfs.
> >
> > 4. Finally, execute the puppet apply command on all nodes and the puppet
> > recipes will automatically figure out it's role. There no need to deploy
> > head node first and then slaves.
> >
> > Ask if you still have questions.
> >
> >
> > Thanks
> >
> > Evans
> > 2016年3月15日 下午4:10於 "David Starina" <david.starina@gmail.com>寫道:
> >
> >> Hi guys,
> >>
> >> me again. I haven't set up Puppet before, can you just point me in the
> >> right direction? I know this is not a Puppet mailing list, but I bet you
> >> have some tips for me :-)
> >>
> >> So what do I need to do? Install Puppet on every node? What does it mean
> >> that the mode of Puppet is "*masterless"* - do I just have to do a basic
> >> installation of Puppet and not care much about setting it up, and then
> >> deploy bigtop on every node separately? And when setting the Hadoop head
> >> node - does that mean that I need to deploy Bigtop on this node first, and
> >> Bigtop will know this is the master node by the url or IP I provide to it?
> >> And then deploy Bigtop via Puppet on all the other nodes and they will
> >> connect to the head node (which is the NameNode, I suppose?).
> >>
> >> I can do this via trial and error, but if you guys can help me reduce the
> >> "error" part, that will be great :) Sorry for such beginner questions, I
> >> haven't used Puppet before ... The reason I'm asking is, I got access to a
> >> "physical" (actually virtual, but it's the same) environment for a limited
> >> amount of time (two weeks), and I need to start testing on it asap - and
> >> since my day job takes a lot of my time, if I wanted to do this without
> >> help it could take me two weeks just to set up the environment - but I need
> >> to install it, which is not the central part of my project, and then run
> >> some ML algorithms on a text corpus, which will be the main part of the
> >> project ...
> >>
> >> --David
> >>
> >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina <david.starina@gmail.com>
> >> wrote:
> >>
> >>> Thanks for the info!
> >>>
> >>> --David
> >>>
> >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas <jayunit100.apache@gmail.com>
> >>> wrote:
> >>>
> >>>> I think there is enough info on youtube that you can cobble together
a
> >>>> syllabus if you need structured learning.
> >>>>
> >>>> See cos/Romans Bigtop talks which are on YouTube and then my Bigtop
> >>>> internals hackathon video.   That gives you both the high and low level
> >>>> details to do whatever you want.
> >>>>
> >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik <cos@apache.org>
> >>>> wrote:
> >>>> >
> >>>> > We also had this discussion to start producing "The Big Book of
Apache
> >>>> > Bigtop" like HBase does, but never got into the materialization
part
> >>>> of that
> >>>> > project.
> >>>> >
> >>>> > Cos
> >>>> >
> >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote:
> >>>> >> Hi David,
> >>>> >>
> >>>> >> If you just want to take Bigtop binaries to deploy a cluster,
you
> >>>> can refer
> >>>> >> to our puppet management recipes:
> >>>> >> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet
> >>>> >>
> >>>> >> BTW bigtop provisioner is the best source to reference for
how bigtop
> >>>> >> provision a cluster:
> >>>> >>
> >>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide
> >>>> >>
> >>>> >> Please ask us if you find anything confusing.
> >>>> >>
> >>>> >>
> >>>> >> Evans
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina <david.starina@gmail.com>:
> >>>> >>
> >>>> >>> Hi,
> >>>> >>>
> >>>> >>> Is there some version of Bigtop User Guide available? I
know it
> >>>> used to be
> >>>> >>> on the be on http://bigtop.apache.org/ under the "User
Guide"
> >>>> link, but
> >>>> >>> there's nothing there anymore ...
> >>>> >>>
> >>>> >>> I need some guide on how to install Bigtop on a physical
cluster
> >>>> ... It
> >>>> >>> was easy to create a virtual cluster on my machine using
Vagrant,
> >>>> but I
> >>>> >>> cannot find a manual to install on a physical cluster ...
> >>>> >>>
> >>>> >>> Cheers,
> >>>> >>> David
> >>>> >>>
> >>>>
> >>>
> >>>
> >>

Mime
View raw message