bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Starina <david.star...@gmail.com>
Subject Re: User guide?
Date Sat, 19 Mar 2016 13:11:31 GMT
Guys, what am I (still) doing wrong?  I keep getting this error:
Error: Execution of '/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No
matching Packages to list

This probably means I am not installing Java in a way Bigtop is expecting
it. Do you have some idea what I should be doing differently? Of course,
installing Oracle JDK manually won't work, because it won't be listed in
yum. On the other hand, when I install OpenJDK via yum, "list jdk" doesn't
list any package either.

What am I missing? Should I be adding some other repo to yum beforehand?

--David

On Thu, Mar 17, 2016 at 7:52 PM, David Starina <david.starina@gmail.com>
wrote:

> Some more problems ... Tried this on CentOS 6.6 now ...
>
> 1) this line in site.yaml: "bigtop::jdk_package_name:
> "openjdk-7-jre-headless"" - not OK, you get the error:
>
>
>
> *Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list
> openjdk-7-jre-headless' returned 1: Error: No matching Packages to
> listError: /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure: change
> from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list
> openjdk-7-jre-headless' returned 1: Error: No matching Packages to list*
> 2) so, I installed OpenJDK manually ... Got the following error:
>
>
>
>
>
>
> *Error: Could not start Service[ignite-hadoop-service]: Execution of
> '/sbin/service ignite-hadoop-service start' returned 1:
> ignite-hadoop-service: unrecognized serviceError:
> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/Service[ignite-hadoop-service]/ensure:
> change from stopped to running failed: Could not start
> Service[ignite-hadoop-service]: Execution of '/sbin/service
> ignite-hadoop-service start' returned 1: ignite-hadoop-service:
> unrecognized serviceDebug: Executing '/bin/rpm -q jdk --nosignature
> --nodigest --qf %{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE}
> %{ARCH}\n'Debug: Executing '/usr/bin/yum -d 0 -e 0 -y list jdk'Error:
> Execution of '/usr/bin/yum -d 0 -e 0 -y list jdk' returned 1: Error: No
> matching Packages to listError: /Stage[main]/Main/Package[jdk]/ensure:
> change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0
> -y list jdk' returned 1: Error: No matching Packages to listNotice:
> /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency
> Package[jdk] has failures: true*
> .....
>
>
> So, '*/usr/bin/yum -d 0 -e 0 -y list jdk*' doesn't return anything ...
> this is how Java is listed when I run 'yum list installed':
> java-1.8.0-openjdk.x86_64
> java-1.8.0-openjdk-headless.x86_64
>
>
> So, still can't install Ignite for now ...
>
> --David
>
> On Thu, Mar 17, 2016 at 12:46 PM, David Starina <david.starina@gmail.com>
> wrote:
>
>> Konstantin,
>>
>> So I should be safer using CentOS 6?
>>
>> Also - was I right to include "hadoop" in site.yaml components? It's not
>> listed in the basic site.yaml, but I am guessing it is not installed
>> automatically?
>>
>> -David
>>
>> On Thu, Mar 17, 2016 at 7:17 AM, David Starina <david.starina@gmail.com>
>> wrote:
>>
>>> That's right, I'm using CentOS 7.
>>>
>>> --David
>>>
>>>
>>> On Thursday, 17 March 2016, Konstantin Boudnik <cos@apache.org> wrote:
>>>
>>>> Hmm... that's plain nuts... Looks like latest ignite is pulling in some
>>>> crap
>>>> like osgi... OMG, WTF?
>>>>
>>>> Thanks for discovering this David! Somehow it never got stepped upon in
>>>> my
>>>> tests. Are you on centos7?
>>>>
>>>> Cos
>>>>
>>>> On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote:
>>>> > Hi,
>>>> >
>>>> > Roman, Evans, thank you for the tips - that helped a lot. Still have
>>>> some
>>>> > problems though, an error when running puppet apply ...
>>>> >
>>>> > This is a sample of errors I got:
>>>> >
>>>> >
>>>> > *Debug: Executing '/bin/yum -d 0 -e 0 -y install
>>>> > ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum
>>>> -d 0
>>>> > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package:
>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>>  Requires:
>>>> > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package:
>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>>  Requires:
>>>> > osgi(org.apache.servicemix.bundles.spring-context-support)Error:
>>>> Package:
>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>>  Requires:
>>>> > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package:
>>>> > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop)
>>>>  Requires:
>>>> > osgi(org.apache.servicemix.bundles.spring-core)*
>>>> > .....
>>>> >
>>>> > Also, after that:
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > *Notice:
>>>> >
>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
>>>> > Dependency Package[ignite-hadoop] has failures: trueWarning:
>>>> >
>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]:
>>>> > Skipping because of failed dependenciesNotice:
>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]:
>>>> Dependency
>>>> > Package[openjdk-7-jre-headless] has failures: trueWarning:
>>>> > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping
>>>> > because of failed dependenciesNotice:
>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]:
>>>> > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning:
>>>> > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]:
>>>> Skipping
>>>> > because of failed dependenciesNotice:
>>>> >
>>>> /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]:
>>>> > Dependency Package[ignite-hadoop] has failures: true*
>>>> > .....
>>>> >
>>>> >
>>>> > This is the "site.yaml" I used ("hadoop" wasn't listed in components,
>>>> but I
>>>> > think it should be there?)
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs:  -
>>>> > /data/1hadoop_cluster_node::cluster_components:  - hadoop  -
>>>> ignite_hadoop
>>>> > - hue  - mahout  - yarn  - zookeeperbigtop::jdk_package_name:
>>>> > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri:
>>>> > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64
>>>> > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64
>>>> >"*
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Any idea what I am doing wrong?
>>>> >
>>>> > --David
>>>> >
>>>> > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <evansye@apache.org>
wrote:
>>>> >
>>>> > > David
>>>> > >
>>>> > > Most of the time you need to run same command on all nodes. "pdsh"
>>>> would
>>>> > > be a good tool to use.
>>>> > >
>>>> > > 1. You need to have puppet agent installed on every node. yum -y
>>>> install
>>>> > > puppet will do the trick, or you need to download repo, or use
>>>> > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no
>>>> deploy effort
>>>> > > other than just install the package.
>>>> > >
>>>> > > 2. Masterless means each puppet agent takes commands and runs
>>>> > > independently.  We DONT use puppet master, which is a centralized
>>>> master
>>>> > > daemon to dispatch works to all the registered puppet agent.
>>>> > >
>>>> > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera
>>>> > > configuration to all nodes, or at least make them available for
all
>>>> nodes
>>>> > > via nfs.
>>>> > >
>>>> > > 4. Finally, execute the puppet apply command on all nodes and the
>>>> puppet
>>>> > > recipes will automatically figure out it's role. There no need
to
>>>> deploy
>>>> > > head node first and then slaves.
>>>> > >
>>>> > > Ask if you still have questions.
>>>> > >
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > > Evans
>>>> > > 2016年3月15日 下午4:10於 "David Starina" <david.starina@gmail.com>寫道:
>>>> > >
>>>> > >> Hi guys,
>>>> > >>
>>>> > >> me again. I haven't set up Puppet before, can you just point
me in
>>>> the
>>>> > >> right direction? I know this is not a Puppet mailing list,
but I
>>>> bet you
>>>> > >> have some tips for me :-)
>>>> > >>
>>>> > >> So what do I need to do? Install Puppet on every node? What
does
>>>> it mean
>>>> > >> that the mode of Puppet is "*masterless"* - do I just have
to do a
>>>> basic
>>>> > >> installation of Puppet and not care much about setting it up,
and
>>>> then
>>>> > >> deploy bigtop on every node separately? And when setting the
>>>> Hadoop head
>>>> > >> node - does that mean that I need to deploy Bigtop on this
node
>>>> first, and
>>>> > >> Bigtop will know this is the master node by the url or IP I
>>>> provide to it?
>>>> > >> And then deploy Bigtop via Puppet on all the other nodes and
they
>>>> will
>>>> > >> connect to the head node (which is the NameNode, I suppose?).
>>>> > >>
>>>> > >> I can do this via trial and error, but if you guys can help
me
>>>> reduce the
>>>> > >> "error" part, that will be great :) Sorry for such beginner
>>>> questions, I
>>>> > >> haven't used Puppet before ... The reason I'm asking is, I
got
>>>> access to a
>>>> > >> "physical" (actually virtual, but it's the same) environment
for a
>>>> limited
>>>> > >> amount of time (two weeks), and I need to start testing on
it asap
>>>> - and
>>>> > >> since my day job takes a lot of my time, if I wanted to do
this
>>>> without
>>>> > >> help it could take me two weeks just to set up the environment
-
>>>> but I need
>>>> > >> to install it, which is not the central part of my project,
and
>>>> then run
>>>> > >> some ML algorithms on a text corpus, which will be the main
part
>>>> of the
>>>> > >> project ...
>>>> > >>
>>>> > >> --David
>>>> > >>
>>>> > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina <
>>>> david.starina@gmail.com>
>>>> > >> wrote:
>>>> > >>
>>>> > >>> Thanks for the info!
>>>> > >>>
>>>> > >>> --David
>>>> > >>>
>>>> > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas <
>>>> jayunit100.apache@gmail.com>
>>>> > >>> wrote:
>>>> > >>>
>>>> > >>>> I think there is enough info on youtube that you can
cobble
>>>> together a
>>>> > >>>> syllabus if you need structured learning.
>>>> > >>>>
>>>> > >>>> See cos/Romans Bigtop talks which are on YouTube and
then my
>>>> Bigtop
>>>> > >>>> internals hackathon video.   That gives you both the
high and
>>>> low level
>>>> > >>>> details to do whatever you want.
>>>> > >>>>
>>>> > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik
<cos@apache.org
>>>> >
>>>> > >>>> wrote:
>>>> > >>>> >
>>>> > >>>> > We also had this discussion to start producing
"The Big Book
>>>> of Apache
>>>> > >>>> > Bigtop" like HBase does, but never got into the
>>>> materialization part
>>>> > >>>> of that
>>>> > >>>> > project.
>>>> > >>>> >
>>>> > >>>> > Cos
>>>> > >>>> >
>>>> > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye
wrote:
>>>> > >>>> >> Hi David,
>>>> > >>>> >>
>>>> > >>>> >> If you just want to take Bigtop binaries to
deploy a cluster,
>>>> you
>>>> > >>>> can refer
>>>> > >>>> >> to our puppet management recipes:
>>>> > >>>> >>
>>>> https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet
>>>> > >>>> >>
>>>> > >>>> >> BTW bigtop provisioner is the best source
to reference for
>>>> how bigtop
>>>> > >>>> >> provision a cluster:
>>>> > >>>> >>
>>>> > >>>>
>>>> https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide
>>>> > >>>> >>
>>>> > >>>> >> Please ask us if you find anything confusing.
>>>> > >>>> >>
>>>> > >>>> >>
>>>> > >>>> >> Evans
>>>> > >>>> >>
>>>> > >>>> >>
>>>> > >>>> >>
>>>> > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina <
>>>> david.starina@gmail.com>:
>>>> > >>>> >>
>>>> > >>>> >>> Hi,
>>>> > >>>> >>>
>>>> > >>>> >>> Is there some version of Bigtop User Guide
available? I know
>>>> it
>>>> > >>>> used to be
>>>> > >>>> >>> on the be on http://bigtop.apache.org/
under the "User
>>>> Guide"
>>>> > >>>> link, but
>>>> > >>>> >>> there's nothing there anymore ...
>>>> > >>>> >>>
>>>> > >>>> >>> I need some guide on how to install Bigtop
on a physical
>>>> cluster
>>>> > >>>> ... It
>>>> > >>>> >>> was easy to create a virtual cluster on
my machine using
>>>> Vagrant,
>>>> > >>>> but I
>>>> > >>>> >>> cannot find a manual to install on a physical
cluster ...
>>>> > >>>> >>>
>>>> > >>>> >>> Cheers,
>>>> > >>>> >>> David
>>>> > >>>> >>>
>>>> > >>>>
>>>> > >>>
>>>> > >>>
>>>> > >>
>>>>
>>>
>>
>

Mime
View raw message