lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhutia, Devansh" <DDhu...@gannett.com>
Subject Re: DevOps question : auto deployment/setup of Solr & Zookeeper on medium-large clusters
Date Wed, 21 Oct 2015 13:52:22 GMT
We are using aws, and standardized deployments using Chef. 

As Jeff points out below, Exhibitor is a good tool to deploy with Zookeeper. We’ve had very
good luck with it. 



On 10/20/15, 7:59 PM, "Jeff Wartes" <jwartes@whitepages.com> wrote:

>
>If you’re using AWS, there’s this:
>https://github.com/LucidWorks/solr-scale-tk
>If you’re using chef, there’s this:
>https://github.com/vkhatri/chef-solrcloud
>
>(There are several other chef cookbooks for Solr out there, but this is
>the only one I’m aware of that supports Solr 5.3.)
>
>For ZK, I’m less familiar, but if you’re using chef there’s this:
>https://github.com/SimpleFinance/chef-zookeeper
>And this might be handy to know about too:
>https://github.com/Netflix/exhibitor/wiki
>
>
>On 10/20/15, 6:37 AM, "Davis, Daniel (NIH/NLM) [C]" <daniel.davis@nih.gov>
>wrote:
>
>>Waste of money in my opinion.   I would point you towards other tools -
>>bash scripts and free configuration managers such as puppet, chef, salt,
>>or ansible.    Depending on what development you are doing, you may want
>>a continuous integration environment.   For a small company starting out,
>>using a free CI, maybe SaaS, is a good choice.   A professional version
>>such as Bamboo, TeamCity, Jenkins are almost essential in a large
>>enterprise if you are doing diverse builds.
>>
>>When you create a VM, you can generally specify a script to run after the
>>VM is mostly created.   There is a protocol (PXE Boot) that enables this
>>- a PXE server listens and hears that a new server with such-and-such
>>Ethernet Address is starting.   The PXE server makes it boot like a
>>CD-ROM/DVD install, booting from installation media on the network and
>>installing.    Once that install is down, a custom script may be invoked.
>>  This script is typically a bash script, because you may not be able to
>>count on too much else being installed.   However, python/perl are also
>>reasonable choices - just be careful that the modules/libraries you are
>>using for the script are present.    The same PXE protocol is used in
>>large on-premises installations (vCenter) and in the cloud (AWS/Digital
>>Ocean).  We don't care about the PXE server - the point is that you can
>>generally run a bash script after your install.
>>
>>The bash script can bootstrap other services such as puppet, chef, or
>>salt, and/or setup keys so that push configuration management tools such
>>as ansible can reach the server.   The bash script may even be smart
>>enough to do all of the setup you need, depending on what other servers
>>you need to configure.   Smart bash scripts are good for a small company,
>>but for large setups, I'd use puppet, chef, salt, and/or ansible.
>>
>>What I tend to do is to deploy things in such a way that puppet (because
>>it is what we use here) can setup things so that a "solradm" account can
>>setup everything else, and solr and zookeeper are running as a "solrapp"
>>user using puppet.    Then, my continuous integration server, which is
>>Atlassian Bamboo (you can also use tools such as Jenkins, TeamCity,
>>BuildBot), installs solr as "solradm" and sets it up to run as "solrapp".
>>
>>I am not a systems administrator, and I'm not really in "DevOps", my job
>>is to be above all of that and do "systems architecture" which I am lucky
>>still involves coding both in system administration and applications
>>development.   So, that's my 2 cents.
>>
>>Dan Davis, Systems/Applications Architect (Contractor),
>>Office of Computer and Communications Systems,
>>National Library of Medicine, NIH
>>
>>-----Original Message-----
>>From: Susheel Kumar [mailto:susheel2777@gmail.com]
>>Sent: Tuesday, October 20, 2015 9:19 AM
>>To: solr-user@lucene.apache.org
>>Subject: DevOps question : auto deployment/setup of Solr & Zookeeper on
>>medium-large clusters
>>
>>Hello,
>>
>>Resending to see opinion from Dev-Ops perspective on the tools for
>>installing/deployment of Solr & ZK on large no of machines and
>>maintaining them. I have heard Bladelogic or HP OO (commercial tools)
>>etc. being used.
>>Please share your experience or pros / cons of such tools.
>>
>>Thanks,
>>Susheel
>>
>>On Mon, Oct 19, 2015 at 3:32 PM, Susheel Kumar <susheel2777@gmail.com>
>>wrote:
>>
>>> Hi,
>>>
>>> I am trying to find the best practises for setting up Solr on new 20+
>>> machines  & ZK (5+) and repeating same on other environments.  What's
>>> the best way to download, extract, setup Solr & ZK in an automated way
>>> along with other dependencies like java etc.  Among shell scripts or
>>> puppet or docker or imaged vm's what is being used & suggested from
>>> Dev-Ops perspective.
>>>
>>> Thanks,
>>> Susheel
>>>
>
Mime
View raw message