manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Can the authority service be clustered to achieve failure tolerance?
Date Wed, 06 Apr 2016 07:42:44 GMT
Hi Markus,

The API service uses session for login, as does the crawler UI, so you'd
need session affinity to be sure these aren't messed up.

The main reason for the process ID for the agents process is because there
is state in the database that needs management -- specifically, state
cleanup when an agents process goes down or is restarted.  I don't remember
at the moment whether resource pools (managed in Zookeeper) are reliant in
any way on the process ID, but I don't think so.  A quick review of the
code doesn't show up anything either.  So I think you are good to go not
specifying process ID's for any non-agents process.

Karl




On Wed, Apr 6, 2016 at 2:12 AM, Markus Schuch <markus_schuch@web.de> wrote:

> Hi Karl,
>
> i copied the start-webapps.sh and added a jvm property for a second
> jetty.xml with a different Port.
>
> This was enough to start the second bunch of webapps in a second jetty.
>
> The authority servlet worked for me on both instances (at least with the
> Null Authority).
>
> I did some jmeter testing to fire requests to both instances with multiple
> threads. No problems occurred.
>
> I did not set the process ID property.
>
> For the record: This test setup also runs the API service and the crawler
> ui in the second instance. Both instances seem to work. But i expect -
> because of their stateful nature - there will be issues when load balancing
> on them. (Login state, parallel write transactions, etc.) But may be a "hot
> standby" architecture can be achieved that way. What do you think?
>
> Regards
> Markus
>
>
>
> Am 5. April 2016 16:59:06 MESZ, schrieb Karl Wright <daddywri@gmail.com>:
>>
>> Hi Markus,
>>
>> >>>>>>
>> If found this JVM Parameter in the start-agents.sh example:
>>   org.apache.manifoldcf.processid
>> Is that it?
>> <<<<<<
>>
>> Yes.
>>
>> >>>>>>
>> Another question:
>> You say the authority services webapp is stateless. There is a max
>> connection parameter for authority connections. How will multiple instances
>> treat this parameter? Are the connections limited throughout the cluster or
>> will each instance respect the limit in its own context?
>> <<<<<<
>>
>> Connection pools in a clustered environment are managed cluster-wide.  So
>> the number you give is the maximum number of connection instances across
>> the entire cluster for that connection.
>>
>> Thanks,
>> Karl
>>
>>
>> On Tue, Apr 5, 2016 at 10:50 AM, Markus Schuch <markus_schuch@web.de>
>> wrote:
>>
>>> Hi Karl,
>>>
>>> what do you mean with seperate ID?
>>>
>>> If found this JVM Parameter in the start-agents.sh example:
>>>   org.apache.manifoldcf.processid
>>> Is that it?
>>>
>>> Another question:
>>> You say the authority services webapp is stateless. There is a max
>>> connection parameter for authority connections. How will multiple instances
>>> treat this parameter? Are the connections limited throughout the cluster or
>>> will each instance respect the limit in its own context?
>>>
>>> Regards,
>>> Markus
>>>
>>> *Gesendet:* Dienstag, 05. April 2016 um 13:09 Uhr
>>> *Von:* "Karl Wright" <daddywri@gmail.com>
>>> *An:* "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>> *Betreff:* Re: Can the authority service be clustered to achieve
>>> failure tolerance?
>>> I should also make it clear that each authority service web application
>>> instance may need a separate ID (just like the agents processes).  It's not
>>> technically necessary, because the authority service is read only, but if
>>> you run into problems that is where the issue will lie.  You would set the
>>> ID though a -D switch given to the application server process.
>>>
>>> Thanks,
>>> Karl
>>>
>>>
>>> On Tue, Apr 5, 2016 at 7:06 AM, Karl Wright <daddywri@gmail.com> wrote:
>>>>
>>>> Hi Markus,
>>>>
>>>> I've heard this question asked maybe once before.
>>>>
>>>> Basically, we rely on whatever support your application server or
>>>> environment has for load balancing.  Since the authority service runs as
a
>>>> web application that does not require session state, this should be
>>>> relatively easy to set up.  Having said that, I've never myself tried to
do
>>>> it.  But if you want to do some experiments, please share the results with
>>>> the group. :-)
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>>
>>>> On Tue, Apr 5, 2016 at 6:31 AM, Markus Schuch <markus_schuch@web.de>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> is it possible to setup ManifoldCF to be failure tolerant when it
>>>>> comes to authority checks?
>>>>> (we are on MCF 2.3)
>>>>>
>>>>> After reading the
>>>>> https://manifoldcf.apache.org/release/trunk/en_US/how-to-build-and-deploy.html
>>>>> page and excercising "Simplified multi-process model using ZooKeeper-based
>>>>> synchronization" example i learned only how to cluster the agents process.
>>>>> I found no information if it is possible to run multiple authority services.
>>>>>
>>>>> Is there more documentation about manifoldcf and clustering that i
>>>>> missed?
>>>>>
>>>>> Thanks in advance
>>>>> Markus
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>
> --
> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail
> gesendet.
>

Mime
View raw message