manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Tavard <olivier.tav...@francelabs.com>
Subject Rép : Best practices for Postgresql configuration
Date Mon, 09 Oct 2017 16:26:21 GMT

Hi,

Thanks for your answer.
Indeed my estimation was not very precise regarding the volumetry !  I meant 15 millions of
files for 3 TB of data roughly.

Thanks,

Olivier



> Le 9 oct. 2017 à 18:17, user-help@manifoldcf.apache.org a écrit :
> 
> De: Karl Wright <daddywri@gmail.com <mailto:daddywri@gmail.com>>
> Objet: Rép : Best practices for Postgresql configuration
> Date: 9 octobre 2017 à 16:58:25 UTC+2
> À: "user@manifoldcf.apache.org <mailto:user@manifoldcf.apache.org>" <user@manifoldcf.apache.org
<mailto:user@manifoldcf.apache.org>>
> 
> 
> Hi Olivier,
> 
> We've tried versions of Postgresql beyond 9.3, and they seem to work, but there's always
a possibility that the query plans will turn out badly.  But this is unlikely.
> 
> The automatic vacuum operation in Postgresql has gotten much better over time.  You do
not need to pause MCF to do it, but you should expect things to take longer while it is running.
 If you do a full vacuum, however, most operations will be blocked until it is done.
> 
> For further optimization, please let us know how many documents you are indexing.  How
big is "very large"?
> 
> Thanks,
> Karl
> 
> 
> On Mon, Oct 9, 2017 at 10:43 AM, Olivier Tavard <olivier.tavard@francelabs.com <mailto:olivier.tavard@francelabs.com>>
wrote:
> 
> Hi community,
> 
> I have some questions regarding Postgresql tuning performance.
> 
> I configured the MCF Postgresql database with the recommended parameters in this page
: https://manifoldcf.apache.org/release/release-2.8.1/en_US/how-to-build-and-deploy.html#Configuring+a+PostgreSQL+database
<https://manifoldcf.apache.org/release/release-2.8.1/en_US/how-to-build-and-deploy.html#Configuring+a+PostgreSQL+database>
> 
> But maybe some sections are outdated and I would like to know if the recommandations
are still valid for current version of MCF.
> 
> 1) The documentation says that MCF was tested for different versions of PostgreSQL up
to 9.3. Is it OK to run MCF with Postgresql versions beyond 9.3 ?
> I know that this question is often present in the mailing list but it would be good to
know if people use it in production without problem.
> There is also the 10.0 version released few days ago, did someone already test it with
MCF ?
> 
> 2) Some parameters recommended for the postgresql.conf do not longer exist in newer versions
of Postgresql like checkpoint_segments (replaced in 9.4 version by min_wal_size <https://www.postgresql.org/docs/9.5/static/runtime-config-wal.html#GUC-MIN-WAL-SIZE>
and max_wal_size <https://www.postgresql.org/docs/9.5/static/runtime-config-wal.html#GUC-MAX-WAL-SIZE>).
> Is there anything new regarding these parameters since the creation of the documentation
? Do you recommend new settings ?
> 
> 3) Regarding full vacuum operation, I imagine that it is better to do this operation
when MCF is not busy, I mean when there is no job running ? Do we need to pause the jobs in
MCF in order to do so for example ?
> And what is the recommended frequency to perform it ? Some people say each month, others
each day, I would be interested by your recommendation !
> 
> We have some customers with a very large volumetry for the Windows Share repository connector
and we try to tweak the Postgresql configuration for increase MCF performance.
> So it would be great to know how MCF users optimize the Postgresql configuration for
their use.
> 
> Thanks,
> 
> Olivier TAVARD


Mime
View raw message