lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shashank Pedamallu <spedama...@vmware.com>
Subject Re: Ingestion not scaling horizontally as I add more cores to Solr
Date Wed, 10 Jan 2018 20:54:09 GMT
Hi Gus,

Thank  for the reply. I’m sending via jmeter running on my local machine to Solr running
on a remote vm.

Thanks,
Shashank

On 1/10/18, 12:34 PM, "Gus Heck" <gus.heck@gmail.com> wrote:

    Ingested how? Sounds like your document sending mechanism is maxed, not the
    solr cluster...
    
    On Wed, Jan 10, 2018 at 2:58 PM, Shashank Pedamallu <spedamallu@vmware.com>
    wrote:
    
    > Hi,
    >
    >
    >
    > I’m trying to find the upper thresholds of ingestion and I have tried the
    > following. In each of the experiments, I’m ingesting random documents with
    > 5 fields.
    >
    >
    > Number of Cores Number of documents ingested per second per core
    > 1       89000
    > 3       33000
    > 5       18000
    >
    >
    > As you can see, the number of documents being ingested per core is not
    > scaling horizontally as I'm adding more cores. Rather the total number of
    > documents getting ingested for Solr JVM is being topped around 90k
    > documents per second.
    >
    >
    > From the iostats and top commands, I do not see any bottlenecks with the
    > iops or cpu respectively, CPU usaeg is around 65% and a sample of iostats
    > is below:
    >
    > avg-cpu:  %user   %nice %system %iowait  %steal   %idle
    >
    >           55.32    0.00    2.33    1.64    0.00   40.71
    >
    >
    > Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
    >
    > sda5           2523.00     45812.00    298312.00      45812     298312
    >
    >
    > Can someone please guide me as to how I can debug this further and
    > root-cause the bottleneck for not being able to increase the ingestion
    > horizontally.
    >
    >
    > Thanks,
    >
    > Shashank
    >
    
    
    
    -- 
    https://urldefense.proofpoint.com/v2/url?u=http-3A__www.the111shift.com&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=DT_33Z3k4h8T1t65CuyH0oMxay15ddkfDYAQefzgpa4&s=6-1wd3YPVRgcvlk3LkK7Wz-3hDFliEGwVGc44HJH1x4&e=
    

Mime
View raw message