ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <>
Subject [jira] [Created] (AMBARI-11529) Work with json module takes too long
Date Fri, 29 May 2015 11:30:17 GMT
Andrew Onischuk created AMBARI-11529:

             Summary: Work with json module takes too long
                 Key: AMBARI-11529
             Project: Ambari
          Issue Type: Bug
            Reporter: Andrew Onischuk
            Assignee: Andrew Onischuk
             Fix For: 2.1.0

Due to profiling below, from 420 seconds of deploy json.load and json.loads
take 40 seconds (which even doesn't include working with json in ambari-agent
itself, only script):

          204    0.007    0.000   20.182    0.099
          351    0.002    0.000   17.894    0.051

This is because json module for Python 2.6 is extremety slow. There are
benchmark for it on here:  

due to it simplejson is at least ~40 times faster for deserializing.

Also another thing to do in scope of this jira is to replace inspect module
usage with os._getframe() which doesn't use realpath function that
intensively, taking away 10 more seconds.

    30049/21438    1.712    0.000   10.873    0.001

This message was sent by Atlassian JIRA

View raw message