cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hair <j...@greenqloud.com>
Subject Re: Virtual Router execute python slowly
Date Sat, 20 Aug 2016 11:30:44 GMT
I brought up something similar to this recently. We notice this with VM
metadata and passwords in the code. The virtual router does not properly
clean up after itself, so there are leftover entries that the
update_config.py script must continually update which are no longer
relevant. If you make a script that continually creates and destroys
machines while watching the management server/agent (depending on
hypervisor) logs for the amount of time it takes to run update_config.py,
you will see a linear increase in time. In the beginning configuring a
router takes maybe 2 seconds. A few hundred VMs later and it's easily up to
30+ seconds. It is actually quite easy to essentially deadlock the
deployment of VMs because the router takes so long to configure itself.

We are working on adding some code to the release step of the
VirtualRouterElement that will send a router command to remove VM
metadata/password entries, and hopefully that will speed things up a bit
too.

*Jeff Hair*
Technical Lead and Software Developer

Tel: (+354) 415 0200
jeff@greenqloud.com
www.greenqloud.com

On Sat, Aug 20, 2016 at 5:30 AM, Gust <digitalgust@163.com> wrote:

> A port forward rule operation spent:
>
> not  optimized          5min
> optimized                 3min
> optmized+cProfile    8min
>
> top 30 :
>
> Fri Aug 19 11:27:50 2016    d:\temp\py_pro.data
>
>          614887355 function calls (614884838 primitive calls) in 548.846
> seconds
>
>    Ordered by: cumulative time
>    List reduced from 404 to 30 due to restriction <30>
>
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.000    0.000  548.846  548.846 <string>:1(<module>)
>         1    0.000    0.000  548.846  548.846 update_config.py:114(mainp)
>         1    0.000    0.000  548.846  548.846 update_config.py:50(process_
> file)
>         1    0.001    0.001  548.837  548.837 update_config.py:44(finish_
> config)
>         1    0.000    0.000  548.836  548.836 configure.py:903(main)
>         1    0.000    0.000  546.228  546.228 configure.py:717(process)
>      1088   53.936    0.050  536.665    0.493
> CsAddress.py:145(ip_in_subnet)
>        16    0.000    0.000  364.540   22.784 configure.py:766(
> processForwardRule)
>        16    0.001    0.000  364.540   22.784 configure.py:772(forward_vr)
>        32    5.869    0.183  351.378   10.981 configure.py:747(
> getNetworkByIp)
>  68170816 65.435    0.000  342.427    0.000 __init__.py:1682(iter_iprange)
>  68177344  182.925    0.000  277.051    0.000 __init__.py:248(__init__)
>        48    3.685    0.077  194.845    4.059 configure.py:741(
> getDeviceByIp)
>        16    0.001    0.000  181.689   11.356 configure.py:882(
> processStaticNatRule)
>  68170816 54.655    0.000  140.186    0.000 __init__.py:63(__hash__)
>  68170976 48.577    0.000   69.112    0.000 __init__.py:439(key)
>  68187121   42.976    0.000   42.976    0.000 {hasattr}
>  68188576   26.202    0.000   26.205    0.000 {isinstance}
>  68178432   24.945    0.000   24.945    0.000 __init__.py:34(__init__)
>  68177504   20.537    0.000   20.537    0.000 __init__.py:232(version)
>  68170816   16.419    0.000   16.419    0.000 {hash}
>         1    0.020    0.020    1.522    1.522 CsNetfilter.py:134(compare)
>       329    0.007    0.000    1.513    0.005 CsHelper.py:182(execute)
>       329    0.002    0.000    1.060    0.003
> subprocess.py:768(communicate)
>       329    0.004    0.000    1.057    0.003 subprocess.py:1400(_
> communicate)
>       329    0.012    0.000    1.046    0.003 subprocess.py:1433(_
> communicate_with_poll)
>      2352    1.021    0.000    1.021    0.000 {built-in method poll}
>         1    0.001    0.001    1.005    1.005 CsAddress.py:99(process)
>        22    0.001    0.000    0.989    0.045 CsAddress.py:271(configure)
>        22    0.001    0.000    0.937    0.043 CsAddress.py:285(post_
> configure)
>
>
>
> > 在 2016年8月20日,09:41,ilya <ilya.mailing.lists@gmail.com> 写道:
> >
> > Hi Gust
> >
> > Are you proposing solution "optimized CsAddress.py in VR" or just
> > analysis on where time is being spent?
> >
> > Regards
> > ilya
> > On 8/18/16 9:34 PM, Gust wrote:
> >>
> >> Hi,all
> >>
> >> we constructed a advance network in cloudstack , it work fine early ,
> but when I added some VR rules ,it execute slow more and more.
> >>
> >> About total 100 rules already exist ,  I add a port forward rule , it
> would execute about 3 minutes , that we changed the agent source  set more
> seconds timeout ,else agent will timeout at 120s,  and optimized
> CsAddress.py in VR.
> >>
> >> So I login to VR vm , and  modify the update_config.py    with
> cProfile , and print out execute result . because add cProfile , python
> execute more than 8 min.
> >>
> >> It shows too many netaddr  object init invoked.
> >>
> >> vr hypervisor hardware :   E3 1230v2  3.4GHz   , vr alloc 3GHz
> >>
> >> ----------------------------------------------------------------------
> >> Optimized CsAddress.py , execute 3 min, else 5 min
> >>
> >> CsAddress.py (line 145)
> >>   def ip_in_subnet(self, ip):
> >>       ipo = IPAddress(ip)
> >>       net = IPNetwork("%s/%s" % (self.get_ip(), self.get_size()))
> >>       aset = set(net)
> >>       return  ipo in aset
> >>
> >> ----------------------------------------------------------------------
> >> def mainp():
> >>   if not (os.path.isfile(jsonCmdConfigPath) and
> os.access(jsonCmdConfigPath, os.R_OK)):
> >>       print "[ERROR] update_config.py :: You are telling me to process
> %s, but i can't access it" % jsonCmdConfigPath
> >>       sys.exit(1)
> >>
> >>   # If the command line json file is unprocessed process it
> >>   # This is important or, the control interfaces will get deleted!
> >>   if os.path.isfile(jsonPath % "cmd_line.json"):
> >>       qf = QueueFile()
> >>       qf.setFile("cmd_line.json")
> >>       qf.load(None)
> >>
> >>   # If the guest network is already configured and have the same IP, do
> not try to configure it again otherwise it will break
> >>   if sys.argv[1] == "guest_network.json":
> >>       if os.path.isfile(currentGuestNetConfig):
> >>           file = open(currentGuestNetConfig)
> >>           guestnet_dict = json.load(file)
> >>
> >>           if not is_guestnet_configured(guestnet_dict, ['eth1',
> 'eth2', 'eth3', 'eth4', 'eth5', 'eth6', 'eth7', 'eth8', 'eth9']):
> >>               print "[INFO] update_config.py :: Processing Guest
> Network."
> >>               process_file()
> >>           else:
> >>               print "[INFO] update_config.py :: No need to process
> Guest Network."
> >>               finish_config()
> >>       else:
> >>           print "[INFO] update_config.py :: No GuestNetwork configured
> yet. Configuring first one now."
> >>           process_file()
> >>   else:
> >>       print "[INFO] update_config.py :: Processing incoming file => %s"
> % sys.argv[1]
> >>       process_file()
> >>
> >> cProfile.run("mainp()", "/var/log/py_pro.data”)
> >>
> >> ----------------------------------------------------------------------
> >>
> >> Fri Aug 19 11:27:50 2016    d:\temp\py_pro.data
> >>
> >>        614887355 function calls (614884838 primitive calls) in 548.846
> seconds
> >>
> >>  Ordered by: internal time
> >>
> >>  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
> >> 68177344  182.925    0.000  277.051    0.000 __init__.py:248(__init__)
> >> 68170816   65.435    0.000  342.427    0.000
> __init__.py:1682(iter_iprange)
> >> 68170816   54.655    0.000  140.186    0.000 __init__.py:63(__hash__)
> >>    1088   53.936    0.050  536.665    0.493
> CsAddress.py:145(ip_in_subnet)
> >> 68170976   48.577    0.000   69.112    0.000 __init__.py:439(key)
> >> 68187121   42.976    0.000   42.976    0.000 {hasattr}
> >> 68188576   26.202    0.000   26.205    0.000 {isinstance}
> >> 68178432   24.945    0.000   24.945    0.000 __init__.py:34(__init__)
> >> 68177504   20.537    0.000   20.537    0.000 __init__.py:232(version)
> >> 68170816   16.419    0.000   16.419    0.000 {hash}
> >>      32    5.869    0.183  351.378   10.981 configure.py:747(
> getNetworkByIp)
> >>      48    3.685    0.077  194.845    4.059 configure.py:741(
> getDeviceByIp)
> >>    2352    1.021    0.000    1.021    0.000 {built-in method poll}
> >>  245441    0.352    0.000    0.510    0.000 CsNetfilter.py:296(__eq__)
> >>    2149    0.251    0.000    0.251    0.000 {posix.read}
> >>     798    0.111    0.000    0.608    0.001 CsNetfilter.py:116(has_rule)
> >>  493585    0.098    0.000    0.098    0.000
> CsNetfilter.py:258(get_table)
> >>     329    0.077    0.000    0.077    0.000 {posix.fork}
> >>    2444    0.059    0.000    0.059    0.000 {method 'flush' of 'file'
> objects}
> >>    2444    0.039    0.000    0.074    0.000 __init__.py:243(__init__)
> >>  113950    0.030    0.000    0.030    0.000
> CsNetfilter.py:270(get_chain)
> >>     329    0.028    0.000    0.367    0.001 subprocess.py:1195(_execute_
> child)
> >>    2444    0.021    0.000    0.078    0.000 __init__.py:458(format)
> >>    2069    0.021    0.000    0.039    0.000
> CsNetfilter.py:236(__convert_to_dict)
> >>       1    0.020    0.020    1.522    1.522 CsNetfilter.py:134(compare)
> >>    2444    0.017    0.000    0.024    0.000 __init__.py:1226(findCaller)
> >>    2444    0.016    0.000    0.017    0.000 __init__.py:318(getMessage)
> >>   41763    0.015    0.000    0.015    0.000 {method 'items' of 'dict'
> objects}
> >>    2444    0.015    0.000    0.015    0.000 {time.localtime}
> >>    1088    0.014    0.000    0.028    0.000 __init__.py:737(parse_ip_
> network)
> >>    2444    0.013    0.000    0.036    0.000 __init__.py:411(formatTime)
> >>    4889    0.013    0.000    0.018    0.000 threading.py:187(release)
> >>    2444    0.013    0.000    0.194    0.000 __init__.py:847(emit)
> >>    4889    0.013    0.000    0.018    0.000 threading.py:147(acquire)
> >>     329    0.012    0.000    1.046    0.003 subprocess.py:1433(_
> communicate_with_poll)
> >>   14709    0.011    0.000    0.011    0.000 {method 'replace' of
> 'unicode' objects}
> >>   47110    0.011    0.000    0.011    0.000 CsNetfilter.py:273(get_rule)
> >>     162    0.009    0.000    0.011    0.000 CsNetfilter.py:276(to_str)
> >>    2444    0.009    0.000    0.362    0.000 __init__.py:1260(_log)
> >> 47397/47389    0.009    0.000    0.009    0.000 {len}
> >>    2444    0.008    0.000    0.090    0.000 __init__.py:836(flush)
> >>    3458    0.008    0.000    0.008    0.000 {method 'write' of 'file'
> objects}
> >>    2082    0.008    0.000    0.312    0.000 __init__.py:1138(debug)
> >>    1088    0.008    0.000    0.031    0.000 __init__.py:648(__iter__)
> >>    2444    0.008    0.000    0.243    0.000
> __init__.py:1313(callHandlers)
> >>     329    0.008    0.000    0.397    0.001 subprocess.py:650(__init__)
> >>    2176    0.008    0.000    0.016    0.000 ipv4.py:123(str_to_int)
> >>    2444    0.008    0.000    0.008    0.000 {time.strftime}
> >>     329    0.007    0.000    1.513    0.005 CsHelper.py:182(execute)
> >>    2444    0.007    0.000    0.235    0.000 __init__.py:744(handle)
> >>    3221    0.007    0.000    0.007    0.000 {method 'split' of
> 'unicode' objects}
> >>    2082    0.007    0.000    0.319    0.000 __init__.py:1624(debug)
> >>    4888    0.007    0.000    0.024    0.000 __init__.py:708(release)
> >>    1088    0.007    0.000    0.039    0.000 __init__.py:855(__init__)
> >>    3948    0.007    0.000    0.007    0.000 {fcntl.fcntl}
> >>    2444    0.007    0.000    0.200    0.000 __init__.py:941(emit)
> >>     658    0.007    0.000    0.007    0.000 {posix.fdopen}
> >>    2309    0.006    0.000    0.006    0.000 CsAddress.py:167(get_attr)
> >>    4888    0.006    0.000    0.024    0.000 __init__.py:701(acquire)
> >>    2444    0.006    0.000    0.079    0.000 __init__.py:1247(makeRecord)
> >>    2444    0.005    0.000    0.006    0.000
> __init__.py:154(getLevelName)
> >>    2444    0.005    0.000    0.006    0.000 genericpath.py:85(_splitext)
> >>    2444    0.005    0.000    0.249    0.000 __init__.py:1281(handle)
> >>    2444    0.004    0.000    0.006    0.000
> __init__.py:1353(isEnabledFor)
> >>    1088    0.004    0.000    0.004    0.000 {_socket.inet_aton}
> >>   14666    0.004    0.000    0.004    0.000 {thread.get_ident}
> >>      21    0.004    0.000    0.018    0.001 CsNetfilter.py:212(delete)
> >>     329    0.004    0.000    1.057    0.003 subprocess.py:1400(_
> communicate)
> >>    9778    0.004    0.000    0.004    0.000 threading.py:64(_note)
> >>    1088    0.003    0.000    0.010    0.000 __init__.py:1003(hostmask)
> >>    1316    0.003    0.000    0.003    0.000 {posix.close}
> >>     987    0.003    0.000    0.003    0.000 {posix.pipe}
> >>    4888    0.003    0.000    0.003    0.000 __init__.py:605(filter)
> >>       1    0.003    0.003    0.019    0.019 CsNetfilter.py:89(get_all_
> rules)
> >>    7332    0.003    0.000    0.003    0.000 {method 'rfind' of 'str'
> objects}
> >>    1974    0.003    0.000    0.010    0.000 subprocess.py:1156(_set_
> cloexec_flag)
> >>    2444    0.003    0.000    0.081    0.000 __init__.py:721(format)
> >>    2444    0.003    0.000    0.009    0.000 posixpath.py:104(splitext)
> >>       1    0.003    0.003    0.269    0.269
> CsNetfilter.py:125(get_unseen)
> >>     329    0.003    0.000    0.003    0.000 {posix.waitpid}
> >>    2444    0.003    0.000    0.003    0.000 threading.py:1152(
> currentThread)
> >>     112    0.003    0.000    0.004    0.000 CsAddress.py:37(get_
> interfaces)
> >>     753    0.003    0.000    0.003    0.000 {method 'close' of 'file'
> objects}
> >>     987    0.003    0.000    0.016    0.000 subprocess.py:1169(pipe_
> cloexec)
> >>    2069    0.003    0.000    0.042    0.000 CsNetfilter.py:227(parse)
> >>    2444    0.003    0.000    0.004    0.000 posixpath.py:119(basename)
> >>    1088    0.003    0.000    0.013    0.000 __init__.py:980(first)
> >>    2444    0.003    0.000    0.004    0.000 __init__.py:452(usesTime)
> >>     658    0.003    0.000    0.004    0.000
> subprocess.py:1216(_close_in_parent)
> >>    2444    0.002    0.000    0.004    0.000 __init__.py:82(<lambda>)
> >>     329    0.002    0.000    0.015    0.000 subprocess.py:1107(_get_
> handles)
> >>    2069    0.002    0.000    0.002    0.000 CsNetfilter.py:220(__init__)
> >>    2444    0.002    0.000    0.002    0.000 threading.py:967(name)
> >>     329    0.002    0.000    0.007    0.000 subprocess.py:1379(wait)
> >>     658    0.002    0.000    0.005    0.000 subprocess.py:1444(close_
> unregister_and_remove)
> >>     231    0.002    0.000    0.002    0.000 {open}
> >>    6407    0.002    0.000    0.002    0.000 {method 'get' of 'dict'
> objects}
> >>      64    0.002    0.000    0.005    0.000 CsHelper.py:92(updatefile)
> >>    6526    0.002    0.000    0.002    0.000 {method 'keys' of 'dict'
> objects}
> >>    1088    0.002    0.000    0.002    0.000 {_socket.inet_pton}
> >>     658    0.002    0.000    0.252    0.000 subprocess.py:473(_eintr_
> retry_call)
> >>     658    0.002    0.000    0.003    0.000 subprocess.py:1440(register_
> and_append)
> >>     329    0.002    0.000    1.060    0.003
> subprocess.py:768(communicate)
> >>    2010    0.002    0.000    0.007    0.000 re.py:143(search)
> >>    2011    0.002    0.000    0.004    0.000 re.py:230(_compile)
> >>    2069    0.002    0.000    0.002    0.000 {zip}
> >>     361    0.002    0.000    0.068    0.000 __init__.py:1616(info)
> >>    2176    0.002    0.000    0.002    0.000 {_struct.unpack}
> >>    2444    0.002    0.000    0.002    0.000 {sys._getframe}
> >>    7349    0.002    0.000    0.002    0.000 {method 'append' of 'list'
> objects}
> >> 3310/1692    0.002    0.000    0.003    0.000
> encoder.py:341(_iterencode_dict)
> >>    1120    0.002    0.000    0.006    0.000 CsAddress.py:130(get_ip)
> >>    2444    0.002    0.000    0.002    0.000 __init__.py:1339(
> getEffectiveLevel)
> >>     361    0.002    0.000    0.066    0.000 __init__.py:1150(info)
> >>    2506    0.002    0.000    0.002    0.000 {method 'find' of 'str'
> objects}
> >>     840    0.002    0.000    0.002    0.000 CsNetfilter.py:54(has_chain)
> >>   824/1    0.001    0.000    0.003    0.003 copy.py:145(deepcopy)
> >>     202    0.001    0.000    0.003    0.000
> abc.py:128(__instancecheck__)
> >>    2445    0.001    0.000    0.001    0.000 {method 'acquire' of
> 'thread.lock' objects}
> >>    1088    0.001    0.000    0.001    0.000 __init__.py:988(last)
> >>      42    0.001    0.000    0.005    0.000
> CsHelper.py:156(definedinfile)
> >>       6    0.001    0.000    0.006    0.001 CsFile.py:119(search)
> >>    2445    0.001    0.000    0.001    0.000 {time.time}
> >>     840    0.001    0.000    0.007    0.000
> CsNetfilter.py:182(add_chain)
> >>    2445    0.001    0.000    0.001    0.000 {method 'release' of
> 'thread.lock' objects}
> >>    5977    0.001    0.000    0.001    0.000 {method 'lstrip' of 'str'
> objects}
> >>      21    0.001    0.000    0.002    0.000 CsAddress.py:356(fw_router)
> >>       7    0.001    0.000    0.001    0.000 decoder.py:372(raw_decode)
> >>      22    0.001    0.000    0.937    0.043 CsAddress.py:285(post_
> configure)
> >>    2033    0.001    0.000    0.001    0.000 {getattr}
> >>    4888    0.001    0.000    0.001    0.000 posixpath.py:51(normcase)
> >>    1501    0.001    0.000    0.001    0.000 {method 'split' of 'str'
> objects}
> >>    2743    0.001    0.000    0.001    0.000 {method 'replace' of 'str'
> objects}
> >>      21    0.001    0.000    0.065    0.003 CsRule.py:39(findMark)
> >>     329    0.001    0.000    0.001    0.000 subprocess.py:754(__del__)
> >>      16    0.001    0.000  364.540   22.784 configure.py:772(forward_vr)
> >>    1088    0.001    0.000    0.002    0.000 CsAddress.py:154(get_size)
> >>     329    0.001    0.000    0.001    0.000 {gc.enable}
> >>     389    0.001    0.000    0.001    0.000
> _weakrefset.py:70(__contains__)
> >>     513    0.001    0.000    0.001    0.000 CsNetfilter.py:40(add_rule)
> >>    2444    0.001    0.000    0.001    0.000 {posix.getpid}
> >>       1    0.001    0.001    1.005    1.005 CsAddress.py:99(process)
> >>    2069    0.001    0.000    0.001    0.000
> CsNetfilter.py:253(set_table)
> >>      24    0.001    0.000    0.001    0.000 CsAddress.py:226(buildlist)
> >>      32    0.001    0.000    0.003    0.000 configure.py:728(getGuestIp)
> >>    2485    0.001    0.000    0.001    0.000 CsAddress.py:126(__init__)
> >>    2010    0.001    0.000    0.001    0.000 {method 'search' of
> '_sre.SRE_Pattern' objects}
> >>    3161    0.001    0.000    0.001    0.000 {method 'startswith' of
> 'str' objects}
> >>    4774    0.001    0.000    0.001    0.000 {method 'strip' of 'str'
> objects}
> >>      42    0.001    0.000    0.014    0.000 CsRoute.py:32(add_table)
> >>     704    0.001    0.000    0.001    0.000 CsAddress.py:186(is_guest)
> >>     329    0.001    0.000    0.001    0.000 subprocess.py:458(_cleanup)
> >>     329    0.001    0.000    0.001    0.000 subprocess.py:1338(_handle_
> exitstatus)
> >>     658    0.001    0.000    0.001    0.000 {built-in method unregister}
> >>      21    0.001    0.000    0.001    0.000 CsAddress.py:693(cpus)
> >>     668    0.001    0.000    0.001    0.000 {method 'remove' of 'set'
> objects}
> >>    2176    0.001    0.000    0.001    0.000 __init__.py:452(__int__)
> >>    1692    0.001    0.000    0.003    0.000 encoder.py:417(_iterencode)
> >>      22    0.001    0.000    0.989    0.045 CsAddress.py:271(configure)
> >>      33    0.001    0.000    0.005    0.000 configure.py:114(add_rule)
> >>      21    0.001    0.000    0.014    0.001 CsAddress.py:668(enable)
> >>     329    0.001    0.000    0.001    0.000 {method 'splitlines' of
> 'str' objects}
> >>      16    0.001    0.000  181.689   11.356 configure.py:882(
> processStaticNatRule)
> >>     658    0.001    0.000    0.001    0.000 {built-in method register}
> >>       1    0.001    0.001  548.837  548.837 update_config.py:44(finish_
> config)
> >>     689    0.001    0.000    0.001    0.000 copy.py:267(_keep_alive)
> >>      22    0.001    0.000    0.058    0.003 CsRoute.py:85(defaultroute_
> exists)
> >>     840    0.001    0.000    0.002    0.000
> CsNetfilter.py:113(has_chain)
> >>      21    0.001    0.000    0.074    0.004
> CsAddress.py:516(post_config_change)
> >>    1974    0.001    0.000    0.001    0.000 {method 'fileno' of 'file'
> objects}
> >>     510    0.001    0.000    0.001    0.000 CsDatabag.py:55(idata)
> >>     658    0.001    0.000    0.001    0.000 {method 'update' of 'set'
> objects}
> >>      71    0.000    0.000    0.000    0.000 {method 'read' of 'file'
> objects}
> >>     695    0.000    0.000    0.000    0.000
> CsNetfilter.py:233(mark_seen)
> >>     950    0.000    0.000    0.000    0.000 {method 'lstrip' of
> 'unicode' objects}
> >>     660    0.000    0.000    0.000    0.000 {method 'join' of 'str'
> objects}
> >>    34/1    0.000    0.000    0.003    0.003 copy.py:253(_deepcopy_dict)
> >>     329    0.000    0.000    0.000    0.000 {method 'count' of 'list'
> objects}
> >>   12/11    0.000    0.000    0.001    0.000 sre_parse.py:388(_parse)
> >>     798    0.000    0.000    0.000    0.000 CsNetfilter.py:107(get)
> >>     658    0.000    0.000    0.000    0.000 {method 'pop' of 'dict'
> objects}
> >>      24    0.000    0.000    0.002    0.000 CsAddress.py:207(__init__)
> >>     179    0.000    0.000    0.001    0.000 CsConfig.py:60(is_vpc)
> >>       2    0.000    0.000    0.010    0.005
> CsHelper.py:190(save_iptables)
> >>       1    0.000    0.000    0.004    0.004 encoder.py:186(encode)
> >>      58    0.000    0.000    0.000    0.000 {posix.stat}
> >>      21    0.000    0.000    0.050    0.002 CsRoute.py:54(set_route)
> >>     329    0.000    0.000    0.000    0.000 {select.poll}
> >>      21    0.000    0.000    0.053    0.003 CsRoute.py:45(add_route)
> >>      66    0.000    0.000    0.000    0.000 {sorted}
> >>      21    0.000    0.000    0.045    0.002 CsHelper.py:35(reconfigure_
> interfaces)
> >>     184    0.000    0.000    0.001    0.000 CsDatabag.py:82(get_type)
> >>      21    0.000    0.000    0.668    0.032 CsAddress.py:598(arpPing)
> >>      47    0.000    0.000    0.001    0.000 genericpath.py:26(isfile)
> >>     329    0.000    0.000    0.000    0.000
> subprocess.py:1352(_internal_poll)
> >>      33    0.000    0.000    0.005    0.000 configure.py:86(__init__)
> >>    1696    0.000    0.000    0.000    0.000 {id}
> >>      21    0.000    0.000    0.000    0.000 CsAddress.py:320(set_mark)
> >>      11    0.000    0.000    0.000    0.000 sre_compile.py:433(_compile_
> info)
> >>     252    0.000    0.000    0.000    0.000 encoder.py:288(_iterencode_
> list)
> >>     129    0.000    0.000    0.000    0.000 CsAddress.py:325(get_type)
> >>     410    0.000    0.000    0.000    0.000 CsNetfilter.py:104(save)
> >>      42    0.000    0.000    0.006    0.000
> CsHelper.py:164(addifmissing)
> >>     695    0.000    0.000    0.000    0.000
> CsNetfilter.py:267(get_count)
> >>      21    0.000    0.000    0.005    0.000 CsAddress.py:683(inKernel)
> >>     179    0.000    0.000    0.000    0.000 sre_parse.py:191(__next)
> >>     123    0.000    0.000    0.000    0.000 {method 'add' of 'set'
> objects}
> >>     329    0.000    0.000    0.000    0.000 {posix.WIFSIGNALED}
> >>       1    0.000    0.000  548.836  548.836 configure.py:903(main)
> >>   20/11    0.000    0.000    0.000    0.000 sre_compile.py:64(_compile)
> >>       2    0.000    0.000    0.010    0.005 configure.py:272(process)
> >>      80    0.000    0.000    0.000    0.000 __init__.py:69(__eq__)
> >>     329    0.000    0.000    0.000    0.000 {posix.WEXITSTATUS}
> >>      61    0.000    0.000    0.000    0.000 CsConfig.py:45(cmdline)
> >>     410    0.000    0.000    0.000    0.000 CsNetfilter.py:48(get_count)
> >>      10    0.000    0.000    0.004    0.000 merge.py:49(load)
> >>      66    0.000    0.000    0.000    0.000 CsAddress.py:589(ip)
> >>      64    0.000    0.000    0.000    0.000 configure.py:759(
> portsToString)
> >>      21    0.000    0.000    0.000    0.000 CsRule.py:28(__init__)
> >>       1    0.000    0.000    0.019    0.019 CsNetfilter.py:188(del_
> standard)
> >>     537    0.000    0.000    0.000    0.000 {_json.encode_basestring_
> ascii}
> >>      69    0.000    0.000    0.000    0.000 CsAddress.py:158(get_device)
> >>   28/19    0.000    0.000    0.000    0.000 sre_parse.py:149(getwidth)
> >>       6    0.000    0.000    0.014    0.002 CsAddress.py:255(__init__)
> >>       6    0.000    0.000    0.014    0.002 CsAddress.py:555(list)
> >>     329    0.000    0.000    0.000    0.000 {gc.isenabled}
> >>       1    0.000    0.000  546.228  546.228 configure.py:717(process)
> >>     465    0.000    0.000    0.000    0.000 CsNetfilter.py:76(last)
> >>      57    0.000    0.000    0.000    0.000 copy.py:226(_deepcopy_list)
> >>     9/8    0.000    0.000    0.003    0.000 CsDatabag.py:24(__init__)
> >>       1    0.000    0.000    0.000    0.000 CsFile.py:30(load)
> >>     410    0.000    0.000    0.000    0.000
> CsNetfilter.py:264(set_count)
> >>     598    0.000    0.000    0.000    0.000
> copy.py:198(_deepcopy_atomic)
> >>     410    0.000    0.000    0.000    0.000
> CsNetfilter.py:261(set_chain)
> >>      21    0.000    0.000    0.067    0.003 CsRule.py:33(addMark)
> >>      25    0.000    0.000    0.000    0.000
> CsDatabag.py:68(is_redundant)
> >>      44    0.000    0.000    0.000    0.000 stat.py:49(S_ISREG)
> >>      22    0.000    0.000    0.000    0.000 CsAddress.py:571(configured)
> >>    10/5    0.000    0.000    0.000    0.000
> abc.py:148(__subclasscheck__)
> >>      63    0.000    0.000    0.000    0.000 CsRoute.py:29(get_tablename)
> >>     167    0.000    0.000    0.000    0.000 sre_parse.py:210(get)
> >>     391    0.000    0.000    0.000    0.000 CsNetfilter.py:230(unseen)
> >>      56    0.000    0.000    0.000    0.000 CsNetfilter.py:31(add)
> >>     329    0.000    0.000    0.000    0.000 {posix.WIFEXITED}
> >>      11    0.000    0.000    0.002    0.000 sre_compile.py:567(compile)
> >>     371    0.000    0.000    0.000    0.000 encoder.py:361(<lambda>)
> >>     137    0.000    0.000    0.000    0.000 sre_parse.py:147(append)
> >>     115    0.000    0.000    0.000    0.000 CsConfig.py:48(address)
> >>      22    0.000    0.000    0.058    0.003 CsRoute.py:69(add_
> defaultroute)
> >>      11    0.000    0.000    0.001    0.000 sre_parse.py:686(parse)
> >>      21    0.000    0.000    0.018    0.001 CsNetfilter.py:206(del_rule)
> >>       1    0.000    0.000    0.009    0.009 merge.py:93(process)
> >>     329    0.000    0.000    0.000    0.000 {gc.disable}
> >>      44    0.000    0.000    0.000    0.000
> CsDatabag.py:147(get_gateway)
> >>      19    0.000    0.000    0.000    0.000 _weakrefset.py:58(__iter__)
> >>       2    0.000    0.000    0.015    0.007 CsHelper.py:211(service)
> >>      50    0.000    0.000    0.000    0.000
> sre_parse.py:139(__getitem__)
> >>      22    0.000    0.000    0.000    0.000
> CsAddress.py:343(setup_router_control)
> >>       1    0.000    0.000    0.000    0.000 {__import__}
> >>       1    0.000    0.000    0.000    0.000 CsFile.py:42(is_changed)
> >>      33    0.000    0.000    0.005    0.000 configure.py:110(create)
> >>      43    0.000    0.000    0.000    0.000 CsRoute.py:26(__init__)
> >>       1    0.000    0.000    0.006    0.006 CsApp.py:43(setup)
> >>      16    0.000    0.000  364.540   22.784 configure.py:766(
> processForwardRule)
> >>       7    0.000    0.000    0.001    0.000 __init__.py:257(load)
> >>       1    0.000    0.000    0.000    0.000 ascii.py:8(<module>)
> >>      44    0.000    0.000    0.000    0.000 stat.py:24(S_IFMT)
> >>   12/11    0.000    0.000    0.001    0.000 sre_parse.py:310(_parse_sub)
> >>      21    0.000    0.000    0.000    0.000 {format}
> >>       7    0.000    0.000    0.001    0.000 decoder.py:361(decode)
> >>      32    0.000    0.000    0.000    0.000
> CsAddress.py:133(get_network)
> >>      11    0.000    0.000    0.001    0.000 sre_compile.py:552(_code)
> >>       2    0.000    0.000    0.000    0.000 {posix.listdir}
> >>       1    0.000    0.000    0.018    0.018
> CsRedundant.py:74(_redundant_off)
> >>      61    0.000    0.000    0.000    0.000 CsConfig.py:39(get_cmdline_
> instance)
> >>       1    0.000    0.000    0.007    0.007 CsProcess.py:61(grep)
> >>      14    0.000    0.000    0.000    0.000 {method 'match' of
> '_sre.SRE_Pattern' objects}
> >>      71    0.000    0.000    0.000    0.000 CsConfig.py:51(get_fw)
> >>      40    0.000    0.000    0.000    0.000 {min}
> >>       1    0.000    0.000    0.007    0.007 CsAddress.py:32(compare)
> >>       1    0.000    0.000    0.000    0.000 {posix.rename}
> >>       1    0.000    0.000  548.846  548.846 update_config.py:50(process_
> file)
> >>      21    0.000    0.000    0.000    0.000 CsAddress.py:665(__init__)
> >>      22    0.000    0.000    0.000    0.000 CsAddress.py:237(
> waitfordevice)
> >>      43    0.000    0.000    0.000    0.000 {hex}
> >>       1    0.000    0.000    0.003    0.003 CsHelper.py:55(is_mounted)
> >>     125    0.000    0.000    0.000    0.000 {ord}
> >>       1    0.000    0.000    0.000    0.000 shutil.py:210(rmtree)
> >>      11    0.000    0.000    0.000    0.000 {_sre.compile}
> >>       1    0.000    0.000    0.003    0.003
> cs_firewallrules.py:22(merge)
> >>       7    0.000    0.000    0.001    0.000 __init__.py:293(loads)
> >>       3    0.000    0.000    0.000    0.000 CsAddress.py:604(compare)
> >>       1    0.000    0.000    0.013    0.013 configure.py:266(
> flushAllowAllEgressRules)
> >>      11    0.000    0.000    0.000    0.000 genericpath.py:15(exists)
> >>      10    0.000    0.000    0.000    0.000 _weakrefset.py:26(__exit__)
> >>       1    0.000    0.000    0.018    0.018 CsRedundant.py:67(set)
> >>       1    0.000    0.000    0.000    0.000 posixpath.py:139(islink)
> >>      23    0.000    0.000    0.000    0.000 CsAddress.py:592(getDevice)
> >>      21    0.000    0.000    0.000    0.000
> CsAddress.py:430(fw_vpcrouter)
> >>       1    0.000    0.000  548.846  548.846 update_config.py:114(mainp)
> >>      22    0.000    0.000    0.000    0.000 sre_compile.py:546(isstring)
> >>       1    0.000    0.000    0.004    0.004 __init__.py:193(dumps)
> >>      10    0.000    0.000    0.000    0.000 _weakrefset.py:20(__enter__)
> >>       1    0.000    0.000    0.000    0.000 encoder.py:212(iterencode)
> >>       8    0.000    0.000    0.000    0.000 sre_compile.py:428(_simple)
> >>    14/5    0.000    0.000    0.000    0.000 {issubclass}
> >>      10    0.000    0.000    0.000    0.000 _weakrefset.py:83(add)
> >>       1    0.000    0.000    0.000    0.000 CsRedundant.py:62(__init__)
> >>      11    0.000    0.000    0.000    0.000 sre_parse.py:187(__init__)
> >>       1    0.000    0.000    0.009    0.009 merge.py:87(__init__)
> >>      78    0.000    0.000    0.000    0.000 {method 'iteritems' of
> 'dict' objects}
> >>       1    0.000    0.000    0.000    0.000 configure.py:446(process)
> >>      22    0.000    0.000    0.000    0.000 CsAddress.py:265(setAddress)
> >>      32    0.000    0.000    0.000    0.000 sre_parse.py:135(__len__)
> >>       1    0.000    0.000    0.009    0.009 merge.py:273(load)
> >>      23    0.000    0.000    0.000    0.000 sre_parse.py:204(match)
> >>       1    0.000    0.000    0.000    0.000 __init__.py:71(search_
> function)
> >>      11    0.000    0.000    0.000    0.000 sre_parse.py:67(__init__)
> >>       1    0.000    0.000    0.000    0.000 merge.py:306(__moveFile)
> >>       1    0.000    0.000    0.000    0.000
> sre_compile.py:256(_optimize_charset)
> >>       1    0.000    0.000    0.000    0.000
> __init__.py:211(_acquireLock)
> >>      10    0.000    0.000    0.000    0.000 merge.py:46(__init__)
> >>       1    0.000    0.000    0.000    0.000 {method 'split' of
> '_sre.SRE_Pattern' objects}
> >>      18    0.000    0.000    0.000    0.000 {method 'extend' of 'list'
> objects}
> >>       1    0.000    0.000    0.007    0.007 CsApp.py:77(start)
> >>      10    0.000    0.000    0.000    0.000 _weakrefset.py:16(__init__)
> >>       2    0.000    0.000    0.000    0.000 CsHelper.py:73(rm)
> >>       1    0.000    0.000    0.000    0.000 CsApp.py:97(add_firewall_
> rules)
> >>       1    0.000    0.000    0.003    0.003 CsHelper.py:68(umount_tmpfs)
> >>       1    0.000    0.000    0.000    0.000 {posix.rmdir}
> >>       1    0.000    0.000    0.000    0.000 codecs.py:77(__new__)
> >>       1    0.000    0.000    0.004    0.004 merge.py:65(save)
> >>      20    0.000    0.000    0.000    0.000 sre_parse.py:90(__init__)
> >>       2    0.000    0.000    0.000    0.000 CsConfig.py:78(use_extdns)
> >>       1    0.000    0.000    0.000    0.000 {posix.lstat}
> >>       1    0.000    0.000    0.019    0.019 CsNetfilter.py:82(__init__)
> >>       1    0.000    0.000    0.000    0.000 CsHelper.py:77(rmdir)
> >>       9    0.000    0.000    0.000    0.000 {method '__subclasses__' of
> 'type' objects}
> >>      10    0.000    0.000    0.000    0.000
> _abcoll.py:98(__subclasshook__)
> >>      10    0.000    0.000    0.000    0.000 _weakrefset.py:52(_commit_
> removals)
> >>      14    0.000    0.000    0.000    0.000 {method 'end' of
> '_sre.SRE_Match' objects}
> >>       1    0.000    0.000    0.000    0.000 __init__.py:1590(error)
> >>       1    0.000    0.000    0.000    0.000 re.py:168(split)
> >>       1    0.000    0.000    0.000    0.000 __init__.py:1176(error)
> >>       1    0.000    0.000    0.000    0.000 {posix.access}
> >>       1    0.000    0.000    0.000    0.000 sre_compile.py:228(_compile_
> charset)
> >>       1    0.000    0.000    0.000    0.000
> CsAddress.py:219(configure_rp)
> >>      10    0.000    0.000    0.000    0.000 merge.py:79(setKey)
> >>       2    0.000    0.000    0.000    0.000 CsApp.py:26(__init__)
> >>       3    0.000    0.000    0.000    0.000 CsNetfilter.py:68(add)
> >>       8    0.000    0.000    0.000    0.000
> sre_parse.py:143(__setitem__)
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:72(has_metadata)
> >>       1    0.000    0.000    0.000    0.000 ascii.py:41(getregentry)
> >>       1    0.000    0.000  548.846  548.846 <string>:1(<module>)
> >>       2    0.000    0.000    0.000    0.000 CsConfig.py:69(has_dns)
> >>       1    0.000    0.000    0.000    0.000 CsHelper.py:235(copy_if_
> needed)
> >>       1    0.000    0.000    0.000    0.000 {round}
> >>      10    0.000    0.000    0.000    0.000 merge.py:76(getDataBag)
> >>       2    0.000    0.000    0.000    0.000 CsAddress.py:335(get_ip_
> address)
> >>       4    0.000    0.000    0.000    0.000 {method 'update' of 'dict'
> objects}
> >>       1    0.000    0.000    0.000    0.000 sre_parse.py:83(closegroup)
> >>       2    0.000    0.000    0.000    0.000 _weakrefset.py:36(__init__)
> >>       1    0.000    0.000    0.000    0.000 {method 'translate' of
> 'str' objects}
> >>       1    0.000    0.000    0.000    0.000 encoder.py:272(_make_
> iterencode)
> >>       1    0.000    0.000    0.000    0.000 configure.py:615(process)
> >>       1    0.000    0.000    0.000    0.000
> __init__.py:1500(basicConfig)
> >>       1    0.000    0.000    0.001    0.001 CsConfig.py:36(set_address)
> >>       2    0.000    0.000    0.000    0.000 CsConfig.py:63(is_router)
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:75(get_domain)
> >>       1    0.000    0.000    0.000    0.000
> __init__.py:220(_releaseLock)
> >>       1    0.000    0.000    0.000    0.000 __init__.py:49(normalize_
> encoding)
> >>       1    0.000    0.000    0.000    0.000 CsNetfilter.py:64(__init__)
> >>       1    0.000    0.000    0.000    0.000
> CsLoadBalancer.py:32(process)
> >>       1    0.000    0.000    0.000    0.000 {built-in method __new__ of
> type object at 0x93c580}
> >>       3    0.000    0.000    0.000    0.000 {method 'setdefault' of
> 'dict' objects}
> >>       1    0.000    0.000    0.003    0.003 merge.py:179(process_
> firewallrules)
> >>       1    0.000    0.000    0.000    0.000 encoder.py:101(__init__)
> >>       1    0.000    0.000    0.000    0.000 sre_parse.py:266(_escape)
> >>       1    0.000    0.000    0.000    0.000 CsFile.py:26(__init__)
> >>       1    0.000    0.000    0.000    0.000 CsAddress.py:249(list)
> >>       1    0.000    0.000    0.000    0.000 ascii.py:28(StreamWriter)
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:33(__init__)
> >>       1    0.000    0.000    0.000    0.000 CsApp.py:74(__init__)
> >>       1    0.000    0.000    0.000    0.000 CsNetfilter.py:26(__init__)
> >>       1    0.000    0.000    0.000    0.000 {method 'find' of
> 'bytearray' objects}
> >>       1    0.000    0.000    0.000    0.000 ascii.py:13(Codec)
> >>       1    0.000    0.000    0.000    0.000 ascii.py:20(
> IncrementalEncoder)
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:54(get_logger)
> >>       1    0.000    0.000    0.000    0.000 merge.py:294(setFile)
> >>       1    0.000    0.000    0.000    0.000 sre_parse.py:72(opengroup)
> >>       2    0.000    0.000    0.000    0.000 shutil.py:222(onerror)
> >>       1    0.000    0.000    0.000    0.000 CsProcess.py:27(__init__)
> >>       1    0.000    0.000    0.000    0.000 {sys.exit}
> >>       1    0.000    0.000    0.000    0.000 ascii.py:24(
> IncrementalDecoder)
> >>       1    0.000    0.000    0.000    0.000 CsDatabag.py:88(get_domain)
> >>       1    0.000    0.000    0.000    0.000 ascii.py:34(StreamConverter)
> >>       1    0.000    0.000    0.000    0.000 {method 'remove' of 'list'
> objects}
> >>       2    0.000    0.000    0.000    0.000 {sys.exc_info}
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:66(is_dhcp)
> >>       1    0.000    0.000    0.000    0.000 {any}
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:57(get_level)
> >>       1    0.000    0.000    0.000    0.000 {method 'disable' of
> '_lsprof.Profiler' objects}
> >>       1    0.000    0.000    0.000    0.000 ascii.py:31(StreamReader)
> >>       1    0.000    0.000    0.000    0.000 CsConfig.py:95(get_format)
> >>
> >>
> >> Fri Aug 19 11:27:50 2016    d:\temp\py_pro.data
> >>
> >>        614887355 function calls (614884838 primitive calls) in 548.846
> seconds
> >>
> >>  Ordered by: cumulative time
> >>  List reduced from 404 to 3 due to restriction <3>
> >>
> >>  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
> >>       1    0.000    0.000  548.846  548.846 <string>:1(<module>)
> >>       1    0.000    0.000  548.846  548.846 update_config.py:114(mainp)
> >>       1    0.000    0.000  548.846  548.846 update_config.py:50(process_
> file)
> >>
> >>
> >>
> >>
> >>
> >>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message