httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian McCallister <bri...@skife.org>
Subject Re: patch to mod_lua(support server scope, better performance)
Date Mon, 14 Mar 2011 23:43:55 GMT
On Thu, Mar 10, 2011 at 9:32 AM, Brian McCallister <brianm@skife.org> wrote:
> Thank you, I'll take a look into this today!

Thank you, patch applied!

-Brian

>
> 2011/3/10 zhiguo zhao <zhaozg@gmail.com>:
>>
>> Hi,
>>    Some days I post my first
>> letter(http://mail-archives.apache.org/mod_mbox/httpd-dev/201103.mbox/raw/%3CAANLkTimfg4yeGExZXNP=YjybjDRmHRQPB4jO--sN6qqr@mail.gmail.com%3E)
>> in http-dev, but the letter can be send to every's box with some reason.,
>>    And now I update mod_lua to support lua server scope with
>> apr_reslist_apis, and I do some test with ab on my Pc.
>>    I got better performance. (75 -> 227 -> 277)  #/s
>>    Change please see patch file, I hope this patch can be merge in svn..  :)
>> mod_lua in svn,head version
>> -----------------------------------------------------------------------
>> This is ApacheBench, Version 2.3 <$Revision: 1004962 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> Benchmarking localhost (be patient)
>>
>> Server Software:        Apache/2.3.12-dev
>> Server Hostname:        localhost
>> Server Port:            80
>> Document Path:          /simple.lua
>> Document Length:        9 bytes
>> Concurrency Level:      50
>> Time taken for tests:   13.310 seconds
>> Complete requests:      1000
>> Failed requests:        0
>> Write errors:           0
>> Total transferred:      164000 bytes
>> HTML transferred:       9000 bytes
>> Requests per second:    75.13 [#/sec] (mean)
>> Time per request:       665.500 [ms] (mean)
>> Time per request:       13.310 [ms] (mean, across all concurrent requests)
>> Transfer rate:          12.03 [Kbytes/sec] received
>> Connection Times (ms)
>>               min  mean[+/-sd] median   max
>> Connect:        0    1   0.5      1       5
>> Processing:    89  662 1946.3    213    9253
>> Waiting:       89  658 1938.1    212    9236
>> Total:         90  662 1946.3    214    9253
>> Percentage of the requests served within a certain time (ms)
>>   50%    214
>>   66%    221
>>   75%    228
>>   80%    235
>>   90%    274
>>   95%   9012
>>   98%   9147
>>   99%   9152
>>  100%   9253 (longest request)
>>  mod_lua updated, Lua_Scope is once
>> -----------------------------------------------------------------------
>> This is ApacheBench, Version 2.3 <$Revision: 1004962 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> Benchmarking localhost (be patient)
>>
>> Server Software:        Apache/2.3.12-dev
>> Server Hostname:        localhost
>> Server Port:            80
>> Document Path:          /simple.lua
>> Document Length:        9 bytes
>> Concurrency Level:      50
>> Time taken for tests:   4.393 seconds
>> Complete requests:      1000
>> Failed requests:        0
>> Write errors:           0
>> Total transferred:      164000 bytes
>> HTML transferred:       9000 bytes
>> Requests per second:    227.63 [#/sec] (mean)
>> Time per request:       219.650 [ms] (mean)
>> Time per request:       4.393 [ms] (mean, across all concurrent requests)
>> Transfer rate:          36.46 [Kbytes/sec] received
>> Connection Times (ms)
>>               min  mean[+/-sd] median   max
>> Connect:        0    1   0.9      1      14
>> Processing:    58  215  34.1    216     317
>> Waiting:       56  210  36.6    214     314
>> Total:         59  216  34.1    217     318
>> Percentage of the requests served within a certain time (ms)
>>   50%    217
>>   66%    227
>>   75%    235
>>   80%    240
>>   90%    251
>>   95%    265
>>   98%    282
>>   99%    301
>>  100%    318 (longest request)
>>
>> mod_lua updated, with Lua_Scope server 10 50
>> ----------------------------------------------------------------------
>> This is ApacheBench, Version 2.3 <$Revision: 1004962 $>
>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
>> Licensed to The Apache Software Foundation, http://www.apache.org/
>> Benchmarking localhost (be patient)
>>
>> Server Software:        Apache/2.3.12-dev
>> Server Hostname:        localhost
>> Server Port:            80
>> Document Path:          /simple.lua
>> Document Length:        9 bytes
>> Concurrency Level:      50
>> Time taken for tests:   3.606 seconds
>> Complete requests:      1000
>> Failed requests:        0
>> Write errors:           0
>> Total transferred:      164000 bytes
>> HTML transferred:       9000 bytes
>> Requests per second:    277.32 [#/sec] (mean)
>> Time per request:       180.300 [ms] (mean)
>> Time per request:       3.606 [ms] (mean, across all concurrent requests)
>> Transfer rate:          44.41 [Kbytes/sec] received
>> Connection Times (ms)
>>               min  mean[+/-sd] median   max
>> Connect:        0    1   0.6      1       7
>> Processing:    43  176  24.9    175     272
>> Waiting:       36  174  27.3    173     262
>> Total:         44  177  24.9    176     272
>> Percentage of the requests served within a certain time (ms)
>>   50%    176
>>   66%    184
>>   75%    189
>>   80%    192
>>   90%    201
>>   95%    216
>>   98%    233
>>   99%    242
>>  100%    272 (longest request)
>> ----------------------------------patch
>> file---------------------------------------------------------
>> Index: lua_vmprep.c
>> ===================================================================
>> --- lua_vmprep.c (版本 1080084)
>> +++ lua_vmprep.c (工作副本)
>> @@ -288,79 +288,141 @@
>>
>>  #endif
>>
>> -AP_LUA_DECLARE(lua_State*)ap_lua_get_lua_state(apr_pool_t *lifecycle_pool,
>> -                                               ap_lua_vm_spec
*spec,
>> -                                               apr_array_header_t
>> *package_paths,
>> -                                               apr_array_header_t
>> *package_cpaths,
>> -                                               ap_lua_state_open_callback
>> cb,
>> -                                               void *btn)
>> +static apr_status_t vm_construct(void **vm, void *params, apr_pool_t
>> *lifecycle_pool)
>>  {
>> + ap_lua_vm_spec *spec = params;
>>
>> -    lua_State *L;
>> -
>> -    if (!apr_pool_userdata_get((void **) &L, spec->file, lifecycle_pool))
{
>> -        ap_log_perror(APLOG_MARK, APLOG_DEBUG, 0, lifecycle_pool,
>> -                      "creating lua_State with file %s", spec->file);
>> -        /* not available, so create */
>> -        L = luaL_newstate();
>> + lua_State* L = luaL_newstate();
>>  #ifdef AP_ENABLE_LUAJIT
>> -        luaopen_jit(L);
>> + luaopen_jit(L);
>>  #endif
>> -        luaL_openlibs(L);
>> -        if (package_paths) {
>> -            munge_path(L, "path", "?.lua", "./?.lua", lifecycle_pool,
>> -                       package_paths, spec->file);
>> -        }
>> -        if (package_cpaths) {
>> -            munge_path(L, "cpath", "?.so", "./?.so", lifecycle_pool,
>> -                       package_cpaths, spec->file);
>> -        }
>> + luaL_openlibs(L);
>> + if (spec->package_paths) {
>> + munge_path(L, "path", "?.lua", "./?.lua", lifecycle_pool,
>> + spec->package_paths, spec->file);
>> + }
>> + if (spec->package_cpaths) {
>> + munge_path(L, "cpath", "?.so", "./?.so", lifecycle_pool,
>> + spec->package_cpaths, spec->file);
>> + }
>>
>> -        if (cb) {
>> -            cb(L, lifecycle_pool, btn);
>> -        }
>> + if (spec->cb) {
>> + spec->cb(L, lifecycle_pool, spec->cb_arg);
>> + }
>>
>> -        apr_pool_userdata_set(L, spec->file, &cleanup_lua, lifecycle_pool);
>>
>> -        if (spec->bytecode && spec->bytecode_len > 0) {
>> -            luaL_loadbuffer(L, spec->bytecode, spec->bytecode_len,
>> spec->file);
>> -            lua_pcall(L, 0, LUA_MULTRET, 0);
>> -        }
>> -        else {
>> -            int rc;
>> -            ap_log_perror(APLOG_MARK, APLOG_DEBUG, 0, lifecycle_pool,
>> -                          "loading lua file %s", spec->file);
>> -            rc = luaL_loadfile(L, spec->file);
>> -            if (rc != 0) {
>> -                char *err;
>> -                switch (rc) {
>> -                case LUA_ERRSYNTAX:
>> -                    err = "syntax error";
>> -                    break;
>> -                case LUA_ERRMEM:
>> -                    err = "memory allocation error";
>> -                    break;
>> -                case LUA_ERRFILE:
>> -                    err = "error opening or reading file";
>> -                    break;
>> -                default:
>> -                    err = "unknown error";
>> -                    break;
>> -                }
>> -                ap_log_perror(APLOG_MARK, APLOG_ERR, 0, lifecycle_pool,
>> -                              "Loading lua file %s: %s",
>> -                              spec->file, err);
>> -                return NULL;
>> -            }
>> -            lua_pcall(L, 0, LUA_MULTRET, 0);
>> -        }
>> + if (spec->bytecode && spec->bytecode_len > 0) {
>> + luaL_loadbuffer(L, spec->bytecode, spec->bytecode_len, spec->file);
>> + lua_pcall(L, 0, LUA_MULTRET, 0);
>> + }
>> + else {
>> + int rc;
>> + ap_log_perror(APLOG_MARK, APLOG_DEBUG, 0, lifecycle_pool,
>> + "loading lua file %s", spec->file);
>> + rc = luaL_loadfile(L, spec->file);
>> + if (rc != 0) {
>> + char *err;
>> + switch (rc) {
>> + case LUA_ERRSYNTAX:
>> + err = "syntax error";
>> + break;
>> + case LUA_ERRMEM:
>> + err = "memory allocation error";
>> + break;
>> + case LUA_ERRFILE:
>> + err = "error opening or reading file";
>> + break;
>> + default:
>> + err = "unknown error";
>> + break;
>> + }
>> + ap_log_perror(APLOG_MARK, APLOG_ERR, 0, lifecycle_pool,
>> + "Loading lua file %s: %s",
>> + spec->file, err);
>> + return APR_EBADF;
>> + }
>> + lua_pcall(L, 0, LUA_MULTRET, 0);
>> + }
>>
>>  #ifdef AP_ENABLE_LUAJIT
>> -        loadjitmodule(L, lifecycle_pool);
>> + loadjitmodule(L, lifecycle_pool);
>>  #endif
>> -        lua_pushlightuserdata(L, lifecycle_pool);
>> -        lua_setfield(L, LUA_REGISTRYINDEX, "Apache2.Wombat.pool");
>> -    }
>> + lua_pushlightuserdata(L, lifecycle_pool);
>> + lua_setfield(L, LUA_REGISTRYINDEX, "Apache2.Wombat.pool");
>> + *vm = L;
>>
>> + return APR_SUCCESS;
>> +}
>> +
>> +static apr_status_t vm_destruct(void *vm, void *params, apr_pool_t *pool)
>> +{
>> + lua_State *L = (lua_State *)vm;
>> +
>> + (void*)params;
>> + (void*)pool;
>> +
>> + cleanup_lua(L);
>> +
>> + return APR_SUCCESS;
>> +}
>> +
>> +static apr_status_t vm_release(lua_State* vm)
>> +{
>> + apr_reslist_t* reslist;
>> + lua_pushlightuserdata(vm,vm);
>> + lua_rawget(vm,LUA_REGISTRYINDEX);
>> + reslist = (apr_reslist_t*)lua_topointer(vm,-1);
>> +
>> + return apr_reslist_release(reslist, vm);
>> +}
>> +
>> +static apr_status_t vm_reslist_destroy(void *data)
>> +{
>> + return apr_reslist_destroy(data);
>> +}
>> +
>> +AP_LUA_DECLARE(lua_State*)ap_lua_get_lua_state(apr_pool_t *lifecycle_pool,
>> +                                               ap_lua_vm_spec
*spec)
>> +{
>> +    lua_State *L = NULL;
>> +
>> + if (spec->scope == APL_SCOPE_SERVER) {
>> + apr_reslist_t *reslist;
>> +
>> + if (apr_pool_userdata_get(&reslist,"mod_lua",spec->pool)==APR_SUCCESS) {
>> + if(reslist==NULL) {
>> + if(apr_reslist_create(&reslist,
>> + spec->vm_server_pool_min,
>> + spec->vm_server_pool_max,
>> + spec->vm_server_pool_max,
>> + 0,
>> + vm_construct,
>> + vm_destruct,
>> + spec,
>> + spec->pool)!=APR_SUCCESS)
>> + return NULL;
>> +
>> + apr_pool_userdata_set(reslist, "mod_lua", vm_reslist_destroy, spec->pool);
>> + }
>> + apr_reslist_acquire(reslist, &L);
>> + lua_pushlightuserdata(L, L);
>> + lua_pushlightuserdata(L, reslist);
>> + lua_rawset(L,LUA_REGISTRYINDEX);
>> + apr_pool_userdata_set(L, spec->file, vm_release, lifecycle_pool);
>> + }
>> + } else {
>> + if (apr_pool_userdata_get((void **) &L, spec->file,
>> lifecycle_pool)==APR_SUCCESS) {
>> +
>> + if(L==NULL) {
>> + ap_log_perror(APLOG_MARK, APLOG_DEBUG, 0, lifecycle_pool,
>> + "creating lua_State with file %s", spec->file);
>> + /* not available, so create */
>> +
>> + if(!vm_construct(&L, spec, lifecycle_pool))
>> + apr_pool_userdata_set(L, spec->file, &cleanup_lua, lifecycle_pool);
>> + }
>> + }
>> + }
>> +
>>      return L;
>>  }
>> Index: lua_vmprep.h
>> ===================================================================
>> --- lua_vmprep.h (版本 1080084)
>> +++ lua_vmprep.h (工作副本)
>> @@ -43,7 +43,24 @@
>>  #define APL_SCOPE_CONN          3
>>  #define APL_SCOPE_SERVER        4
>>
>> +
>>  /**
>> + * the ap_lua_?getvm family of functions is used to create and/or obtain
>> + * a handle to a lua state. If there is not an extant vm matching the
>> + * spec then a new one is created.
>> + */
>> +/* lua_State* ap_lua_rgetvm(request_rec *r, ap_lua_vm_spec *spec); */
>> +
>> +/* returns NULL if the spec requires a request scope */
>> +/* lua_State* ap_lua_cgetvm(conn_rec *r, ap_lua_vm_spec *spec);*/
>> +
>> +/* returns NULL if the spec requires a request scope or conn scope */
>> +/* lua_State* ap_lua_sgetvm(server_rec *r, ap_lua_vm_spec *spec); */
>> +
>> +typedef void (*ap_lua_state_open_callback) (lua_State *L, apr_pool_t *p,
>> +                                             void *ctx);
>> +
>> +/**
>>   * Specification for a lua virtual machine
>>   */
>>  typedef struct
>> @@ -61,7 +78,11 @@
>>
>>      /* APL_SCOPE_ONCE | APL_SCOPE_REQUEST | APL_SCOPE_CONN |
>> APL_SCOPE_SERVER */
>>      int scope;
>> + unsigned int vm_server_pool_min;
>> + unsigned int vm_server_pool_max;
>>
>> + ap_lua_state_open_callback cb;
>> + void* cb_arg;
>>      /* pool to use for lifecycle if APL_SCOPE_ONCE is set, otherwise unused
>> */
>>      apr_pool_t *pool;
>>
>> @@ -102,22 +123,6 @@
>>   */
>>  AP_LUA_DECLARE(void) ap_lua_load_apache2_lmodule(lua_State *L);
>>
>> -/**
>> - * the ap_lua_?getvm family of functions is used to create and/or obtain
>> - * a handle to a lua state. If there is not an extant vm matching the
>> - * spec then a new one is created.
>> - */
>> -/* lua_State* ap_lua_rgetvm(request_rec *r, ap_lua_vm_spec *spec); */
>> -
>> -/* returns NULL if the spec requires a request scope */
>> -/* lua_State* ap_lua_cgetvm(conn_rec *r, ap_lua_vm_spec *spec);*/
>> -
>> -/* returns NULL if the spec requires a request scope or conn scope */
>> -/* lua_State* ap_lua_sgetvm(server_rec *r, ap_lua_vm_spec *spec); */
>> -
>> -typedef void (*ap_lua_state_open_callback) (lua_State *L, apr_pool_t *p,
>> -                                             void *ctx);
>> -
>>  /*
>>   * alternate means of getting lua_State (preferred eventually)
>>   * Obtain a lua_State which has loaded file and is associated with
>> lifecycle_pool
>> @@ -131,11 +136,7 @@
>>   * @ctx a baton passed to cb
>>   */
>>  AP_LUA_DECLARE(lua_State*) ap_lua_get_lua_state(apr_pool_t *lifecycle_pool,
>> -                                                ap_lua_vm_spec
*spec,
>> -                                                apr_array_header_t
>> *package_paths,
>> -                                                apr_array_header_t
>> *package_cpaths,
>> -                                                ap_lua_state_open_callback
>> cb,
>> -                                                void *btn);
>> +                                                ap_lua_vm_spec
*spec);
>>
>>
>>
>> Index: mod_lua.c
>> ===================================================================
>> --- mod_lua.c (版本 1080084)
>> +++ mod_lua.c (工作副本)
>> @@ -121,9 +121,15 @@
>>              d = apr_palloc(r->pool, sizeof(mapped_request_details));
>>              spec = apr_pcalloc(r->pool, sizeof(ap_lua_vm_spec));
>>              spec->scope = dcfg->vm_scope;
>> -            spec->pool = r->pool;
>> + spec->pool = spec->scope==APL_SCOPE_SERVER ? cfg->pool : r->pool;
>>              spec->file = r->filename;
>>              spec->code_cache_style = dcfg->code_cache_style;
>> + spec->package_paths = cfg->package_paths;
>> + spec->package_cpaths = cfg->package_cpaths;
>> + spec->vm_server_pool_min = cfg->vm_server_pool_min;
>> + spec->vm_server_pool_max = cfg->vm_server_pool_max;
>> + spec->cb = &lua_open_callback;
>> + spec->cb_arg = NULL;
>>              d->spec = spec;
>>              d->function_name = "handle";
>>          }
>> @@ -135,10 +141,7 @@
>>                        d->spec->file,
>>                        d->function_name);
>>          L = ap_lua_get_lua_state(r->pool,
>> -                              d->spec,
>> -                              cfg->package_paths,
>> -                              cfg->package_cpaths,
>> -                              &lua_open_callback, NULL);
>> +                              d->spec);
>>
>>          if (!L) {
>>              /* TODO annotate spec with failure reason */
>> @@ -246,17 +249,20 @@
>>              spec->file = hook_spec->file_name;
>>              spec->code_cache_style = hook_spec->code_cache_style;
>>              spec->scope = hook_spec->scope;
>> + spec->vm_server_pool_min = cfg->vm_server_pool_min;
>> + spec->vm_server_pool_max = cfg->vm_server_pool_max;
>>              spec->bytecode = hook_spec->bytecode;
>>              spec->bytecode_len = hook_spec->bytecode_len;
>> -            spec->pool = r->pool;
>> +            spec->pool = spec->scope==APL_SCOPE_SERVER ? cfg->pool
:
>> r->pool;
>> + spec->package_paths = cfg->package_paths;
>> + spec->package_cpaths = cfg->package_cpaths;
>> + spec->cb = &lua_open_callback;
>> + spec->cb_arg = NULL;
>>
>>              apr_filepath_merge(&spec->file, server_cfg->root_path,
>>                                 spec->file, APR_FILEPATH_NOTRELATIVE,
>> r->pool);
>>              L = ap_lua_get_lua_state(r->pool,
>> -                                  spec,
>> -                                  cfg->package_paths,
>> -                                  cfg->package_cpaths,
>> -                                  &lua_open_callback, NULL);
>> +                                  spec);
>>
>>
>>
>>
>

Mime
View raw message