Mailing-List: contact dev-help@perl.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (herse.apache.org: local policy)
Subject: Re: offtopic: Embedding Perl in C++: Help needed
From: Heiko Jansen <jansen@hbz-nrw.de>
To: "Philip M. Gollucci" <pgollucci@p6m7g8.com>
Cc: dev@perl.apache.org
In-Reply-To: <4628E992.8070406@p6m7g8.com>
References: <1177063898.3850.86.camel@tron.hbz-nrw.de>
	 <4628E992.8070406@p6m7g8.com>
Content-Type: text/plain; charset=UTF-8
Organization: hbz NRW
Date: Fri, 20 Apr 2007 17:24:50 +0000
Message-Id: <1177089890.3850.163.camel@tron.hbz-nrw.de>
Mime-Version: 1.0
Content-Transfer-Encoding: 8bit

Am Freitag, den 20.04.2007, 09:25 -0700 schrieb Philip M. Gollucci:
>  > I have to maintain a multithreaded (150+) C++ application (running as
>  > 64Bit App on Solaris 10, SPARC, SunCC) which uses embedded perl to allow
>  > for easy data modification (utf-8 encoded strings) in certain places.
>  > This does work in general, but every now and then the application
>  > crashes although the same perl code has been run for a few days.
> 
> I only briefly read this, but you're not hitting a system limit of some 
> kind are you ?  Max open files, sockets, processes ?
> 
> sockstat, systat, fstat, lsof ... etc ?

The proc has a limit of 4096 file handles and I have never seen it come
even close to that limit.
Inside the perl scripts I do not open any files, databases handles or
the like (neither reading nor writing).
The server has 16GB of real memory and I have never seen it run out of
memory.
The data exchange with the scripts is done via the stack, as seen in the
initial post.

> Are you sure everything is cleaning up after itself correctly and 
> completely ?

I see quite a number of "Scalars leaked" message on STDERR but have
never been able to track that down.
The scripts are fairly simple so I´m quite sure there are no leftovers
(also the interpreter is cloned, used and thrown away - there should be
nothing left?!)

The only thing I didn´t mention so far is, that I remember seeing output
like this sometimes before the app crashed:

-- snip --
Use of uninitialized value in pattern match (m//) at /tools/lib/perl5/5.8.8/utf8_heavy.pl line 211.
Use of uninitialized value in scalar assignment at /tools/lib/perl5/5.8.8/utf8_heavy.pl line 227.
Use of uninitialized value in pattern match (m//) at /tools/lib/perl5/5.8.8/utf8_heavy.pl line 228.
-- / snip --

The text above is from today. This is my utf8_heavy.pl:

-- snip --
    188     my $extras;
    189     my $bits = 0;
    190
    191     my $ORIG = $list;
    192     if ($list) {
    193         my @tmp = split(/^/m, $list);
    194         my %seen;
    195         no warnings;
    196         $extras = join '', grep /^[^0-9a-fA-F]/, @tmp;
    197         $list = join '',
    198             map  { $_->[1] }
    199             sort { $a->[0] <=> $b->[0] }
    200             map  { /^([0-9a-fA-F]+)/; [ hex($1), $_ ] }
    201             grep { /^([0-9a-fA-F]+)/ and not $seen{$1}++ } @tmp; # XXX doesn't do ranges right
    202     }
    203
    204     if ($none) {
    205         my $hextra = sprintf "%04x", $none + 1;
    206         $list =~ s/\tXXXX$/\t$hextra/mg;
    207     }
    208
    209     if ($minbits < 32) {
    210         my $top = 0;
    211         while ($list =~ /^([0-9a-fA-F]+)(?:[\t]([0-9a-fA-F]+)?)(?:[ \t]([0-9a-fA-F]+))?/mg) {
    212             my $min = hex $1;
    213             my $max = defined $2 ? hex $2 : $min;
    214             my $val = defined $3 ? hex $3 : 0;
    215             $val += $max - $min if defined $3;
    216             $top = $val if $val > $top;
    217         }
    218         $bits =
    219             $top > 0xffff ? 32 :
    220             $top > 0xff ? 16 :
    221             $top > 1 ? 8 : 1
    222     }
    223     $bits = $minbits if $bits < $minbits;
    224
    225     my @extras;
    226     for my $x ($extras) {
    227         pos $x = 0;
    228         while ($x =~ /^([^0-9a-fA-F\n])(.*)/mg) {
-- / snip --

I don`t understand how the values can be undefined but maybe my perl is
even worse than my english.

The scripts should always get some data to work on. I´m not sure though,
that it may never be invalid utf-8 - but since the data is parsed from
and to XML I should have seen log entries on that (unless the scripts
always die on invalid utf-8...)

If there is anything else I could check, please tell me.

Many thanks
heiko


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@perl.apache.org
For additional commands, e-mail: dev-help@perl.apache.org