Clearly the logs must have rolled then?  Either that or you are using a broken jdk.

Karl


On Wed, Apr 15, 2015 at 7:37 AM, Kamil Żyta <kamil.zyta@pwr.edu.pl> wrote:
On Wed, Apr 15, 2015 at 07:27:56AM -0400, Karl Wright wrote:
> Hi Kamil:
>
> kawright@duck76:/data/kawright/analysis$ gzip --version
> gzip 1.4
> Copyright (C) 2007 Free Software Foundation, Inc.
> Copyright (C) 1993 Jean-loup Gailly.
> This is free software.  You may redistribute copies of it under the terms of
> the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by Jean-loup Gailly.
> kawright@duck76:/data/kawright/analysis$
>
>
> But in any case the key part of the stack trace is further down, probably
> MUCH further down.
>
> If I were you, I'd unzip the whole log and use head, tail, and grep to find
> where the exception trace ends.

I use grep -v and send you logs before but you don't belive me.
It's all mcf logs http://pastebin.com/T54NKwTh http://pastebin.com/uMxaUnGi

K

>
> On Wed, Apr 15, 2015 at 7:18 AM, Kamil Żyta <kamil.zyta@pwr.edu.pl> wrote:
>
> > hmm, try tar -xf manifoldcf.log.gz or maybe zless?
> > It's work for me with:
> > > gzip --version
> > gzip 1.6
> >
> > For sure I attached uncompressed file.
> >
> > K
> >
> > On Wed, Apr 15, 2015 at 07:10:07AM -0400, Karl Wright wrote:
> > > Hi Kamil,
> > >
> > > >>>>>>
> > > kawright@duck76:~$ cd /data/kawright/analysis/
> > > kawright@duck76:/data/kawright/analysis$ gunzip manifoldcf.log.gz
> > >
> > > gzip: manifoldcf.log.gz: invalid compressed data--crc error
> > >
> > > gzip: manifoldcf.log.gz: invalid compressed data--length error
> > > kawright@duck76:/data/kawright/analysis$
> > >
> > > <<<<<<
> > >
> > > Karl
> > >
> > >
> > > On Wed, Apr 15, 2015 at 6:41 AM, Kamil Żyta <kamil.zyta@pwr.edu.pl>
> > wrote:
> > >
> > > > these 1k lines are the same. I attached full manifoldcf.log.
> > > >
> > > > K
> > > >
> > > > On Wed, Apr 15, 2015 at 06:33:06AM -0400, Karl Wright wrote:
> > > > > Hi Kamil,
> > > > >
> > > > > There is a complete trace in there, believe me.  The JVM did not
> > say: "
> > > > (...)
> > > > > ~1k lines".  What I need is at the bottom of those 1K lines.
> > > > >
> > > > > Karl
> > > > >
> > > > >
> > > > > On Wed, Apr 15, 2015 at 6:23 AM, Kamil Żyta <kamil.zyta@pwr.edu.pl>
> > > > wrote:
> > > > >
> > > > > > How can I provide usable stack trace? I can only copy what logs
> > says.
> > > > > > Now it's a lot of:
> > > > > > FATAL 2015-04-15 12:14:35,645 (Worker thread '5') - Error tossed:
> > null
> > > > > > java.lang.StackOverflowError
> > > > > >         at
> > > > java.util.regex.Pattern$CharProperty.match(Pattern.java:3776)
> > > > > >         at java.util.regex.Pattern$Curly.match0(Pattern.java:4250)
> > > > > >         at java.util.regex.Pattern$Curly.match0(Pattern.java:4263)
> > > > > >         (...) ~1k lines
> > > > > >
> > > > > > for continuous job but agents is not exiting. Propably this two
> > errors
> > > > > > below isn't correlated (patterns and agents oom).
> > > > > >
> > > > > > K
> > > > > >
> > > > > > On Tue, Apr 14, 2015 at 05:28:18PM -0400, Karl Wright wrote:
> > > > > > > Without some kind of usable stack trace I can't really help
> > you.  It
> > > > > > looks
> > > > > > > like some regular expression is going completely haywire, but I
> > have
> > > > no
> > > > > > > idea which one.
> > > > > > >
> > > > > > > Karl
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 14, 2015 at 4:31 PM, Kamil Żyta <
> > kamil.zyta@pwr.edu.pl>
> > > > > > wrote:
> > > > > > >
> > > > > > > > On Tue, Apr 14, 2015 at 04:12:55PM -0400, Karl Wright wrote:
> > > > > > > > > Hi Kamil,
> > > > > > > > >
> > > > > > > > > Without the bottom of the stack trace, I can't even tell
> > what it
> > > > is
> > > > > > > > doing.
> > > > > > > > > Where are you supplying a regular expression?
> > > > > > > >
> > > > > > > > It's all I have, the only regular expression is in 'Paths':
> > > > > > > > 3. Exclude file(s) or directory(s) matching */.*
> > > > > > > >
> > > > > > > > I found files (~500MB, logs) where solr logs ends,
> > > > > > > > exclude them solves the problem. mcf use tika for extracting
> > > > > > > > and only /update to solr, these files causes problem befor,
> > > > > > > > when using solr for extract docs. Now mcf dies and I do not
> > even
> > > > know
> > > > > > why.
> > > > > > > >
> > > > > > > > K
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Running out of memory might be a side effect of running out
> > of
> > > > stack.
> > > > > > > > >
> > > > > > > > > Karl
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Apr 14, 2015 at 2:49 PM, Kamil Żyta <
> > > > kamil.zyta@pwr.edu.pl>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > > agent process exit with:
> > > > > > > > > >
> > > > > > > > > > agents process ran out of memory - shutting down
> > > > > > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > > > > >         at java.util.Arrays.copyOfRange(Arrays.java:3664)
> > > > > > > > > >         at java.lang.String.<init>(String.java:201)
> > > > > > > > > >         at
> > > > java.lang.StringBuilder.toString(StringBuilder.java:407)
> > > > > > > > > >         at
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.buildSolrDocument(HttpPoster.java:987)
> > > > > > > > > >         at
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:882)
> > > > > > > > > >
> > > > > > > > > > workers threads:
> > > > > > > > > >
> > > > > > > > > > FATAL 2015-04-14 18:59:11,172 (Worker thread '32') - Error
> > > > tossed:
> > > > > > null
> > > > > > > > > > java.lang.StackOverflowError
> > > > > > > > > >         at
> > > > > > > > java.util.regex.Pattern$CharProperty.match(Pattern.java:3776)
> > > > > > > > > >         at
> > > > java.util.regex.Pattern$Curly.match0(Pattern.java:4250)
> > > > > > > > > >         at
> > > > java.util.regex.Pattern$Curly.match0(Pattern.java:4263)
> > > > > > > > > >         at
> > > > java.util.regex.Pattern$Curly.match0(Pattern.java:4263)
> > > > > > > > > >         at
> > > > java.util.regex.Pattern$Curly.match0(Pattern.java:4263)
> > > > > > > > > >         (...) ~1k lines
> > > > > > > > > >         at
> > > > java.util.regex.Pattern$Curly.match0(Pattern.java:4263)
> > > > > > > > > >
> > > > > > > > > > no errors/warns in solr logs.
> > > > > > > > > >
> > > > > > > > > > it's bug or just corrupted file?
> > > > > > > > > >
> > > > > > > > > > K
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> >