incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Williams <william...@gmail.com>
Subject Re: git commit: BLUR:95 fixed.
Date Sun, 26 May 2013 15:31:27 GMT
The ERROR log o the IOException immediately above my comment:)

--tim

On Sunday, May 26, 2013, Aaron McCurry wrote:

> I'm looking at the logging statements, and they look like they have shard
> context in them.  Can you point me to a line(s) that don't?  Thanks!
>
> Aaron
>
>
> On Sun, May 26, 2013 at 7:40 AM, Tim Williams <williamstw@gmail.com<javascript:;>>
> wrote:
>
> > On Sun, May 26, 2013 at 1:17 AM,  <gagz@apache.org <javascript:;>>
> wrote:
> > > Updated Branches:
> > >   refs/heads/0.1.5 a18bce613 -> 91afa673f
> > >
> > >
> > > BLUR:95 fixed.
> > >
> > >
> > > Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo
> > > Commit:
> > http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673
> > > Tree:
> > http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673
> > > Diff:
> > http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673
> > >
> > > Branch: refs/heads/0.1.5
> > > Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d
> > > Parents: a18bce6
> > > Author: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
> > > Authored: Sun May 26 10:46:57 2013 +0530
> > > Committer: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
> > > Committed: Sun May 26 10:46:57 2013 +0530
> > >
> > > ----------------------------------------------------------------------
> > >  .../apache/blur/manager/writer/IndexImporter.java  |   54 +++++-
> > >  .../blur/manager/writer/IndexImporterTest.java     |  141
> > +++++++++++++++
> > >  2 files changed, 185 insertions(+), 10 deletions(-)
> > > ----------------------------------------------------------------------
> > >
> > >
> > >
> >
> http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > > ----------------------------------------------------------------------
> > > diff --git
> >
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > > index 5c09ab3..4efd908 100644
> > > ---
> >
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > > +++
> >
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > > @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock;
> > >
> > >  import org.apache.blur.log.Log;
> > >  import org.apache.blur.log.LogFactory;
> > > +import org.apache.blur.manager.BlurPartitioner;
> > >  import org.apache.blur.server.ShardContext;
> > >  import org.apache.blur.store.hdfs.HdfsDirectory;
> > >  import org.apache.blur.utils.BlurConstants;
> > > +import org.apache.blur.utils.BlurUtil;
> > >  import org.apache.hadoop.conf.Configuration;
> > >  import org.apache.hadoop.fs.FileStatus;
> > >  import org.apache.hadoop.fs.FileSystem;
> > >  import org.apache.hadoop.fs.Path;
> > > +import org.apache.hadoop.io.Text;
> > >  import org.apache.lucene.index.AtomicReader;
> > >  import org.apache.lucene.index.AtomicReaderContext;
> > >  import org.apache.lucene.index.CompositeReaderContext;
> > > @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask
> > implements Closeable {
> > >          for (HdfsDirectory directory : indexesToImport) {
> > >            LOG.info("Starting import [{0}], commiting on [{1}/{2}]",
> > directory, shard, table);
> > >            indexWriter.commit();
> > > -          applyDeletes(directory, indexWriter);
> > > -          LOG.info("Add index [{0}] [{1}/{2}]", directory, shard,
> > table);
> > > -          indexWriter.addIndexes(directory);
> > > -          LOG.info("Finishing import [{0}], commiting on [{1}/{2}]",
> > directory, shard, table);
> > > -          indexWriter.commit();
> > > -          Path dirPath = directory.getPath();
> > > -          LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]",
> > dirPath, shard, table);
> > > -          fileSystem.delete(dirPath, true);
> > > -          LOG.info("Import complete on [{0}/{1}]", shard, table);
> > > +          boolean isSuccess = false;
> > > +          boolean isRollbackDueToException = false;
> > > +          try {
> > > +            isSuccess = applyDeletes(directory, indexWriter, shard);
> > > +          }catch(IOException e){
> > > +            LOG.error("Some issue with deleting the old index", e);
> >
> > Shouldn't we add some context (e.g. shard) to the log so the problem's
> > easier to hunt down?
> >
> > --tim
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message