incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: git commit: BLUR:95 fixed.
Date Sun, 26 May 2013 13:08:57 GMT
I'm looking at the logging statements, and they look like they have shard
context in them.  Can you point me to a line(s) that don't?  Thanks!

Aaron


On Sun, May 26, 2013 at 7:40 AM, Tim Williams <williamstw@gmail.com> wrote:

> On Sun, May 26, 2013 at 1:17 AM,  <gagz@apache.org> wrote:
> > Updated Branches:
> >   refs/heads/0.1.5 a18bce613 -> 91afa673f
> >
> >
> > BLUR:95 fixed.
> >
> >
> > Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo
> > Commit:
> http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673
> > Tree:
> http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673
> > Diff:
> http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673
> >
> > Branch: refs/heads/0.1.5
> > Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d
> > Parents: a18bce6
> > Author: Gagan <gagandeepjuneja@gmail.com>
> > Authored: Sun May 26 10:46:57 2013 +0530
> > Committer: Gagan <gagandeepjuneja@gmail.com>
> > Committed: Sun May 26 10:46:57 2013 +0530
> >
> > ----------------------------------------------------------------------
> >  .../apache/blur/manager/writer/IndexImporter.java  |   54 +++++-
> >  .../blur/manager/writer/IndexImporterTest.java     |  141
> +++++++++++++++
> >  2 files changed, 185 insertions(+), 10 deletions(-)
> > ----------------------------------------------------------------------
> >
> >
> >
> http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > ----------------------------------------------------------------------
> > diff --git
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > index 5c09ab3..4efd908 100644
> > ---
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > +++
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> > @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock;
> >
> >  import org.apache.blur.log.Log;
> >  import org.apache.blur.log.LogFactory;
> > +import org.apache.blur.manager.BlurPartitioner;
> >  import org.apache.blur.server.ShardContext;
> >  import org.apache.blur.store.hdfs.HdfsDirectory;
> >  import org.apache.blur.utils.BlurConstants;
> > +import org.apache.blur.utils.BlurUtil;
> >  import org.apache.hadoop.conf.Configuration;
> >  import org.apache.hadoop.fs.FileStatus;
> >  import org.apache.hadoop.fs.FileSystem;
> >  import org.apache.hadoop.fs.Path;
> > +import org.apache.hadoop.io.Text;
> >  import org.apache.lucene.index.AtomicReader;
> >  import org.apache.lucene.index.AtomicReaderContext;
> >  import org.apache.lucene.index.CompositeReaderContext;
> > @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask
> implements Closeable {
> >          for (HdfsDirectory directory : indexesToImport) {
> >            LOG.info("Starting import [{0}], commiting on [{1}/{2}]",
> directory, shard, table);
> >            indexWriter.commit();
> > -          applyDeletes(directory, indexWriter);
> > -          LOG.info("Add index [{0}] [{1}/{2}]", directory, shard,
> table);
> > -          indexWriter.addIndexes(directory);
> > -          LOG.info("Finishing import [{0}], commiting on [{1}/{2}]",
> directory, shard, table);
> > -          indexWriter.commit();
> > -          Path dirPath = directory.getPath();
> > -          LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]",
> dirPath, shard, table);
> > -          fileSystem.delete(dirPath, true);
> > -          LOG.info("Import complete on [{0}/{1}]", shard, table);
> > +          boolean isSuccess = false;
> > +          boolean isRollbackDueToException = false;
> > +          try {
> > +            isSuccess = applyDeletes(directory, indexWriter, shard);
> > +          }catch(IOException e){
> > +            LOG.error("Some issue with deleting the old index", e);
>
> Shouldn't we add some context (e.g. shard) to the log so the problem's
> easier to hunt down?
>
> --tim
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message