incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: git commit: BLUR:95 fixed.
Date Mon, 27 May 2013 11:53:01 GMT
Fixed.
https://git-wip-us.apache.org/repos/asf?p=incubator-blur.git;a=commit;h=bb42f196


On Sun, May 26, 2013 at 12:41 PM, Aaron McCurry <amccurry@gmail.com> wrote:

> I see it. Yep we should. Missed it in the code.
>
> Sent from my iPhone
>
> On May 26, 2013, at 11:31 AM, Tim Williams <williamstw@gmail.com> wrote:
>
> > The ERROR log o the IOException immediately above my comment:)
> >
> > --tim
> >
> > On Sunday, May 26, 2013, Aaron McCurry wrote:
> >
> >> I'm looking at the logging statements, and they look like they have
> shard
> >> context in them.  Can you point me to a line(s) that don't?  Thanks!
> >>
> >> Aaron
> >>
> >>
> >> On Sun, May 26, 2013 at 7:40 AM, Tim Williams <williamstw@gmail.com
> <javascript:;>>
> >> wrote:
> >>
> >>> On Sun, May 26, 2013 at 1:17 AM,  <gagz@apache.org <javascript:;>>
> >> wrote:
> >>>> Updated Branches:
> >>>>  refs/heads/0.1.5 a18bce613 -> 91afa673f
> >>>>
> >>>>
> >>>> BLUR:95 fixed.
> >>>>
> >>>>
> >>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo
> >>>> Commit:
> >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673
> >>>> Tree:
> >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673
> >>>> Diff:
> >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673
> >>>>
> >>>> Branch: refs/heads/0.1.5
> >>>> Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d
> >>>> Parents: a18bce6
> >>>> Author: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
> >>>> Authored: Sun May 26 10:46:57 2013 +0530
> >>>> Committer: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
> >>>> Committed: Sun May 26 10:46:57 2013 +0530
> >>>>
> >>>> ----------------------------------------------------------------------
> >>>> .../apache/blur/manager/writer/IndexImporter.java  |   54 +++++-
> >>>> .../blur/manager/writer/IndexImporterTest.java     |  141
> >>> +++++++++++++++
> >>>> 2 files changed, 185 insertions(+), 10 deletions(-)
> >>>> ----------------------------------------------------------------------
> >>>>
> >>>>
> >>>>
> >>>
> >>
> http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >>>> ----------------------------------------------------------------------
> >>>> diff --git
> >>>
> >>
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >>>
> >>
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >>>> index 5c09ab3..4efd908 100644
> >>>> ---
> >>>
> >>
> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >>>> +++
> >>>
> >>
> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
> >>>> @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock;
> >>>>
> >>>> import org.apache.blur.log.Log;
> >>>> import org.apache.blur.log.LogFactory;
> >>>> +import org.apache.blur.manager.BlurPartitioner;
> >>>> import org.apache.blur.server.ShardContext;
> >>>> import org.apache.blur.store.hdfs.HdfsDirectory;
> >>>> import org.apache.blur.utils.BlurConstants;
> >>>> +import org.apache.blur.utils.BlurUtil;
> >>>> import org.apache.hadoop.conf.Configuration;
> >>>> import org.apache.hadoop.fs.FileStatus;
> >>>> import org.apache.hadoop.fs.FileSystem;
> >>>> import org.apache.hadoop.fs.Path;
> >>>> +import org.apache.hadoop.io.Text;
> >>>> import org.apache.lucene.index.AtomicReader;
> >>>> import org.apache.lucene.index.AtomicReaderContext;
> >>>> import org.apache.lucene.index.CompositeReaderContext;
> >>>> @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask
> >>> implements Closeable {
> >>>>         for (HdfsDirectory directory : indexesToImport) {
> >>>>           LOG.info("Starting import [{0}], commiting on [{1}/{2}]",
> >>> directory, shard, table);
> >>>>           indexWriter.commit();
> >>>> -          applyDeletes(directory, indexWriter);
> >>>> -          LOG.info("Add index [{0}] [{1}/{2}]", directory, shard,
> >>> table);
> >>>> -          indexWriter.addIndexes(directory);
> >>>> -          LOG.info("Finishing import [{0}], commiting on [{1}/{2}]",
> >>> directory, shard, table);
> >>>> -          indexWriter.commit();
> >>>> -          Path dirPath = directory.getPath();
> >>>> -          LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]",
> >>> dirPath, shard, table);
> >>>> -          fileSystem.delete(dirPath, true);
> >>>> -          LOG.info("Import complete on [{0}/{1}]", shard, table);
> >>>> +          boolean isSuccess = false;
> >>>> +          boolean isRollbackDueToException = false;
> >>>> +          try {
> >>>> +            isSuccess = applyDeletes(directory, indexWriter, shard);
> >>>> +          }catch(IOException e){
> >>>> +            LOG.error("Some issue with deleting the old index", e);
> >>>
> >>> Shouldn't we add some context (e.g. shard) to the log so the problem's
> >>> easier to hunt down?
> >>>
> >>> --tim
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message