incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <amccu...@gmail.com>
Subject Re: git commit: BLUR:95 fixed.
Date Sun, 26 May 2013 16:41:54 GMT
I see it. Yep we should. Missed it in the code. 

Sent from my iPhone

On May 26, 2013, at 11:31 AM, Tim Williams <williamstw@gmail.com> wrote:

> The ERROR log o the IOException immediately above my comment:)
> 
> --tim
> 
> On Sunday, May 26, 2013, Aaron McCurry wrote:
> 
>> I'm looking at the logging statements, and they look like they have shard
>> context in them.  Can you point me to a line(s) that don't?  Thanks!
>> 
>> Aaron
>> 
>> 
>> On Sun, May 26, 2013 at 7:40 AM, Tim Williams <williamstw@gmail.com<javascript:;>>
>> wrote:
>> 
>>> On Sun, May 26, 2013 at 1:17 AM,  <gagz@apache.org <javascript:;>>
>> wrote:
>>>> Updated Branches:
>>>>  refs/heads/0.1.5 a18bce613 -> 91afa673f
>>>> 
>>>> 
>>>> BLUR:95 fixed.
>>>> 
>>>> 
>>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo
>>>> Commit:
>>> http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673
>>>> Tree:
>>> http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673
>>>> Diff:
>>> http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673
>>>> 
>>>> Branch: refs/heads/0.1.5
>>>> Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d
>>>> Parents: a18bce6
>>>> Author: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
>>>> Authored: Sun May 26 10:46:57 2013 +0530
>>>> Committer: Gagan <gagandeepjuneja@gmail.com <javascript:;>>
>>>> Committed: Sun May 26 10:46:57 2013 +0530
>>>> 
>>>> ----------------------------------------------------------------------
>>>> .../apache/blur/manager/writer/IndexImporter.java  |   54 +++++-
>>>> .../blur/manager/writer/IndexImporterTest.java     |  141
>>> +++++++++++++++
>>>> 2 files changed, 185 insertions(+), 10 deletions(-)
>>>> ----------------------------------------------------------------------
>>>> 
>>>> 
>>>> 
>>> 
>> http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
>>>> ----------------------------------------------------------------------
>>>> diff --git
>>> 
>> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
>>> 
>> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
>>>> index 5c09ab3..4efd908 100644
>>>> ---
>>> 
>> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
>>>> +++
>>> 
>> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java
>>>> @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock;
>>>> 
>>>> import org.apache.blur.log.Log;
>>>> import org.apache.blur.log.LogFactory;
>>>> +import org.apache.blur.manager.BlurPartitioner;
>>>> import org.apache.blur.server.ShardContext;
>>>> import org.apache.blur.store.hdfs.HdfsDirectory;
>>>> import org.apache.blur.utils.BlurConstants;
>>>> +import org.apache.blur.utils.BlurUtil;
>>>> import org.apache.hadoop.conf.Configuration;
>>>> import org.apache.hadoop.fs.FileStatus;
>>>> import org.apache.hadoop.fs.FileSystem;
>>>> import org.apache.hadoop.fs.Path;
>>>> +import org.apache.hadoop.io.Text;
>>>> import org.apache.lucene.index.AtomicReader;
>>>> import org.apache.lucene.index.AtomicReaderContext;
>>>> import org.apache.lucene.index.CompositeReaderContext;
>>>> @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask
>>> implements Closeable {
>>>>         for (HdfsDirectory directory : indexesToImport) {
>>>>           LOG.info("Starting import [{0}], commiting on [{1}/{2}]",
>>> directory, shard, table);
>>>>           indexWriter.commit();
>>>> -          applyDeletes(directory, indexWriter);
>>>> -          LOG.info("Add index [{0}] [{1}/{2}]", directory, shard,
>>> table);
>>>> -          indexWriter.addIndexes(directory);
>>>> -          LOG.info("Finishing import [{0}], commiting on [{1}/{2}]",
>>> directory, shard, table);
>>>> -          indexWriter.commit();
>>>> -          Path dirPath = directory.getPath();
>>>> -          LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]",
>>> dirPath, shard, table);
>>>> -          fileSystem.delete(dirPath, true);
>>>> -          LOG.info("Import complete on [{0}/{1}]", shard, table);
>>>> +          boolean isSuccess = false;
>>>> +          boolean isRollbackDueToException = false;
>>>> +          try {
>>>> +            isSuccess = applyDeletes(directory, indexWriter, shard);
>>>> +          }catch(IOException e){
>>>> +            LOG.error("Some issue with deleting the old index", e);
>>> 
>>> Shouldn't we add some context (e.g. shard) to the log so the problem's
>>> easier to hunt down?
>>> 
>>> --tim
>>> 
>> 

Mime
View raw message