lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr MapReduce Indexer : go-live option throwing exception
Date Tue, 26 Jul 2016 21:40:54 GMT
Can't really deal with the security issues, but...

The resulting indexes created by MRIT are just plain vanilla
Solr/Lucene indexes. All the --go-live step does is issue a
MERGEINDEXES command from the core where they live to the directory
MRIT leaves them in, you might get some joy there, see:
https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API#CoreAdminAPI-MERGEINDEXES

Or you can copy them around by hand and start Solr.

You have to be _really_ sure that you get the right index for each
replica though, if you get the index intended for a replica on shard1
on a replica for shard2 it's A Bad Thing.

Best,
Erick

On Tue, Jul 26, 2016 at 10:52 AM, Archana Satheesh Kumar
<akumar@kogentix.com> wrote:
> Hi,
>
>
> I was trying to use the Mapreduce Indexer tool from cloudera, to index my data in Hive
table using Solr.
>
>
> hadoop jar /path/to/lib/solr/contrib/mr/search-mr-*-job.jar  org.apache.solr.hadoop.MapReduceIndexerTool
-Djute.maxbuffer=<buff size>--morphline-file /path/to/morphlines.conf --output-dir hdfs://path/to/output/dir
--reducers -1 --mappers -1 --verbose --go-live --zk-host <zookeeperHostIP>:2181/solr
--shards 2 --collection <collection name> hdfs://location/of/hive/table
>
> My MR job runs successfully and I am able to view _SUCCESS flag in the specified output
loc
>
>  hadoop fs -ls /path/to/output/results
> Found 2 items
> -rwxrwx--x+  3 hive hive          0 2016-07-26 11:35 /path/to/output/results/_SUCCESS
> drwxrwx--x+  - hive hive          0 2016-07-26 11:20 /path/to/output/results/part-00000
>
> But my go-live option is not working.
>
> Exception:
> java.util.concurrent.ExecutionException: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Expected mime type application/octet-stream but got text/html
>
> I also tried using jaas-client.conf
>
> Client {
>  com.sun.security.auth.module.Krb5LoginModule required
>  useKeyTab=false
>  useTicketCache=true
>  principal="<MyID@DOMAIN>";
>  };
>
> So, before executing the Mapreduce job, HADOOP_OPTS was set to the jass-client.conf
> export HADOOP_OPTS="-Djava.security.auth.login.config=/path/to/jaas-client.conf"
>
>
> 1. What could be the issue?
> 2. Am I missing something?
> 3. Since I have my data indexed locally, is there a way to perform the go-live option
separately?
>
> Thanks in advance
>
> Archana
>
>
>
>
>

Mime
View raw message