accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corey Nolet <cjno...@gmail.com>
Subject Re: AccumuloMultiTableInputFormat IllegalStatementException
Date Sat, 23 Aug 2014 05:42:40 GMT
Sure. I was able to deserialize the base64 that you posted earlier and it
looks fine. The code I used to do this was like this:

    byte[] serialized =
"AQEAOm9yZy5hcGFjaGUuYWNjdW11bG8uY29yZS5jbGllbnQubWFwcmVkdWNlLklucHV0VGFibGVDb25maWcAAAACjAlmb2xsb3dpbmcBAAAAAAAAAAIAAAYGBgYxMDQ1ODeIf/////////8ABwcHBzEwNDU4NwCIf/////////8AAQAAAAYGBgYxMDUyNTWIf/////////8ABwcHBzEwNTI1NQCIf/////////8AAQAAAAAAAQAAjAx0d2l0dGVyZWRnZXMBAAAAAAAAAAIAAAYGBgYxMDQ1ODeIf/////////8ABwcHBzEwNDU4NwCIf/////////8AAQAAAAYGBgYxMDUyNTWIf/////////8ABwcHBzEwNTI1NQCIf/////////8AAQAAAAAAAQAA".getBytes(
        Constants.UTF8);

    byte[] decoded = Base64.decodeBase64(serialized);

    MapWritable mapWritable = new MapWritable();
    ByteArrayInputStream bais = new ByteArrayInputStream(decoded);
    mapWritable.readFields(new DataInputStream(bais));
    bais.close();

    for(Map.Entry entry : mapWritable.entrySet()) {
      InputTableConfig config = (InputTableConfig)entry.getValue();
      log.debug(entry.getKey() + " - " + config.getRanges());
    }

That IllegalStateException would not be getting thrown if the contents of
the input table config key in the configuration was null.


On Sat, Aug 23, 2014 at 1:31 AM, JavaHokie <soozandjohnyost@gmail.com>
wrote:

> Agreed, should have used getConf(), I cleaned that up, so now things look
> like this:
>
>         Job job = Job.getInstance(getConf());
>
>                 /*
>                  * Set the basic stuff
>                  */
>                 job.setJobName("TwitterJoin Query");
>                 job.setJarByClass(TwitterJoin.class);
>
>                 /*
>                  * Set Mapper and Reducer Classes
>                  */
>                 job.setMapperClass(TwitterJoinMapper.class);
>                 job.setReducerClass(TwitterJoinReducer.class);
>
>             /*
>              * Set the Mapper MapOutputKeyClass and MapOutputValueClass
>              */
>             job.setMapOutputKeyClass(Text.class);
>             job.setMapOutputValueClass(Text.class);
>
>             /*
>              * Set the Reducer OutputKeyClass and OutputValueClass
>              */
>             job.setOutputKeyClass(Text.class);
>             job.setOutputValueClass(Mutation.class);
>
>                 /*
>                  * Set InputFormat and OutputFormat classes
>                  */
>
> job.setInputFormatClass(AccumuloMultiTableInputFormat.class);
>                 job.setOutputFormatClass(AccumuloOutputFormat.class);
>
>                 /*
>                  * Configure InputFormat and OutputFormat Classes
>                  */
>                 Map<String,InputTableConfig> configs = new
> HashMap<String,InputTableConfig>();
>
>                 List<Range> ranges = Lists.newArrayList(new
> Range("104587"),new
> Range("105255"));
>
>                 InputTableConfig edgeConfig = new InputTableConfig();
>                 edgeConfig.setRanges(ranges);
>                 edgeConfig.setAutoAdjustRanges(true);
>
>                 InputTableConfig followerConfig = new InputTableConfig();
>                 followerConfig.setRanges(ranges);
>                 followerConfig.setAutoAdjustRanges(true);
>
>                 configs.put("following",followerConfig);
>                 configs.put("twitteredges",edgeConfig);
>
>
> AccumuloMultiTableInputFormat.setConnectorInfo(job,"root",new
> PasswordToken("!yost8932!".getBytes()));
>
>
> AccumuloMultiTableInputFormat.setZooKeeperInstance(job,"localhost","localhost");
>                 AccumuloMultiTableInputFormat.setScanAuthorizations(job,new
> Authorizations("private"));
>                 AccumuloMultiTableInputFormat.setInputTableConfigs(job,
> configs);
>
>
>
> log.debug(job.getConfiguration().get("AccumuloInputFormat.ScanOpts.TableConfigs"));
>
>
> AccumuloOutputFormat.setZooKeeperInstance(job,"localhost","localhost");
>                 AccumuloOutputFormat.setConnectorInfo(job,"root",new
> PasswordToken("!yost8932!".getBytes()));
>                 AccumuloOutputFormat.setCreateTables(job,true);
>
> AccumuloOutputFormat.setDefaultTableName(job,"twitteredgerollup");
>
>                 /*
>                  * Kick off the job, wait for completion, and return
> applicable code
>                  */
>             boolean success = job.waitForCompletion(true);
>
>             if (success) {
>                 return 0;
>             }
>
>             return 1;
>
> Still getting java.lang.IllegalStateException: The table query
> configurations could not be deserialized from the given configuration
>
> --John
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/AccumuloMultiTableInputFormat-IllegalStateException-tp11186p11197.html
> Sent from the Users mailing list archive at Nabble.com.
>

Mime
View raw message