Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 1E165200B5E for ; Wed, 27 Jul 2016 04:23:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 1C98F160AA6; Wed, 27 Jul 2016 02:23:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C7032160AA4 for ; Wed, 27 Jul 2016 04:23:07 +0200 (CEST) Received: (qmail 846 invoked by uid 500); 27 Jul 2016 02:23:07 -0000 Mailing-List: contact commits-help@pirk.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pirk.incubator.apache.org Delivered-To: mailing list commits@pirk.incubator.apache.org Received: (qmail 836 invoked by uid 99); 27 Jul 2016 02:23:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Jul 2016 02:23:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 73AEA1A7901 for ; Wed, 27 Jul 2016 02:23:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id OBngDkU7gFBh for ; Wed, 27 Jul 2016 02:22:54 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with SMTP id 15D9F5F341 for ; Wed, 27 Jul 2016 02:22:51 +0000 (UTC) Received: (qmail 665 invoked by uid 99); 27 Jul 2016 02:22:51 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Jul 2016 02:22:51 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id E5E96E0B40; Wed, 27 Jul 2016 02:22:50 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: eawilliams@apache.org To: commits@pirk.incubator.apache.org Date: Wed, 27 Jul 2016 02:22:51 -0000 Message-Id: In-Reply-To: <3974ebedfef94923a4fcd085bab9f05b@git.apache.org> References: <3974ebedfef94923a4fcd085bab9f05b@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/7] incubator-pirk git commit: [PIRK-19] Make DataSchema/QuerySchema Agnostic of Persistent Representation -- closes apache/incubator-pirk#26 archived-at: Wed, 27 Jul 2016 02:23:10 -0000 http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/CSVOutputUtils.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/CSVOutputUtils.html b/docs/org/apache/pirk/utils/class-use/CSVOutputUtils.html index 4c07827..44d8e0a 100644 --- a/docs/org/apache/pirk/utils/class-use/CSVOutputUtils.html +++ b/docs/org/apache/pirk/utils/class-use/CSVOutputUtils.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.CSVOutputUtils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/EpochDateParser.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/EpochDateParser.html b/docs/org/apache/pirk/utils/class-use/EpochDateParser.html index 9355786..7b18e70 100644 --- a/docs/org/apache/pirk/utils/class-use/EpochDateParser.html +++ b/docs/org/apache/pirk/utils/class-use/EpochDateParser.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.EpochDateParser http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/FileConst.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/FileConst.html b/docs/org/apache/pirk/utils/class-use/FileConst.html index 6c627aa..69d6c02 100644 --- a/docs/org/apache/pirk/utils/class-use/FileConst.html +++ b/docs/org/apache/pirk/utils/class-use/FileConst.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.FileConst http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/FileIOUtils.Callable.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/FileIOUtils.Callable.html b/docs/org/apache/pirk/utils/class-use/FileIOUtils.Callable.html index 9596a58..dc7a0bf 100644 --- a/docs/org/apache/pirk/utils/class-use/FileIOUtils.Callable.html +++ b/docs/org/apache/pirk/utils/class-use/FileIOUtils.Callable.html @@ -2,7 +2,7 @@ - + Uses of Interface org.apache.pirk.utils.FileIOUtils.Callable http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/FileIOUtils.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/FileIOUtils.html b/docs/org/apache/pirk/utils/class-use/FileIOUtils.html index 3a989a1..ae77fe2 100644 --- a/docs/org/apache/pirk/utils/class-use/FileIOUtils.html +++ b/docs/org/apache/pirk/utils/class-use/FileIOUtils.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.FileIOUtils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/HDFS.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/HDFS.html b/docs/org/apache/pirk/utils/class-use/HDFS.html index ce1e816..39f1640 100644 --- a/docs/org/apache/pirk/utils/class-use/HDFS.html +++ b/docs/org/apache/pirk/utils/class-use/HDFS.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.HDFS http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/ISO8601DateParser.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/ISO8601DateParser.html b/docs/org/apache/pirk/utils/class-use/ISO8601DateParser.html index ddc7e77..3298b32 100644 --- a/docs/org/apache/pirk/utils/class-use/ISO8601DateParser.html +++ b/docs/org/apache/pirk/utils/class-use/ISO8601DateParser.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.ISO8601DateParser http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/KeyedHash.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/KeyedHash.html b/docs/org/apache/pirk/utils/class-use/KeyedHash.html index 4456af9..54e02de 100644 --- a/docs/org/apache/pirk/utils/class-use/KeyedHash.html +++ b/docs/org/apache/pirk/utils/class-use/KeyedHash.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.KeyedHash http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/PIRException.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/PIRException.html b/docs/org/apache/pirk/utils/class-use/PIRException.html index fa6c68e..2233bfa 100644 --- a/docs/org/apache/pirk/utils/class-use/PIRException.html +++ b/docs/org/apache/pirk/utils/class-use/PIRException.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.PIRException @@ -266,7 +266,9 @@ DataSchema -DataSchemaLoader.loadSchemaFile(java.io.InputStream stream)  +DataSchemaLoader.loadSchema(java.io.InputStream stream) +
Returns the data schema as defined in XML format on the given stream.
+ http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/QueryParserUtils.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/QueryParserUtils.html b/docs/org/apache/pirk/utils/class-use/QueryParserUtils.html index a45bb2d..892cc1d 100644 --- a/docs/org/apache/pirk/utils/class-use/QueryParserUtils.html +++ b/docs/org/apache/pirk/utils/class-use/QueryParserUtils.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.QueryParserUtils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/StopListUtils.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/StopListUtils.html b/docs/org/apache/pirk/utils/class-use/StopListUtils.html index 6a34ce7..ce16fc2 100644 --- a/docs/org/apache/pirk/utils/class-use/StopListUtils.html +++ b/docs/org/apache/pirk/utils/class-use/StopListUtils.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.StopListUtils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/StringUtils.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/StringUtils.html b/docs/org/apache/pirk/utils/class-use/StringUtils.html index 9bab0e5..79eaec2 100644 --- a/docs/org/apache/pirk/utils/class-use/StringUtils.html +++ b/docs/org/apache/pirk/utils/class-use/StringUtils.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.StringUtils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/class-use/SystemConfiguration.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/class-use/SystemConfiguration.html b/docs/org/apache/pirk/utils/class-use/SystemConfiguration.html index 20130a7..82816f7 100644 --- a/docs/org/apache/pirk/utils/class-use/SystemConfiguration.html +++ b/docs/org/apache/pirk/utils/class-use/SystemConfiguration.html @@ -2,7 +2,7 @@ - + Uses of Class org.apache.pirk.utils.SystemConfiguration http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/package-frame.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/package-frame.html b/docs/org/apache/pirk/utils/package-frame.html index 6475e80..8cb0e4a 100644 --- a/docs/org/apache/pirk/utils/package-frame.html +++ b/docs/org/apache/pirk/utils/package-frame.html @@ -2,7 +2,7 @@ - + org.apache.pirk.utils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/package-summary.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/package-summary.html b/docs/org/apache/pirk/utils/package-summary.html index 3430e79..150d6c9 100644 --- a/docs/org/apache/pirk/utils/package-summary.html +++ b/docs/org/apache/pirk/utils/package-summary.html @@ -2,7 +2,7 @@ - + org.apache.pirk.utils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/package-tree.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/package-tree.html b/docs/org/apache/pirk/utils/package-tree.html index fa25474..41aef82 100644 --- a/docs/org/apache/pirk/utils/package-tree.html +++ b/docs/org/apache/pirk/utils/package-tree.html @@ -2,7 +2,7 @@ - + org.apache.pirk.utils Class Hierarchy http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/org/apache/pirk/utils/package-use.html ---------------------------------------------------------------------- diff --git a/docs/org/apache/pirk/utils/package-use.html b/docs/org/apache/pirk/utils/package-use.html index 8a51046..b2093e3 100644 --- a/docs/org/apache/pirk/utils/package-use.html +++ b/docs/org/apache/pirk/utils/package-use.html @@ -2,7 +2,7 @@ - + Uses of Package org.apache.pirk.utils http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/overview-frame.html ---------------------------------------------------------------------- diff --git a/docs/overview-frame.html b/docs/overview-frame.html index dd9f53e..75f6630 100644 --- a/docs/overview-frame.html +++ b/docs/overview-frame.html @@ -2,7 +2,7 @@ - + Overview List http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/overview-summary.html ---------------------------------------------------------------------- diff --git a/docs/overview-summary.html b/docs/overview-summary.html index 5c132a4..cbb10e7 100644 --- a/docs/overview-summary.html +++ b/docs/overview-summary.html @@ -2,7 +2,7 @@ - + Overview http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/overview-tree.html ---------------------------------------------------------------------- diff --git a/docs/overview-tree.html b/docs/overview-tree.html index eefb6ad..31c1bc0 100644 --- a/docs/overview-tree.html +++ b/docs/overview-tree.html @@ -2,7 +2,7 @@ - + Class Hierarchy @@ -169,7 +169,6 @@
  • org.apache.pirk.utils.KeyedHash
  • test.general.KeyedHashTest
  • test.schema.data.LoadDataSchemaTest
    • -
  • org.apache.pirk.schema.query.LoadQuerySchemas
  • test.schema.query.LoadQuerySchemaTest
  • org.apache.pirk.serialization.LocalFileSystemStore
  • org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT> @@ -197,6 +196,8 @@
  • test.general.QueryParserUtilsTest
  • org.apache.pirk.schema.response.QueryResponseJSON (implements java.io.Serializable)
  • org.apache.pirk.schema.query.QuerySchema (implements java.io.Serializable)
    • +
  • org.apache.pirk.schema.query.QuerySchemaLoader
    • +
  • org.apache.pirk.schema.query.QuerySchemaRegistry
  • org.apache.pirk.query.wideskies.QueryUtils
  • org.apache.hadoop.mapreduce.RecordReader<KEYIN,VALUEIN> (implements java.io.Closeable)
      http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/serialized-form.html ---------------------------------------------------------------------- diff --git a/docs/serialized-form.html b/docs/serialized-form.html index b7c5a9b..37d5696 100644 --- a/docs/serialized-form.html +++ b/docs/serialized-form.html @@ -2,7 +2,7 @@ - + Serialized Form @@ -777,24 +777,24 @@ 
      java.lang.String dataSchemaName
    • -

      elementNames

      -
      java.util.TreeSet<E> elementNames
      +

      selectorName

      +
      java.lang.String selectorName
    • -

      filter

      -
      java.lang.String filter
      +

      elementNames

      +
      java.util.List<E> elementNames
    • -

      filterObj

      -
      java.lang.Object filterObj
      +

      filterTypeName

      +
      java.lang.String filterTypeName
    • -

      filterElementNames

      -
      java.util.HashSet<E> filterElementNames
      +

      filter

      +
      DataFilter filter
    • -

      selectorName

      -
      java.lang.String selectorName
      +

      filteredElementNames

      +
      java.util.Set<E> filteredElementNames

    • dataElementSize

      @@ -825,11 +825,11 @@

      • filterSet

        -
        java.util.HashSet<E> filterSet
        +
        java.util.Set<E> filterSet

      • stopList

        -
        java.util.HashSet<E> stopList
        +
        java.util.Set<E> stopList
      • http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/ISO8601DateParserTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/ISO8601DateParserTest.html b/docs/test/general/ISO8601DateParserTest.html index a417e44..2b5ea05 100644 --- a/docs/test/general/ISO8601DateParserTest.html +++ b/docs/test/general/ISO8601DateParserTest.html @@ -2,7 +2,7 @@ - + ISO8601DateParserTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/KeyedHashTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/KeyedHashTest.html b/docs/test/general/KeyedHashTest.html index e46f861..a57a8eb 100644 --- a/docs/test/general/KeyedHashTest.html +++ b/docs/test/general/KeyedHashTest.html @@ -2,7 +2,7 @@ - + KeyedHashTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/PaillierTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/PaillierTest.html b/docs/test/general/PaillierTest.html index 6d57ef0..63096b2 100644 --- a/docs/test/general/PaillierTest.html +++ b/docs/test/general/PaillierTest.html @@ -2,7 +2,7 @@ - + PaillierTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/PartitionUtilsTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/PartitionUtilsTest.html b/docs/test/general/PartitionUtilsTest.html index ded49e9..e29692d 100644 --- a/docs/test/general/PartitionUtilsTest.html +++ b/docs/test/general/PartitionUtilsTest.html @@ -2,7 +2,7 @@ - + PartitionUtilsTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/QueryParserUtilsTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/QueryParserUtilsTest.html b/docs/test/general/QueryParserUtilsTest.html index 5b31834..d46b583 100644 --- a/docs/test/general/QueryParserUtilsTest.html +++ b/docs/test/general/QueryParserUtilsTest.html @@ -2,7 +2,7 @@ - + QueryParserUtilsTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/class-use/ISO8601DateParserTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/class-use/ISO8601DateParserTest.html b/docs/test/general/class-use/ISO8601DateParserTest.html index d2e5772..e118a4f 100644 --- a/docs/test/general/class-use/ISO8601DateParserTest.html +++ b/docs/test/general/class-use/ISO8601DateParserTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.general.ISO8601DateParserTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/class-use/KeyedHashTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/class-use/KeyedHashTest.html b/docs/test/general/class-use/KeyedHashTest.html index 1663492..0444cde 100644 --- a/docs/test/general/class-use/KeyedHashTest.html +++ b/docs/test/general/class-use/KeyedHashTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.general.KeyedHashTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/class-use/PaillierTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/class-use/PaillierTest.html b/docs/test/general/class-use/PaillierTest.html index f9261d3..85bf3b6 100644 --- a/docs/test/general/class-use/PaillierTest.html +++ b/docs/test/general/class-use/PaillierTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.general.PaillierTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/class-use/PartitionUtilsTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/class-use/PartitionUtilsTest.html b/docs/test/general/class-use/PartitionUtilsTest.html index 687f58a..afb087c 100644 --- a/docs/test/general/class-use/PartitionUtilsTest.html +++ b/docs/test/general/class-use/PartitionUtilsTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.general.PartitionUtilsTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/class-use/QueryParserUtilsTest.html ---------------------------------------------------------------------- diff --git a/docs/test/general/class-use/QueryParserUtilsTest.html b/docs/test/general/class-use/QueryParserUtilsTest.html index dbf93a4..58e4ab1 100644 --- a/docs/test/general/class-use/QueryParserUtilsTest.html +++ b/docs/test/general/class-use/QueryParserUtilsTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.general.QueryParserUtilsTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/package-frame.html ---------------------------------------------------------------------- diff --git a/docs/test/general/package-frame.html b/docs/test/general/package-frame.html index 0398c7d..ea32008 100644 --- a/docs/test/general/package-frame.html +++ b/docs/test/general/package-frame.html @@ -2,7 +2,7 @@ - + test.general http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/package-summary.html ---------------------------------------------------------------------- diff --git a/docs/test/general/package-summary.html b/docs/test/general/package-summary.html index 752d26e..e76fca1 100644 --- a/docs/test/general/package-summary.html +++ b/docs/test/general/package-summary.html @@ -2,7 +2,7 @@ - + test.general http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/package-tree.html ---------------------------------------------------------------------- diff --git a/docs/test/general/package-tree.html b/docs/test/general/package-tree.html index e6e042f..3e423bf 100644 --- a/docs/test/general/package-tree.html +++ b/docs/test/general/package-tree.html @@ -2,7 +2,7 @@ - + test.general Class Hierarchy http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/general/package-use.html ---------------------------------------------------------------------- diff --git a/docs/test/general/package-use.html b/docs/test/general/package-use.html index 04dffae..2d72288 100644 --- a/docs/test/general/package-use.html +++ b/docs/test/general/package-use.html @@ -2,7 +2,7 @@ - + Uses of Package test.general http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/LoadDataSchemaTest.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/LoadDataSchemaTest.html b/docs/test/schema/data/LoadDataSchemaTest.html index 35c003d..f1b961e 100644 --- a/docs/test/schema/data/LoadDataSchemaTest.html +++ b/docs/test/schema/data/LoadDataSchemaTest.html @@ -2,7 +2,7 @@ - + LoadDataSchemaTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/class-use/LoadDataSchemaTest.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/class-use/LoadDataSchemaTest.html b/docs/test/schema/data/class-use/LoadDataSchemaTest.html index 2bb1ca5..ff5979b 100644 --- a/docs/test/schema/data/class-use/LoadDataSchemaTest.html +++ b/docs/test/schema/data/class-use/LoadDataSchemaTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.schema.data.LoadDataSchemaTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/package-frame.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/package-frame.html b/docs/test/schema/data/package-frame.html index 581cb62..db29808 100644 --- a/docs/test/schema/data/package-frame.html +++ b/docs/test/schema/data/package-frame.html @@ -2,7 +2,7 @@ - + test.schema.data http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/package-summary.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/package-summary.html b/docs/test/schema/data/package-summary.html index 916519c..916c193 100644 --- a/docs/test/schema/data/package-summary.html +++ b/docs/test/schema/data/package-summary.html @@ -2,7 +2,7 @@ - + test.schema.data http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/package-tree.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/package-tree.html b/docs/test/schema/data/package-tree.html index 38d5f72..fb53036 100644 --- a/docs/test/schema/data/package-tree.html +++ b/docs/test/schema/data/package-tree.html @@ -2,7 +2,7 @@ - + test.schema.data Class Hierarchy http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/data/package-use.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/data/package-use.html b/docs/test/schema/data/package-use.html index f84fcdb..883aa3f 100644 --- a/docs/test/schema/data/package-use.html +++ b/docs/test/schema/data/package-use.html @@ -2,7 +2,7 @@ - + Uses of Package test.schema.data http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/LoadQuerySchemaTest.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/LoadQuerySchemaTest.html b/docs/test/schema/query/LoadQuerySchemaTest.html index 8b95753..7c09cbc 100644 --- a/docs/test/schema/query/LoadQuerySchemaTest.html +++ b/docs/test/schema/query/LoadQuerySchemaTest.html @@ -2,7 +2,7 @@ - + LoadQuerySchemaTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/class-use/LoadQuerySchemaTest.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/class-use/LoadQuerySchemaTest.html b/docs/test/schema/query/class-use/LoadQuerySchemaTest.html index 95bf610..b93b43b 100644 --- a/docs/test/schema/query/class-use/LoadQuerySchemaTest.html +++ b/docs/test/schema/query/class-use/LoadQuerySchemaTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.schema.query.LoadQuerySchemaTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/package-frame.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/package-frame.html b/docs/test/schema/query/package-frame.html index 6837b2c..1b01452 100644 --- a/docs/test/schema/query/package-frame.html +++ b/docs/test/schema/query/package-frame.html @@ -2,7 +2,7 @@ - + test.schema.query http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/package-summary.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/package-summary.html b/docs/test/schema/query/package-summary.html index 9cdc2b4..a9a6c3b 100644 --- a/docs/test/schema/query/package-summary.html +++ b/docs/test/schema/query/package-summary.html @@ -2,7 +2,7 @@ - + test.schema.query http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/package-tree.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/package-tree.html b/docs/test/schema/query/package-tree.html index aa8c08c..4b76f8a 100644 --- a/docs/test/schema/query/package-tree.html +++ b/docs/test/schema/query/package-tree.html @@ -2,7 +2,7 @@ - + test.schema.query Class Hierarchy http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/schema/query/package-use.html ---------------------------------------------------------------------- diff --git a/docs/test/schema/query/package-use.html b/docs/test/schema/query/package-use.html index cc965d3..f90976a 100644 --- a/docs/test/schema/query/package-use.html +++ b/docs/test/schema/query/package-use.html @@ -2,7 +2,7 @@ - + Uses of Package test.schema.query http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/StandaloneTest.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/StandaloneTest.html b/docs/test/wideskies/standalone/StandaloneTest.html index 9639e4f..c4ce32a 100644 --- a/docs/test/wideskies/standalone/StandaloneTest.html +++ b/docs/test/wideskies/standalone/StandaloneTest.html @@ -2,7 +2,7 @@ - + StandaloneTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/class-use/StandaloneTest.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/class-use/StandaloneTest.html b/docs/test/wideskies/standalone/class-use/StandaloneTest.html index 78541c1..a02a5d2 100644 --- a/docs/test/wideskies/standalone/class-use/StandaloneTest.html +++ b/docs/test/wideskies/standalone/class-use/StandaloneTest.html @@ -2,7 +2,7 @@ - + Uses of Class test.wideskies.standalone.StandaloneTest http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/package-frame.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/package-frame.html b/docs/test/wideskies/standalone/package-frame.html index b77fe32..ce413e3 100644 --- a/docs/test/wideskies/standalone/package-frame.html +++ b/docs/test/wideskies/standalone/package-frame.html @@ -2,7 +2,7 @@ - + test.wideskies.standalone http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/package-summary.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/package-summary.html b/docs/test/wideskies/standalone/package-summary.html index 0e06d57..3745f60 100644 --- a/docs/test/wideskies/standalone/package-summary.html +++ b/docs/test/wideskies/standalone/package-summary.html @@ -2,7 +2,7 @@ - + test.wideskies.standalone http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/package-tree.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/package-tree.html b/docs/test/wideskies/standalone/package-tree.html index 4dd8995..d7c8710 100644 --- a/docs/test/wideskies/standalone/package-tree.html +++ b/docs/test/wideskies/standalone/package-tree.html @@ -2,7 +2,7 @@ - + test.wideskies.standalone Class Hierarchy http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/docs/test/wideskies/standalone/package-use.html ---------------------------------------------------------------------- diff --git a/docs/test/wideskies/standalone/package-use.html b/docs/test/wideskies/standalone/package-use.html index c04b4c8..abf0cf1 100644 --- a/docs/test/wideskies/standalone/package-use.html +++ b/docs/test/wideskies/standalone/package-use.html @@ -2,7 +2,7 @@ - + Uses of Package test.wideskies.standalone http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/inputformat/hadoop/InputFormatConst.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/inputformat/hadoop/InputFormatConst.java b/src/main/java/org/apache/pirk/inputformat/hadoop/InputFormatConst.java index d904c53..8c33b2c 100644 --- a/src/main/java/org/apache/pirk/inputformat/hadoop/InputFormatConst.java +++ b/src/main/java/org/apache/pirk/inputformat/hadoop/InputFormatConst.java @@ -26,9 +26,9 @@ import java.util.Arrays; */ public class InputFormatConst { - public static String BASE_FORMAT = "base"; + public static final String BASE_FORMAT = "base"; - public static String ES = "elasticsearch"; + public static final String ES = "elasticsearch"; - public static ArrayList ALLOWED_FORMATS = new ArrayList<>(Arrays.asList(BASE_FORMAT, ES)); + public static final ArrayList ALLOWED_FORMATS = new ArrayList<>(Arrays.asList(BASE_FORMAT, ES)); } http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/inputformat/hadoop/json/JSONRecordReader.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/inputformat/hadoop/json/JSONRecordReader.java b/src/main/java/org/apache/pirk/inputformat/hadoop/json/JSONRecordReader.java index 06e395f..5adf853 100644 --- a/src/main/java/org/apache/pirk/inputformat/hadoop/json/JSONRecordReader.java +++ b/src/main/java/org/apache/pirk/inputformat/hadoop/json/JSONRecordReader.java @@ -30,8 +30,8 @@ import org.apache.hadoop.mapreduce.TaskAttemptContext; import org.apache.hadoop.mapreduce.lib.input.LineRecordReader; import org.apache.pirk.inputformat.hadoop.TextArrayWritable; import org.apache.pirk.schema.data.DataSchema; -import org.apache.pirk.schema.data.DataSchemaLoader; import org.apache.pirk.schema.data.DataSchemaRegistry; +import org.apache.pirk.schema.data.DataSchemaLoader; import org.apache.pirk.utils.QueryParserUtils; import org.apache.pirk.utils.StringUtils; import org.apache.pirk.utils.SystemConfiguration; http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriver.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriver.java b/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriver.java index efa0533..44f2ba0 100644 --- a/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriver.java +++ b/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriver.java @@ -28,7 +28,7 @@ import org.apache.pirk.querier.wideskies.decrypt.DecryptResponse; import org.apache.pirk.querier.wideskies.encrypt.EncryptQuery; import org.apache.pirk.query.wideskies.QueryInfo; import org.apache.pirk.response.wideskies.Response; -import org.apache.pirk.schema.query.LoadQuerySchemas; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.serialization.LocalFileSystemStore; import org.apache.pirk.utils.FileIOUtils; import org.apache.pirk.utils.PIRException; @@ -128,10 +128,13 @@ public class QuerierDriver implements Serializable } // Check to ensure we have a valid queryType - if (!LoadQuerySchemas.containsSchema(queryType)) + if (QuerySchemaRegistry.get(queryType) == null) { logger.error("Invalid schema: " + queryType + "; The following schemas are loaded:"); - LoadQuerySchemas.printSchemas(); + for (String schema : QuerySchemaRegistry.getNames()) + { + logger.info("schema = " + schema); + } System.exit(0); } @@ -167,7 +170,7 @@ public class QuerierDriver implements Serializable if (SystemConfiguration.getProperty("pir.embedQuerySchema").equals("true")) { - queryInfo.addQuerySchema(LoadQuerySchemas.getSchema(queryType)); + queryInfo.addQuerySchema(QuerySchemaRegistry.get(queryType)); } Paillier paillier = new Paillier(paillierBitSize, certainty, bitSet); // throws PIRException if certainty conditions are not satisfied http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriverCLI.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriverCLI.java b/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriverCLI.java index 879f028..b6dc2a7 100644 --- a/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriverCLI.java +++ b/src/main/java/org/apache/pirk/querier/wideskies/QuerierDriverCLI.java @@ -25,7 +25,7 @@ import org.apache.commons.cli.HelpFormatter; import org.apache.commons.cli.Option; import org.apache.commons.cli.Options; import org.apache.pirk.schema.data.DataSchemaLoader; -import org.apache.pirk.schema.query.LoadQuerySchemas; +import org.apache.pirk.schema.query.QuerySchemaLoader; import org.apache.pirk.utils.SystemConfiguration; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -41,28 +41,28 @@ public class QuerierDriverCLI private CommandLine commandLine = null; // General variables - public static String ACTION = "action"; - public static String INPUTFILE = "inputFile"; - public static String OUTPUTFILE = "outputFile"; - public static String TYPE = "queryType"; - public static String NUMTHREADS = "numThreads"; - public static String EMBEDQUERYSCHEMA = "embedQuerySchema"; + public static final String ACTION = "action"; + public static final String INPUTFILE = "inputFile"; + public static final String OUTPUTFILE = "outputFile"; + public static final String TYPE = "queryType"; + public static final String NUMTHREADS = "numThreads"; + public static final String EMBEDQUERYSCHEMA = "embedQuerySchema"; // Encryption variables - public static String HASHBITSIZE = "hashBitSize"; - public static String HASHKEY = "hashKey"; - public static String DATAPARTITIONSIZE = "dataPartitionBitSize"; - public static String PAILLIERBITSIZE = "paillierBitSize"; - public static String BITSET = "bitSet"; - public static String CERTAINTY = "certainty"; - public static String QUERYNAME = "queryName"; - public static String QUERYSCHEMAS = "querySchemas"; - public static String DATASCHEMAS = "dataSchemas"; - public static String EMBEDSELECTOR = "embedSelector"; - public static String USEMEMLOOKUPTABLE = "memLookupTable"; - public static String USEHDFSLOOKUPTABLE = "useHDFSLookupTable"; - public static String SR_ALGORITHM = "secureRandomAlg"; - public static String SR_PROVIDER = "secureRandomProvider"; + public static final String HASHBITSIZE = "hashBitSize"; + public static final String HASHKEY = "hashKey"; + public static final String DATAPARTITIONSIZE = "dataPartitionBitSize"; + public static final String PAILLIERBITSIZE = "paillierBitSize"; + public static final String BITSET = "bitSet"; + public static final String CERTAINTY = "certainty"; + public static final String QUERYNAME = "queryName"; + public static final String QUERYSCHEMAS = "querySchemas"; + public static final String DATASCHEMAS = "dataSchemas"; + public static final String EMBEDSELECTOR = "embedSelector"; + public static final String USEMEMLOOKUPTABLE = "memLookupTable"; + public static final String USEHDFSLOOKUPTABLE = "useHDFSLookupTable"; + public static final String SR_ALGORITHM = "secureRandomAlg"; + public static final String SR_PROVIDER = "secureRandomProvider"; // Decryption variables public static String QUERIERFILE = "querierFile"; @@ -316,7 +316,7 @@ public class QuerierDriverCLI try { DataSchemaLoader.initialize(); - LoadQuerySchemas.initialize(); + QuerySchemaLoader.initialize(); } catch (Exception e) { http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/querier/wideskies/decrypt/DecryptResponseRunnable.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/querier/wideskies/decrypt/DecryptResponseRunnable.java b/src/main/java/org/apache/pirk/querier/wideskies/decrypt/DecryptResponseRunnable.java index ea9d6bb..6f1341c 100644 --- a/src/main/java/org/apache/pirk/querier/wideskies/decrypt/DecryptResponseRunnable.java +++ b/src/main/java/org/apache/pirk/querier/wideskies/decrypt/DecryptResponseRunnable.java @@ -25,8 +25,8 @@ import java.util.TreeMap; import org.apache.pirk.query.wideskies.QueryInfo; import org.apache.pirk.query.wideskies.QueryUtils; -import org.apache.pirk.schema.query.LoadQuerySchemas; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.schema.response.QueryResponseJSON; import org.apache.pirk.utils.SystemConfiguration; import org.slf4j.Logger; @@ -65,7 +65,7 @@ public class DecryptResponseRunnable implements Runnable { if ((qSchema = queryInfo.getQuerySchema()) == null) { - qSchema = LoadQuerySchemas.getSchema(queryInfo.getQueryType()); + qSchema = QuerySchemaRegistry.get(queryInfo.getQueryType()); } } resultMap = new HashMap<>(); @@ -83,7 +83,7 @@ public class DecryptResponseRunnable implements Runnable int dataPartitionBitSize = queryInfo.getDataPartitionBitSize(); int numPartitionsPerDataElement = queryInfo.getNumPartitionsPerDataElement(); - QuerySchema qSchema = LoadQuerySchemas.getSchema(queryInfo.getQueryType()); + QuerySchema qSchema = QuerySchemaRegistry.get(queryInfo.getQueryType()); String selectorName = qSchema.getSelectorName(); // Initialize - removes checks below http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/querier/wideskies/encrypt/EncryptQuery.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/querier/wideskies/encrypt/EncryptQuery.java b/src/main/java/org/apache/pirk/querier/wideskies/encrypt/EncryptQuery.java index df38851..b12e837 100644 --- a/src/main/java/org/apache/pirk/querier/wideskies/encrypt/EncryptQuery.java +++ b/src/main/java/org/apache/pirk/querier/wideskies/encrypt/EncryptQuery.java @@ -32,8 +32,8 @@ import org.apache.pirk.query.wideskies.QueryInfo; import org.apache.pirk.query.wideskies.QueryUtils; import org.apache.pirk.schema.data.DataSchema; import org.apache.pirk.schema.data.DataSchemaRegistry; -import org.apache.pirk.schema.query.LoadQuerySchemas; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.utils.KeyedHash; import org.apache.pirk.utils.PIRException; import org.slf4j.Logger; @@ -173,7 +173,7 @@ public class EncryptQuery private void populateEmbeddedSelectorMap() { - QuerySchema qSchema = LoadQuerySchemas.getSchema(queryInfo.getQueryType()); + QuerySchema qSchema = QuerySchemaRegistry.get(queryInfo.getQueryType()); DataSchema dSchema = DataSchemaRegistry.get(qSchema.getDataSchemaName()); String type = dSchema.getElementType(qSchema.getSelectorName()); int sNum = 0; http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/query/wideskies/QueryInfo.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/query/wideskies/QueryInfo.java b/src/main/java/org/apache/pirk/query/wideskies/QueryInfo.java index 0822ca9..8c69fa9 100644 --- a/src/main/java/org/apache/pirk/query/wideskies/QueryInfo.java +++ b/src/main/java/org/apache/pirk/query/wideskies/QueryInfo.java @@ -20,8 +20,8 @@ package org.apache.pirk.query.wideskies; import java.io.Serializable; -import org.apache.pirk.schema.query.LoadQuerySchemas; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -83,7 +83,7 @@ public class QueryInfo implements Serializable useHDFSExpLookupTable = useHDFSExpLookupTableInput; embedSelector = embedSelectorInput; - numBitsPerDataElement = LoadQuerySchemas.getSchema(queryType).getDataElementSize(); + numBitsPerDataElement = QuerySchemaRegistry.get(queryType).getDataElementSize(); dataPartitionBitSize = dataPartitionBitSizeInput; numPartitionsPerDataElement = numBitsPerDataElement / dataPartitionBitSizeInput; http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/query/wideskies/QueryUtils.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/query/wideskies/QueryUtils.java b/src/main/java/org/apache/pirk/query/wideskies/QueryUtils.java index 8344630..70273f8 100644 --- a/src/main/java/org/apache/pirk/query/wideskies/QueryUtils.java +++ b/src/main/java/org/apache/pirk/query/wideskies/QueryUtils.java @@ -23,7 +23,6 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; -import java.util.TreeSet; import org.apache.hadoop.io.ArrayWritable; import org.apache.hadoop.io.MapWritable; @@ -75,7 +74,7 @@ public class QueryUtils logger.debug("Extracted embedded selector = " + embeddedSelector + " parts.size() = " + parts.size()); } - TreeSet dataFieldsToExtract = qSchema.getElementNames(); + List dataFieldsToExtract = qSchema.getElementNames(); for (String fieldName : dataFieldsToExtract) { int numElements = 1; @@ -122,7 +121,7 @@ public class QueryUtils } // Add all appropriate data fields - TreeSet dataFieldsToExtract = qSchema.getElementNames(); + List dataFieldsToExtract = qSchema.getElementNames(); for (String fieldName : dataFieldsToExtract) { Object dataElement = null; @@ -185,7 +184,7 @@ public class QueryUtils } // Add all appropriate data fields - TreeSet dataFieldsToExtract = qSchema.getElementNames(); + List dataFieldsToExtract = qSchema.getElementNames(); for (String fieldName : dataFieldsToExtract) { Object dataElement = null; http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/responder/wideskies/ResponderCLI.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/responder/wideskies/ResponderCLI.java b/src/main/java/org/apache/pirk/responder/wideskies/ResponderCLI.java index 1a2fd32..d38867e 100644 --- a/src/main/java/org/apache/pirk/responder/wideskies/ResponderCLI.java +++ b/src/main/java/org/apache/pirk/responder/wideskies/ResponderCLI.java @@ -26,7 +26,7 @@ import org.apache.commons.cli.Option; import org.apache.commons.cli.Options; import org.apache.pirk.inputformat.hadoop.InputFormatConst; import org.apache.pirk.schema.data.DataSchemaLoader; -import org.apache.pirk.schema.query.LoadQuerySchemas; +import org.apache.pirk.schema.query.QuerySchemaLoader; import org.apache.pirk.utils.SystemConfiguration; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -42,35 +42,35 @@ public class ResponderCLI private CommandLine commandLine = null; // Required args - public static String PLATFORM = "platform"; - public static String QUERYINPUT = "queryInput"; - public static String DATAINPUTFORMAT = "dataInputFormat"; - public static String INPUTDATA = "inputData"; - public static String BASEQUERY = "baseQuery"; - public static String ESRESOURCE = "esResource"; - public static String ESQUERY = "esQuery"; - public static String OUTPUTFILE = "outputFile"; + public static final String PLATFORM = "platform"; + public static final String QUERYINPUT = "queryInput"; + public static final String DATAINPUTFORMAT = "dataInputFormat"; + public static final String INPUTDATA = "inputData"; + public static final String BASEQUERY = "baseQuery"; + public static final String ESRESOURCE = "esResource"; + public static final String ESQUERY = "esQuery"; + public static final String OUTPUTFILE = "outputFile"; // Optional args - public static String BASEINPUTFORMAT = "baseInputFormat"; - public static String STOPLISTFILE = "stopListFile"; - private static String NUMREDUCETASKS = "numReduceTasks"; - public static String USELOCALCACHE = "useLocalCache"; - public static String LIMITHITSPERSELECTOR = "limitHitsPerSelector"; - public static String MAXHITSPERSELECTOR = "maxHitsPerSelector"; - private static String MAPMEMORY = "mapreduceMapMemoryMb"; - private static String REDUCEMEMORY = "mapreduceReduceMemoryMb"; - private static String MAPJAVAOPTS = "mapreduceMapJavaOpts"; - private static String REDUCEJAVAOPTS = "mapreduceReduceJavaOpts"; - public static String QUERYSCHEMAS = "querySchemas"; - public static String DATASCHEMAS = "dataSchemas"; - public static String NUMEXPLOOKUPPARTS = "numExpLookupPartitions"; - private static String USEHDFSLOOKUPTABLE = "useHDFSLookupTable"; - private static String NUMDATAPARTITIONS = "numDataPartitions"; - public static String NUMCOLMULTPARTITIONS = "numColMultPartitions"; - public static String USEMODEXPJOIN = "useModExpJoin"; - public static String COLMULTREDUCEBYKEY = "colMultReduceByKey"; - public static String ALLOWEMBEDDEDQUERYSCHEMAS = "allowAdHocQuerySchemas"; + public static final String BASEINPUTFORMAT = "baseInputFormat"; + public static final String STOPLISTFILE = "stopListFile"; + private static final String NUMREDUCETASKS = "numReduceTasks"; + public static final String USELOCALCACHE = "useLocalCache"; + public static final String LIMITHITSPERSELECTOR = "limitHitsPerSelector"; + public static final String MAXHITSPERSELECTOR = "maxHitsPerSelector"; + private static final String MAPMEMORY = "mapreduceMapMemoryMb"; + private static final String REDUCEMEMORY = "mapreduceReduceMemoryMb"; + private static final String MAPJAVAOPTS = "mapreduceMapJavaOpts"; + private static final String REDUCEJAVAOPTS = "mapreduceReduceJavaOpts"; + public static final String QUERYSCHEMAS = "querySchemas"; + public static final String DATASCHEMAS = "dataSchemas"; + public static final String NUMEXPLOOKUPPARTS = "numExpLookupPartitions"; + private static final String USEHDFSLOOKUPTABLE = "useHDFSLookupTable"; + private static final String NUMDATAPARTITIONS = "numDataPartitions"; + public static final String NUMCOLMULTPARTITIONS = "numColMultPartitions"; + public static final String USEMODEXPJOIN = "useModExpJoin"; + public static final String COLMULTREDUCEBYKEY = "colMultReduceByKey"; + public static final String ALLOWEMBEDDEDQUERYSCHEMAS = "allowAdHocQuerySchemas"; /** * Create and parse allowable options @@ -357,7 +357,7 @@ public class ResponderCLI try { DataSchemaLoader.initialize(); - LoadQuerySchemas.initialize(); + QuerySchemaLoader.initialize(); } catch (Exception e) { http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/ComputeResponseTool.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/ComputeResponseTool.java b/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/ComputeResponseTool.java index f124bc2..5e3077f 100644 --- a/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/ComputeResponseTool.java +++ b/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/ComputeResponseTool.java @@ -47,8 +47,9 @@ import org.apache.pirk.inputformat.hadoop.InputFormatConst; import org.apache.pirk.query.wideskies.Query; import org.apache.pirk.query.wideskies.QueryInfo; import org.apache.pirk.schema.data.DataSchemaLoader; -import org.apache.pirk.schema.query.LoadQuerySchemas; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaLoader; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.serialization.HadoopFileSystemStore; import org.apache.pirk.utils.FileConst; import org.apache.pirk.utils.HDFS; @@ -126,7 +127,7 @@ public class ComputeResponseTool extends Configured implements Tool // Load the schemas DataSchemaLoader.initialize(true, fs); - LoadQuerySchemas.initialize(true, fs); + QuerySchemaLoader.initialize(true, fs); query = new HadoopFileSystemStore(fs).recall(queryInputDir, Query.class); queryInfo = query.getQueryInfo(); @@ -136,7 +137,7 @@ public class ComputeResponseTool extends Configured implements Tool } if (qSchema == null) { - qSchema = LoadQuerySchemas.getSchema(queryInfo.getQueryType()); + qSchema = QuerySchemaRegistry.get(queryInfo.getQueryType()); } logger.info("outputFile = " + outputFile + " outputDirInit = " + outputDirInit + " outputDirColumnMult = " + outputDirColumnMult + " queryInputDir = " http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/HashSelectorsAndPartitionDataMapper.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/HashSelectorsAndPartitionDataMapper.java b/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/HashSelectorsAndPartitionDataMapper.java index 73bc4cd..75186e4 100644 --- a/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/HashSelectorsAndPartitionDataMapper.java +++ b/src/main/java/org/apache/pirk/responder/wideskies/mapreduce/HashSelectorsAndPartitionDataMapper.java @@ -33,8 +33,9 @@ import org.apache.pirk.responder.wideskies.common.HashSelectorAndPartitionData; import org.apache.pirk.schema.data.DataSchema; import org.apache.pirk.schema.data.DataSchemaLoader; import org.apache.pirk.schema.data.DataSchemaRegistry; -import org.apache.pirk.schema.query.LoadQuerySchemas; +import org.apache.pirk.schema.query.QuerySchemaLoader; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.schema.query.filter.DataFilter; import org.apache.pirk.serialization.HadoopFileSystemStore; import org.apache.pirk.utils.StringUtils; @@ -88,7 +89,7 @@ public class HashSelectorsAndPartitionDataMapper extends Mapper accum = accumIn; QueryInfo queryInfo = bbVarsIn.getQueryInfo(); - QuerySchema qSchema = LoadQuerySchemas.getSchema(queryInfo.getQueryType()); + QuerySchema qSchema = QuerySchemaRegistry.get(queryInfo.getQueryType()); dSchema = DataSchemaRegistry.get(qSchema.getDataSchemaName()); - filter = qSchema.getFilterInstance(); + filter = qSchema.getFilter(); logger.info("Initialized FilterData"); } http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/responder/wideskies/standalone/Responder.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/responder/wideskies/standalone/Responder.java b/src/main/java/org/apache/pirk/responder/wideskies/standalone/Responder.java index 80baea5..9b15f3b 100644 --- a/src/main/java/org/apache/pirk/responder/wideskies/standalone/Responder.java +++ b/src/main/java/org/apache/pirk/responder/wideskies/standalone/Responder.java @@ -30,8 +30,8 @@ import org.apache.pirk.query.wideskies.Query; import org.apache.pirk.query.wideskies.QueryInfo; import org.apache.pirk.query.wideskies.QueryUtils; import org.apache.pirk.response.wideskies.Response; -import org.apache.pirk.schema.query.LoadQuerySchemas; import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.QuerySchemaRegistry; import org.apache.pirk.serialization.LocalFileSystemStore; import org.apache.pirk.utils.KeyedHash; import org.apache.pirk.utils.SystemConfiguration; @@ -79,7 +79,7 @@ public class Responder } if (qSchema == null) { - qSchema = LoadQuerySchemas.getSchema(queryType); + qSchema = QuerySchemaRegistry.get(queryType); } response = new Response(queryInfo); http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/schema/data/DataSchemaLoader.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/schema/data/DataSchemaLoader.java b/src/main/java/org/apache/pirk/schema/data/DataSchemaLoader.java index 1a60d42..f0cca32 100644 --- a/src/main/java/org/apache/pirk/schema/data/DataSchemaLoader.java +++ b/src/main/java/org/apache/pirk/schema/data/DataSchemaLoader.java @@ -120,7 +120,7 @@ public class DataSchemaLoader try { - DataSchema dataSchema = loader.loadSchemaFile(is); + DataSchema dataSchema = loader.loadSchema(is); DataSchemaRegistry.put(dataSchema); } finally { @@ -136,7 +136,18 @@ public class DataSchemaLoader public DataSchemaLoader() {} - public DataSchema loadSchemaFile(InputStream stream) throws IOException, PIRException + /** + * Returns the data schema as defined in XML format on the given stream. + * + * @param stream + * The source of the XML data schema description. + * @return The data schema. + * @throws IOException + * A problem occurred reading from the given stream. + * @throws PIRException + * The schema description is invalid. + */ + public DataSchema loadSchema(InputStream stream) throws IOException, PIRException { // Read in and parse the XML schema file Document doc; http://git-wip-us.apache.org/repos/asf/incubator-pirk/blob/89052a78/src/main/java/org/apache/pirk/schema/data/DataSchemaRegistry.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/pirk/schema/data/DataSchemaRegistry.java b/src/main/java/org/apache/pirk/schema/data/DataSchemaRegistry.java index 2cd6673..82fa1da 100644 --- a/src/main/java/org/apache/pirk/schema/data/DataSchemaRegistry.java +++ b/src/main/java/org/apache/pirk/schema/data/DataSchemaRegistry.java @@ -20,6 +20,7 @@ package org.apache.pirk.schema.data; import java.util.HashMap; import java.util.Map; +import java.util.Set; /** * The data schema registry is a global location for data schema descriptors. @@ -63,4 +64,14 @@ public class DataSchemaRegistry { return registry.get(schemaName); } + + /** + * Returns the set of data schema names held in the registry. + * + * @return The possibly empty set of data schema names. + */ + public static Set getNames() + { + return registry.keySet(); + } }