hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2809) StorageHandler authorization providers
Date Mon, 27 Feb 2012 23:27:50 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-2809:
------------------------------

    Attachment: HIVE-2809.D1953.2.patch

enis updated the revision "HIVE-2809 [jira] StorageHandler authorization providers".
Reviewers: JIRA



  Second shot at the patch with more test fixes related to adding db input outputs. Couple
of things to note:
   - This patch adds DB as a Read/WriteEntity type, and adds DBs to inputs/outputs in SemanticAnalyzer.
(This patch captures some of the changes at https://issues.apache.org/jira/browse/HIVE-2093)
   - When PARTITION_LEVEL_PRIVILEGE = TRUE in TBLPROPERTIES, Hive's default auth provider
does check for user, db, and partition level privs, but not for table level. This seems inconsistent
with the javadoc, and counter intuitive. This patch fixes default auth provider to check for
user/db/table/partition privs in order.
   - Still a few tests might fail. Will fix that in the next iteration.

REVISION DETAIL
  https://reviews.facebook.net/D1953

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
  ql/src/java/org/apache/hadoop/hive/ql/hooks/ReadEntity.java
  ql/src/java/org/apache/hadoop/hive/ql/hooks/WriteEntity.java
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
  ql/src/java/org/apache/hadoop/hive/ql/security/authorization/DefaultHiveAuthorizationProvider.java
  ql/src/java/org/apache/hadoop/hive/ql/security/authorization/HdfsAuthorizationProvider.java
  ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageDelegationAuthorizationProvider.java
  ql/src/test/org/apache/hadoop/hive/ql/security/authorization/AuthTestUtils.java
  ql/src/test/org/apache/hadoop/hive/ql/security/authorization/TestHdfsAuthorizationProvider.java
  ql/src/test/queries/clientnegative/authorization_fail_5.q
  ql/src/test/queries/clientnegative/authorization_part.q
  ql/src/test/queries/clientpositive/alter_rename_partition_authorization.q
  ql/src/test/queries/clientpositive/exim_23_import_part_authsuccess.q
  ql/src/test/results/clientnegative/addpart1.q.out
  ql/src/test/results/clientnegative/authorization_fail_5.q.out
  ql/src/test/results/clientnegative/authorization_part.q.out
  ql/src/test/results/clientnegative/database_drop_not_empty.q.out
  ql/src/test/results/clientnegative/database_drop_not_empty_restrict.q.out
  ql/src/test/results/clientnegative/drop_partition_failure.q.out
  ql/src/test/results/clientnegative/exim_01_nonpart_over_loaded.q.out
  ql/src/test/results/clientnegative/exim_02_all_part_over_overlap.q.out
  ql/src/test/results/clientnegative/exim_03_nonpart_noncompat_colschema.q.out
  ql/src/test/results/clientnegative/exim_04_nonpart_noncompat_colnumber.q.out
  ql/src/test/results/clientnegative/exim_05_nonpart_noncompat_coltype.q.out
  ql/src/test/results/clientnegative/exim_06_nonpart_noncompat_storage.q.out
  ql/src/test/results/clientnegative/exim_07_nonpart_noncompat_ifof.q.out
  ql/src/test/results/clientnegative/exim_08_nonpart_noncompat_serde.q.out
  ql/src/test/results/clientnegative/exim_09_nonpart_noncompat_serdeparam.q.out
  ql/src/test/results/clientnegative/exim_10_nonpart_noncompat_bucketing.q.out
  ql/src/test/results/clientnegative/exim_11_nonpart_noncompat_sorting.q.out
  ql/src/test/results/clientnegative/exim_13_nonnative_import.q.out
  ql/src/test/results/clientnegative/exim_14_nonpart_part.q.out
  ql/src/test/results/clientnegative/exim_15_part_nonpart.q.out
  ql/src/test/results/clientnegative/exim_16_part_noncompat_schema.q.out
  ql/src/test/results/clientnegative/exim_17_part_spec_underspec.q.out
  ql/src/test/results/clientnegative/exim_18_part_spec_missing.q.out
  ql/src/test/results/clientnegative/exim_19_external_over_existing.q.out
  ql/src/test/results/clientnegative/exim_20_managed_location_over_existing.q.out
  ql/src/test/results/clientnegative/exim_21_part_managed_external.q.out
  ql/src/test/results/clientnegative/exim_23_import_exist_authfail.q.out
  ql/src/test/results/clientnegative/exim_24_import_part_authfail.q.out
  ql/src/test/results/clientnegative/exim_25_import_nonexist_authfail.q.out
  ql/src/test/results/clientnegative/protectmode_part_no_drop.q.out
  ql/src/test/results/clientnegative/protectmode_tbl2.q.out
  ql/src/test/results/clientnegative/protectmode_tbl3.q.out
  ql/src/test/results/clientnegative/protectmode_tbl4.q.out
  ql/src/test/results/clientnegative/protectmode_tbl5.q.out
  ql/src/test/results/clientnegative/protectmode_tbl_no_drop.q.out
  ql/src/test/results/clientpositive/add_part_exist.q.out
  ql/src/test/results/clientpositive/alter1.q.out
  ql/src/test/results/clientpositive/alter2.q.out
  ql/src/test/results/clientpositive/alter3.q.out
  ql/src/test/results/clientpositive/alter4.q.out
  ql/src/test/results/clientpositive/alter5.q.out
  ql/src/test/results/clientpositive/alter_index.q.out
  ql/src/test/results/clientpositive/alter_merge_2.q.out
  ql/src/test/results/clientpositive/alter_merge_stats.q.out
  ql/src/test/results/clientpositive/alter_partition_format_loc.q.out
  ql/src/test/results/clientpositive/alter_rename_partition.q.out
  ql/src/test/results/clientpositive/alter_rename_partition_authorization.q.out
  ql/src/test/results/clientpositive/alter_table_serde.q.out
  ql/src/test/results/clientpositive/alter_view_rename.q.out
  ql/src/test/results/clientpositive/archive_corrupt.q.out
  ql/src/test/results/clientpositive/authorization_5.q.out
  ql/src/test/results/clientpositive/authorization_7.q.out
  ql/src/test/results/clientpositive/autogen_colalias.q.out
  ql/src/test/results/clientpositive/ba_table1.q.out
  ql/src/test/results/clientpositive/ba_table2.q.out
  ql/src/test/results/clientpositive/ba_table_udfs.q.out
  ql/src/test/results/clientpositive/ba_table_union.q.out
  ql/src/test/results/clientpositive/binary_table_bincolserde.q.out
  ql/src/test/results/clientpositive/binary_table_colserde.q.out
  ql/src/test/results/clientpositive/bucket_groupby.q.out
  ql/src/test/results/clientpositive/columnarserde_create_shortcut.q.out
  ql/src/test/results/clientpositive/combine2.q.out
  ql/src/test/results/clientpositive/combine3.q.out
  ql/src/test/results/clientpositive/create_1.q.out
  ql/src/test/results/clientpositive/create_default_prop.q.out
  ql/src/test/results/clientpositive/create_escape.q.out
  ql/src/test/results/clientpositive/create_insert_outputformat.q.out
  ql/src/test/results/clientpositive/create_like.q.out
  ql/src/test/results/clientpositive/create_like_view.q.out
  ql/src/test/results/clientpositive/create_nested_type.q.out
  ql/src/test/results/clientpositive/create_or_replace_view.q.out
  ql/src/test/results/clientpositive/create_view.q.out
  ql/src/test/results/clientpositive/create_view_partitioned.q.out
  ql/src/test/results/clientpositive/ctas.q.out
  ql/src/test/results/clientpositive/database.q.out
  ql/src/test/results/clientpositive/database_location.q.out
  ql/src/test/results/clientpositive/database_properties.q.out
  ql/src/test/results/clientpositive/ddltime.q.out
  ql/src/test/results/clientpositive/default_partition_name.q.out
  ql/src/test/results/clientpositive/describe_formatted_view_partitioned.q.out
  ql/src/test/results/clientpositive/describe_table.q.out
  ql/src/test/results/clientpositive/drop_multi_partitions.q.out
  ql/src/test/results/clientpositive/exim_00_nonpart_empty.q.out
  ql/src/test/results/clientpositive/exim_01_nonpart.q.out
  ql/src/test/results/clientpositive/exim_02_00_part_empty.q.out
  ql/src/test/results/clientpositive/exim_02_part.q.out
  ql/src/test/results/clientpositive/exim_03_nonpart_over_compat.q.out
  ql/src/test/results/clientpositive/exim_04_all_part.q.out
  ql/src/test/results/clientpositive/exim_04_evolved_parts.q.out
  ql/src/test/results/clientpositive/exim_05_some_part.q.out
  ql/src/test/results/clientpositive/exim_06_one_part.q.out
  ql/src/test/results/clientpositive/exim_07_all_part_over_nonoverlap.q.out
  ql/src/test/results/clientpositive/exim_08_nonpart_rename.q.out
  ql/src/test/results/clientpositive/exim_09_part_spec_nonoverlap.q.out
  ql/src/test/results/clientpositive/exim_10_external_managed.q.out
  ql/src/test/results/clientpositive/exim_11_managed_external.q.out
  ql/src/test/results/clientpositive/exim_12_external_location.q.out
  ql/src/test/results/clientpositive/exim_13_managed_location.q.out
  ql/src/test/results/clientpositive/exim_14_managed_location_over_existing.q.out
  ql/src/test/results/clientpositive/exim_15_external_part.q.out
  ql/src/test/results/clientpositive/exim_16_part_external.q.out
  ql/src/test/results/clientpositive/exim_17_part_managed.q.out
  ql/src/test/results/clientpositive/exim_18_part_external.q.out
  ql/src/test/results/clientpositive/exim_19_00_part_external_location.q.out
  ql/src/test/results/clientpositive/exim_19_part_external_location.q.out
  ql/src/test/results/clientpositive/exim_20_part_managed_location.q.out
  ql/src/test/results/clientpositive/exim_22_import_exist_authsuccess.q.out
  ql/src/test/results/clientpositive/exim_23_import_part_authsuccess.q.out
  ql/src/test/results/clientpositive/exim_24_import_nonexist_authsuccess.q.out
  ql/src/test/results/clientpositive/fileformat_sequencefile.q.out
  ql/src/test/results/clientpositive/fileformat_text.q.out
  ql/src/test/results/clientpositive/index_creation.q.out
  ql/src/test/results/clientpositive/inoutdriver.q.out
  ql/src/test/results/clientpositive/input1.q.out
  ql/src/test/results/clientpositive/input10.q.out
  ql/src/test/results/clientpositive/input15.q.out
  ql/src/test/results/clientpositive/input2.q.out
  ql/src/test/results/clientpositive/input3.q.out
  ql/src/test/results/clientpositive/input46.q.out
  ql/src/test/results/clientpositive/input_part10.q.out
  ql/src/test/results/clientpositive/inputddl2.q.out
  ql/src/test/results/clientpositive/inputddl3.q.out
  ql/src/test/results/clientpositive/inputddl4.q.out
  ql/src/test/results/clientpositive/inputddl5.q.out
  ql/src/test/results/clientpositive/inputddl6.q.out
  ql/src/test/results/clientpositive/inputddl7.q.out
  ql/src/test/results/clientpositive/inputddl8.q.out
  ql/src/test/results/clientpositive/insert_into6.q.out
  ql/src/test/results/clientpositive/join_thrift.q.out
  ql/src/test/results/clientpositive/load_dyn_part1.q.out
  ql/src/test/results/clientpositive/load_dyn_part10.q.out
  ql/src/test/results/clientpositive/load_dyn_part11.q.out
  ql/src/test/results/clientpositive/load_dyn_part12.q.out
  ql/src/test/results/clientpositive/load_dyn_part13.q.out
  ql/src/test/results/clientpositive/load_dyn_part14.q.out
  ql/src/test/results/clientpositive/load_dyn_part15.q.out
  ql/src/test/results/clientpositive/load_dyn_part2.q.out
  ql/src/test/results/clientpositive/load_dyn_part3.q.out
  ql/src/test/results/clientpositive/load_dyn_part4.q.out
  ql/src/test/results/clientpositive/load_dyn_part5.q.out
  ql/src/test/results/clientpositive/load_dyn_part6.q.out
  ql/src/test/results/clientpositive/load_dyn_part7.q.out
  ql/src/test/results/clientpositive/load_dyn_part8.q.out
  ql/src/test/results/clientpositive/load_dyn_part9.q.out
  ql/src/test/results/clientpositive/load_fs.q.out
  ql/src/test/results/clientpositive/merge3.q.out
  ql/src/test/results/clientpositive/merge4.q.out
  ql/src/test/results/clientpositive/merge_dynamic_partition3.q.out
  ql/src/test/results/clientpositive/mi.q.out
  ql/src/test/results/clientpositive/multi_sahooks.q.out
  ql/src/test/results/clientpositive/part_inherit_tbl_props.q.out
  ql/src/test/results/clientpositive/part_inherit_tbl_props_empty.q.out
  ql/src/test/results/clientpositive/part_inherit_tbl_props_with_star.q.out
  ql/src/test/results/clientpositive/partition_schema1.q.out
  ql/src/test/results/clientpositive/partition_special_char.q.out
  ql/src/test/results/clientpositive/print_header.q.out
  ql/src/test/results/clientpositive/protectmode.q.out
  ql/src/test/results/clientpositive/protectmode2.q.out
  ql/src/test/results/clientpositive/rcfile_bigdata.q.out
  ql/src/test/results/clientpositive/rcfile_columnar.q.out
  ql/src/test/results/clientpositive/rcfile_default_format.q.out
  ql/src/test/results/clientpositive/rename_column.q.out
  ql/src/test/results/clientpositive/show_partitions.q.out
  ql/src/test/results/clientpositive/show_tables.q.out
  ql/src/test/results/clientpositive/showparts.q.out
  ql/src/test/results/clientpositive/split_sample.q.out
  ql/src/test/results/clientpositive/stats0.q.out
  ql/src/test/results/clientpositive/stats1.q.out
  ql/src/test/results/clientpositive/stats10.q.out
  ql/src/test/results/clientpositive/stats12.q.out
  ql/src/test/results/clientpositive/stats13.q.out
  ql/src/test/results/clientpositive/stats14.q.out
  ql/src/test/results/clientpositive/stats15.q.out
  ql/src/test/results/clientpositive/stats16.q.out
  ql/src/test/results/clientpositive/stats2.q.out
  ql/src/test/results/clientpositive/stats3.q.out
  ql/src/test/results/clientpositive/stats4.q.out
  ql/src/test/results/clientpositive/stats5.q.out
  ql/src/test/results/clientpositive/stats6.q.out
  ql/src/test/results/clientpositive/stats7.q.out
  ql/src/test/results/clientpositive/stats8.q.out
  ql/src/test/results/clientpositive/stats9.q.out
  ql/src/test/results/clientpositive/tablename_with_select.q.out
  ql/src/test/results/clientpositive/udf_map_keys.q.out
  ql/src/test/results/clientpositive/udf_map_values.q.out
  ql/src/test/results/clientpositive/udf_printf.q.out
  ql/src/test/results/clientpositive/udf_sort_array.q.out
  ql/src/test/results/clientpositive/updateAccessTime.q.out

                
> StorageHandler authorization providers
> --------------------------------------
>
>                 Key: HIVE-2809
>                 URL: https://issues.apache.org/jira/browse/HIVE-2809
>             Project: Hive
>          Issue Type: New Feature
>    Affects Versions: 0.9.0
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>         Attachments: HIVE-2809.D1953.1.patch, HIVE-2809.D1953.2.patch
>
>
> In this issue, we would like to discuss the possibility of supplementing the Hive authorization
model with authorization at the storage level. As discussed in HIVE-1943, Hive should also
check for operation permissions in hdfs and hbase, since otherwise, data and metadata can
be in an inconsistent state or be orphaned. Going a step further, some of the setups might
not need the full featured auth model by Hive, but want to rely on managing the permissions
at the data layer. In this model, the metadata operations are checked first from hdfs/hbase
and it is allowed only if they are allowed at the data layer. The semantics are documented
at https://cwiki.apache.org/confluence/display/HCATALOG/Hcat+Security+Design. 
> So, the goals of this issue are: 
>  - Port storage handler specific authorization providers, and the StorageDelegationAuthorizationProvider
from HCATALOG-245 and HCATALOG-260 to Hive. 
>  - Keep current Hive's default authorization provider, and enable user to use this and/or
the storage one. auth providers are already configurable.
>  - Move the manual checks that had to be performed about authorization in Hcat to Hive,
specifically:
>   -- CREATE DATABASE/TABLE, ADD PARTITION statements does not call 
>    HiveAuthorizationProvider.authorize() with the candidate objects, which means that
>    we cannot do checks against defined LOCATION.
>   -- HiveOperation does not define sufficient Privileges for most of the operations,

>     especially database operations. 
>   -- For some of the operations, Hive SemanticAnalyzer does not add the changed 
>     object as a WriteEntity or ReadEntity.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message