phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankit Singhal (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-1311) HBase namespaces surfaced in phoenix
Date Tue, 01 Mar 2016 07:44:18 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173382#comment-15173382
] 

Ankit Singhal edited comment on PHOENIX-1311 at 3/1/16 7:43 AM:
----------------------------------------------------------------

bq. I don't think we need to support SELECT SCHEMA() or SHOW SCHEMAS as there's a standard
JDBC method in DatabaseMetaData that would return the list of schemas. Plus we don't have
that kind of thing for tables. We could start introducing stuff like that (or we could leave
it to the tooling), but if we are going to introduce that, let's do it in a separate JIRA.

Agreed [~giacomotaylor], we will modify api's in DatabaseMetaData if necessary and just wanted
to know whether we should store schema entity in SYSTEM.SCHEMA or in SYSTEM.CATALOG with empty
tablename.

bq. Would you mind providing a couple of examples in SELECT queries for how the schema would
be used and resolved? You're not proposing using a different SELECT * FROM my_schema:my_table
syntax are you?

{code}
> Use test_schema
> select * from T// schema will be resolved as 'test_schema' and hbase table "test_schema:T"
will be referenced
> select * from new_schema.T //  schema will be resolved as 'new_schema' and hbase table
"new_schema:T" will be referenced.
{code}

No, I'm not proposing my_schema:my_table syntax.

bq. For b/w compat, I'm not sure a version flag on PTable is enough. We need something outside
of this, as this will change the way we find the PTable in the first place. How do we know
how to look for it, as we currently look for an HTable with a name of "MY_SCHEMA.MY_TABLE".
Perhaps a global config on whether this feature is on or off, plus a requirement that the
upgrade is done if it's turned on?

Ok, so started modifying one flow to understand if any extra config required. For system tables,
I can see the requirement of global config. can you please refer any code where we need to
resolve table before forming PTable?

Below is the test case on which I'll check:-

{code}
public void testBackWardCompatibility() throws Exception {
		String namespace="TEST_SCHEMA";
		String schemaName = namespace;
		String tableName="TEST";
		
		String phoenixFullTableName=schemaName+"."+tableName;
		String hbaseFullTableName=schemaName+":"+tableName;
		HBaseAdmin admin = driver.getConnectionQueryServices(getUrl(), TestUtil.TEST_PROPERTIES).getAdmin();
		admin.createNamespace(NamespaceDescriptor.create(namespace).build());
		admin.createTable(new HTableDescriptor(TableName.valueOf(namespace, tableName))
				.addFamily(new HColumnDescriptor(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES)));
		admin.createTable(new HTableDescriptor(TableName.valueOf(phoenixFullTableName))
				.addFamily(new HColumnDescriptor(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES)));
		
		Put put=new Put(PVarchar.INSTANCE.toBytes(phoenixFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				QueryConstants.EMPTY_COLUMN_VALUE_BYTES);
		HTable phoenixSchematable=new HTable(admin.getConfiguration(), phoenixFullTableName);
		phoenixSchematable.put(put);
		
		put=new Put(PVarchar.INSTANCE.toBytes(hbaseFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				QueryConstants.EMPTY_COLUMN_VALUE_BYTES);
		
		HTable namespaceMappedtable=new HTable(admin.getConfiguration(), hbaseFullTableName);
		namespaceMappedtable.put(put);
		
		Properties props = new Properties();
		Connection conn = DriverManager.getConnection(getUrl(), props);
		String ddl = "create table "+phoenixFullTableName+"(tableName varchar primary key)";
		conn.createStatement().execute(ddl);
		String query = "select tableName from "+phoenixFullTableName;
		ResultSet rs = conn.createStatement().executeQuery(query);
		assertTrue(rs.next());
		assertEquals(phoenixFullTableName, rs.getString(1));
		
		put=new Put(SchemaUtil.getTableKey(null,schemaName,phoenixFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				PBoolean.INSTANCE.toBytes(Boolean.TRUE));
		
		HTable metatable=new HTable(admin.getConfiguration(), TableName.valueOf(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES));
		metatable.put(put);
		rs = conn.createStatement().executeQuery(query);
		assertTrue(rs.next());
		assertEquals(hbaseFullTableName, rs.getString(1));
	}

{code}



was (Author: ankit.singhal):
bq. I don't think we need to support SELECT SCHEMA() or SHOW SCHEMAS as there's a standard
JDBC method in DatabaseMetaData that would return the list of schemas. Plus we don't have
that kind of thing for tables. We could start introducing stuff like that (or we could leave
it to the tooling), but if we are going to introduce that, let's do it in a separate JIRA.

Agreed [~giacomotaylor], we will modify api's in DatabaseMetaData if necessary and just wanted
to know whether we should store schema entity in SYSTEM.SCHEMA or in SYSTEM.CATALOG with empty
tablename.

bq. Would you mind providing a couple of examples in SELECT queries for how the schema would
be used and resolved? You're not proposing using a different SELECT * FROM my_schema:my_table
syntax are you?

{code}
> Use test_schema
> select * from T// schema will be resolved as 'test_schema' and hbase table "test_schema:T"
will be referenced
> select * from new_schema.T //  schema will be resolved as 'new_schema' and hbase table
"new_schema:T" will be referenced.
{code}

No, I'm not proposing my_schema:my_table syntax.

bq. For b/w compat, I'm not sure a version flag on PTable is enough. We need something outside
of this, as this will change the way we find the PTable in the first place. How do we know
how to look for it, as we currently look for an HTable with a name of "MY_SCHEMA.MY_TABLE".
Perhaps a global config on whether this feature is on or off, plus a requirement that the
upgrade is done if it's turned on?

Ok, I'm not sure on this. so started modifying one flow to understand if any extra config
required. For system tables, I can see the requirement of global config. can you please refer
any code where we need to resolve table to form PTable?

Below is the test case on which I'll check:-

{code}
public void testBackWardCompatibility() throws Exception {
		String namespace="TEST_SCHEMA";
		String schemaName = namespace;
		String tableName="TEST";
		
		String phoenixFullTableName=schemaName+"."+tableName;
		String hbaseFullTableName=schemaName+":"+tableName;
		HBaseAdmin admin = driver.getConnectionQueryServices(getUrl(), TestUtil.TEST_PROPERTIES).getAdmin();
		admin.createNamespace(NamespaceDescriptor.create(namespace).build());
		admin.createTable(new HTableDescriptor(TableName.valueOf(namespace, tableName))
				.addFamily(new HColumnDescriptor(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES)));
		admin.createTable(new HTableDescriptor(TableName.valueOf(phoenixFullTableName))
				.addFamily(new HColumnDescriptor(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES)));
		
		Put put=new Put(PVarchar.INSTANCE.toBytes(phoenixFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				QueryConstants.EMPTY_COLUMN_VALUE_BYTES);
		HTable phoenixSchematable=new HTable(admin.getConfiguration(), phoenixFullTableName);
		phoenixSchematable.put(put);
		
		put=new Put(PVarchar.INSTANCE.toBytes(hbaseFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				QueryConstants.EMPTY_COLUMN_VALUE_BYTES);
		
		HTable namespaceMappedtable=new HTable(admin.getConfiguration(), hbaseFullTableName);
		namespaceMappedtable.put(put);
		
		Properties props = new Properties();
		Connection conn = DriverManager.getConnection(getUrl(), props);
		String ddl = "create table "+phoenixFullTableName+"(tableName varchar primary key)";
		conn.createStatement().execute(ddl);
		String query = "select tableName from "+phoenixFullTableName;
		ResultSet rs = conn.createStatement().executeQuery(query);
		assertTrue(rs.next());
		assertEquals(phoenixFullTableName, rs.getString(1));
		
		put=new Put(SchemaUtil.getTableKey(null,schemaName,phoenixFullTableName));
		put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_BYTES,
				PBoolean.INSTANCE.toBytes(Boolean.TRUE));
		
		HTable metatable=new HTable(admin.getConfiguration(), TableName.valueOf(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES));
		metatable.put(put);
		rs = conn.createStatement().executeQuery(query);
		assertTrue(rs.next());
		assertEquals(hbaseFullTableName, rs.getString(1));
	}

{code}


> HBase namespaces surfaced in phoenix
> ------------------------------------
>
>                 Key: PHOENIX-1311
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1311
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: nicolas maillard
>            Assignee: Ankit Singhal
>            Priority: Minor
>             Fix For: 4.8.0
>
>         Attachments: PHOENIX-1311.docx, PHOENIX-1311_wip.patch, PHOENIX-1311_wip_2.patch
>
>
> Hbase (HBASE-8015) has the concept of namespaces in the form of myNamespace:MyTable it
would be great if Phoenix leveraged this feature to give a database like feature on top of
the table.
> Maybe to stay close to Hbase it could also be a create DB:Table...
> or DB.Table which is a more standard annotation?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message