hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Beggs (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1064) HBase REST xml/json improvements
Date Tue, 16 Dec 2008 22:32:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657197#action_12657197
] 

Brian Beggs commented on HBASE-1064:
------------------------------------

Ok, first I misquoted the Jira issues,  The issues this patch is based off of are HBASE-814
and HBASE-815.

+ The REST API looks good. Very RESTy. There might be little nitpicks later but for it looks
great. How different is it from current REST. Should the two be the same? If not, how to deprecate
the old?

I can't take credit for the full architecture of the system as this patch was initially submitted
by Michael Gottesman.  I did rework the way that the input/output flowed out of the system
mainly.  I also have added some missing functionality that was in the original REST implementation
that was not in this one.  

I've been trying to stay as close to the original implementation as possible.  Some of the
calls have changed, though I've tried to keep the xml output/input as close as possible to
the original implementation.  

I will start working on a changelog to hold the differences and attach it to this issue.

It's possible with some package name changes that both interfaces could be bundled with hbase
for a release or 2 with the old one eventually being removed.

+ Where does the jar come from and whats its license
Micheal included this jar with his original issue in HBASE-814.  after I bit of research I
found here: http://github.com/gottesmm/agile-json-2.0/tree/master
It appears that the jar is based on the org.json implementation 

Here is the license from json.org:
Copyright (c) 2002 JSON.org

Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute,
sublicense, and/or sell copies of the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.

The Software shall be used for Good, not Evil.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 

If needed I could rewrite this code at some future point if this library would prove unsatisfactory,
thought it would need to happen once I get the initial implementation finished.

+ The annotations look interesting. Could their serializations be used in other contexts,
the shell say? Or - warning -> crazy-thought coming - somehow producing thrift IDL?

something like this is quite possible I would think, and does make serialization much easier.
 The tradeoff however is that your domain/model objects end up with quite a few annotations
in them and it can be ugly.  Though it will save you from writing quite a bit of code.  

I am not familiar with thift IDL but if something using annotations could be used for serialization
it would be something that could be written as a separate module and could have an impact
beyond this project.

I had initially started using annotations for the XML serialization but pulled them out and
opted for a simpler approach, but this is something that could be changed at a later time.
 I was using this library for the annotation serialization: http://simple.sourceforge.net/

+ In hadoop/hbase, line lengths are < 80 chars (Your new classes are missing apache license,
classes are missing comments describing what class is about, etc).

Yes I will fix these issues with the next patch I supply.

+ Is there anything we could do refactoring HTable say, so your modeling was easier? Looks
like lots of code in your Cell and Database controllers. Should our client be taking on this
model?

Possibly.  I haven't been down that far into the interface yet to fully comment on that issue
at this point.


+ Can you add a note on how you've changed how REST works, high-level?
Yes I will supply this in the coming days.


> HBase REST xml/json improvements
> --------------------------------
>
>                 Key: HBASE-1064
>                 URL: https://issues.apache.org/jira/browse/HBASE-1064
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: rest
>            Reporter: Brian Beggs
>         Attachments: json2.jar, RESTPatch-pass1.patch
>
>
> I've begun work on creating a REST based interface for HBase that can use both JSON and
XML and would be extensible enough to add new formats down the road.  I'm at a point with
this where I would like to submit it for review and to get feedback as I continue to work
towards new features.
> Attached to this issue you will find the patch for the changes to this point along with
a necessary jar file for the JSON serialization.  Also below you will find my notes on how
to use what is finished with the interface to this point.
> This patch is based off of jira issues: 
> HBASE-815 and HBASE-816
> I am interested on gaining feedback on:
> -what you guys think works
> -what doesn't work for the project
> -anything that may need to be added
> -code style
> -anything else...
> Finished components:
> -framework around parsing json/xml input
> -framework around serialzing xml/json output
> -changes to exception handing
> -changes to the response object to better handle the serializing of output data
> -table CRUD calls
> -Full table fetching
> -creating/fetching scanners
> TODO:
> -fix up the filtering with scanners
> -row insert/delete operations
> -individual row fetching
> -cell fetching interface
> -scanner use interface
> Here are the wiki(ish) notes for what is done to this point:
> REST Service for HBASE Notes:
> GET / 
> -retrieves a list of all the tables with their meta data in HBase
> curl -v -H "Accept: text/xml" -X GET -T - http://localhost:60050/
> curl -v -H "Accept: application/json" -X GET -T - http://localhost:60050/
> POST / 
> -Create a table
> curl -H "Content-Type: text/xml" -H "Accept: text/xml" -v -X POST -T - http://localhost:60050/newTable
> <table>
>   <name>test14</name>
>   <columnfamilies>
>     <columnfamily>
>       <name>subscription</name>
>       <max-versions>2</max-versions>
>       <compression>NONE</compression>
>       <in-memory>false</in-memory>
>       <block-cache>true</block-cache>
>     </columnfamily>
>   </columnfamilies>
> </table>
> Response:
> <status><code>200</code><message>success</message></status>
> JSON:
> curl -H "Content-Type: application/json" -H "Accept: application/json" -v -X POST -T
- http://localhost:60050/newTable
> {"name":"test5", "column_families":[{
>              "name":"columnfam1",
>              "bloomfilter":true,
>              "time_to_live":10,
>              "in_memory":false,
>              "max_versions":2,
>              "compression":"", 
>              "max_value_length":50,
>              "block_cache_enabled":true
>           }
> ]}
> *NOTE* this is an enum defined in class HColumnDescriptor.CompressionType
> GET /[table_name]
> -returns all records for the table
> curl -v -H "Accept: text/xml" -X GET -T - http://localhost:60050/tablename
> curl -v -H "Accept: application/json" -X GET -T - http://localhost:60050/tablename
> GET /[table_name]
> -Parameter Action 
> 	metadata - returns the metadata for this table.
> 	regions - returns the regions for this table
> curl -v -H "Accept: text/xml" -X GET -T - http://localhost:60050/pricing1?action=metadata
> Update Table
> PUT /[table_name]
> -updates a table 
> curl -v -H "Content-Type: text/xml" -H "Accept: text/xml" -X PUT -T - http://localhost:60050/pricing1
>   <columnfamilies>
>     <columnfamily>
>       <name>subscription</name>
>       <max-versions>3</max-versions>
>       <compression>NONE</compression>
>       <in-memory>false</in-memory>
>       <block-cache>true</block-cache>
>     </columnfamily>
>     <columnfamily>
>       <name>subscription1</name>
>       <max-versions>3</max-versions>
>       <compression>NONE</compression>
>       <in-memory>false</in-memory>
>       <block-cache>true</block-cache>
>     </columnfamily>
>   </columnfamilies>
> curl -v -H "Content-Type: application/json" -H "Accept: application/json" -X PUT -T -
http://localhost:60050/pricing1
> {"column_families":[{
>              "name":"columnfam1",
>              "bloomfilter":true,
>              "time_to_live":10,
>              "in_memory":false,
>              "max_versions":2,
>              "compression":"", 
>              "max_value_length":50,
>              "block_cache_enabled":true
>           }, 
>           {
>              "name":"columnfam2",
>              "bloomfilter":true,
>              "time_to_live":10,
>              "in_memory":false,
>              "max_versions":2,
>              "compression":"", 
>              "max_value_length":50,
>              "block_cache_enabled":true
>           }
> ]}
> Delete Table
> curl -v -H "Content-Type: text/xml" -H "Accept: text/xml" -X DELETE -T - http://localhost:60050/TEST16
> creating a scanner
> curl -v -H "Content-Type: application/json" -H "Accept: application/json" -X POST -T
- http://localhost:60050/TEST16?action=newscanner
> //TODO fix up the scanner filters.
> response:
> xml:
> <scanner>
>   <id>
>     2
>   </id>
> </scanner>
> json:
> {"id":1}
> Using a scanner
> curl -v -H "Content-Type: application/json" -H "Accept: application/json" -X POST -T
- "http://localhost:60050/TEST16?action=scan&scannerId=<scannerID>&numrows=<num
rows to return>"
> This would be my first submission to an open source project of this size, so please,
give it to me rough.  =)
> Thanks.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message