incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colton McInroy <col...@dosarrest.com>
Subject Re: Column Types
Date Mon, 21 Oct 2013 19:50:00 GMT
I just tested, yes, that does appear to be the case...

blur (default)> mutate Program_sshd 1 1 event Date:20131013124823
blur (default)> schema Program_sshd
Schema(table:Program_sshd, 
families:{event={Date=ColumnDefinition(family:event, columnName:Date, 
subColumnName:null, fieldLessIndexed:false, fieldType:date, 
properties:{dateFormat=yyyyMMddHHmmss})}})
Program_sshd
family : event
         column   : Date
                 fieldType : date
                 prop      : dateFormat=yyyyMMddHHmmss

Makes it kind of hard to look at the current schema before committing to 
start pushing data into it.

Thanks,
Colton McInroy

  * Director of Security Engineering

	
Phone
(Toll Free) 	
_US_ 	(888)-818-1344 Press 2
_UK_ 	0-800-635-0551 Press 2

My Extension 	101
24/7 Support 	support@dosarrest.com <mailto:support@dosarrest.com>
Email 	colton@dosarrest.com <mailto:colton@dosarrest.com>
Website 	http://www.dosarrest.com

On 10/21/2013 12:28 PM, Aaron McCurry wrote:
> Yes it might.  I will take a look this evening.
>
>
> On Mon, Oct 21, 2013 at 3:20 PM, Colton McInroy <colton@dosarrest.com>wrote:
>
>> Hmmm... Is this expected? Does the schema only show up once I have added
>> something? Also, strictTypes doesn't appear to be getting changed to true.
>>
>> blur (default)> create -c 10 -l file:///tmp/blur/tables/**Program_sshd -t
>> Program_sshd -s
>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>> yyyyMMddHHmmss
>> blur (default)> schema Program_sshd
>> Schema(table:Program_sshd, families:{})
>> Program_sshd
>> blur (default)> describe Program_sshd
>> cluster                          : default
>> name                             : Program_sshd
>> enabled                          : true
>> tableUri                         : file:///tmp/blur/tables/**Program_sshd
>> shardCount                       : 10
>> readOnly                         : false
>> columnPreCache                   : null
>> blockCaching                     : true
>> blockCachingFileTypes            : null
>> tableProperties                  : null
>> strictTypes                      : false
>> defaultMissingFieldType          : text
>> defaultMissingFieldLessIndexin**g  : true
>> defaultMissingFieldProps         : null
>> blur (default)> disable Program_sshd
>> blur (default)> remove Program_sshd
>> blur (default)> create -c 10 -l file:///tmp/blur/tables/**Program_sshd -t
>> Program_sshd -s -p strictTypes true
>> blur (default)> describe Program_sshd
>> cluster                          : default
>> name                             : Program_sshd
>> enabled                          : true
>> tableUri                         : file:///tmp/blur/tables/**Program_sshd
>> shardCount                       : 10
>> readOnly                         : false
>> columnPreCache                   : null
>> blockCaching                     : true
>> blockCachingFileTypes            : null
>> tableProperties                  : null
>> strictTypes                      : false
>> defaultMissingFieldType          : text
>> defaultMissingFieldLessIndexin**g  : true
>> defaultMissingFieldProps         : null
>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>> yyyyMMddHHmmss
>> blur (default)> schema Program_sshd
>> Schema(table:Program_sshd, families:{})
>> Program_sshd
>> blur (default)>
>>
>>
>>
>>
>> Thanks,
>> Colton McInroy
>>
>>   * Director of Security Engineering
>>
>>
>> Phone
>> (Toll Free)
>> _US_    (888)-818-1344 Press 2
>> _UK_    0-800-635-0551 Press 2
>>
>> My Extension    101
>> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com>
>> Email   colton@dosarrest.com <mailto:colton@dosarrest.com>
>> Website         http://www.dosarrest.com
>>
>> On 10/21/2013 12:08 PM, Colton McInroy wrote:
>>
>>> Thanks,
>>> Colton McInroy
>>>
>>>   * Director of Security Engineering
>>>
>>>
>>> Phone
>>> (Toll Free)
>>> _US_     (888)-818-1344 Press 2
>>> _UK_     0-800-635-0551 Press 2
>>>
>>> My Extension     101
>>> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com>
>>> Email     colton@dosarrest.com <mailto:colton@dosarrest.com>
>>> Website     http://www.dosarrest.com
>>>
>>> On 10/21/2013 11:41 AM, Aaron McCurry wrote:
>>>
>>>> Yes, however with the API changes that have/are being discussed in others
>>>> threads (Document vs. Record, Document Collection vs.Row, etc) I want to
>>>> change the value portion of the Column to have a Value type that will be
>>>> a
>>>> union in Thrift instead of a struct.  This would allow us to have all the
>>>> basic types be defined in separate fields.  This stringValue for string
>>>> types, textValue for text types, intValue for int types, etc that way
>>>> when
>>>> a table is not in strict mode it could better guess the correct type
>>>> instead of blindly choosing text.
>>>>
>>> Makes sense, for my purposes this should work well  enough though. I will
>>> want strictTypes=true and define column types for the column I know about,
>>> and let automatic determination take care of the unspecified fields,
>>> although that shouldn't be very many, since I should know all field types
>>> as I define them in my interface. Kinda sucks that they cannot be changed
>>> though, I'll have to build that into my GUI and make sure to take note of
>>> that.
>>>
>>>> Ok so this code is to merely add the type to be available as a type that
>>>> can be used.  After this runs your new type will act just like "text",
>>>> "int", "date", or any other built in type.  After the type is registered
>>>> in
>>>> the system, either by table or system wide you will still need to call
>>>> definecolumn to make use of the new type.
>>>>
>>>> So in the example.
>>>>
>>>> tableDescriptor.**putToTableProperties("blur.**fieldtype.customfield1",
>>>> "org.apache.blur.analysis.**type.ExampleType1");
>>>>
>>>> "blur.fieldtype." is the important part for the loader.  The prefix tells
>>>> the TableContext that this property is a new field type.  So it takes the
>>>> value "org.apache.blur.analysis.**type.ExampleType1" for example and
>>>> tries to
>>>> load the class via the Class.forName method.  If successfully and if
>>>> it's a
>>>> FieldTypeDefinition it will register is in the BaseFieldManager.  Then
>>>> the
>>>> type is available for use.
>>>>
>>>> So "customfield1" is not even used.  It's only there to makes the
>>>> property
>>>> be a unique name.
>>>>
>>>> Hope this helps.
>>>>
>>> Ahh... *sings* I can see clearly now the sun is out *sings*, I was
>>> misunderstanding what you meant first, that makes a lot more sense, Thanks.
>>>
>>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message