incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colton McInroy <col...@dosarrest.com>
Subject Re: Column Types
Date Mon, 21 Oct 2013 19:53:27 GMT
Oh... strict is still showing up as false, even with -s -p strictTypes 
true and adding a first mutate.

Is this a bug?

Thanks,
Colton McInroy

  * Director of Security Engineering

	
Phone
(Toll Free) 	
_US_ 	(888)-818-1344 Press 2
_UK_ 	0-800-635-0551 Press 2

My Extension 	101
24/7 Support 	support@dosarrest.com <mailto:support@dosarrest.com>
Email 	colton@dosarrest.com <mailto:colton@dosarrest.com>
Website 	http://www.dosarrest.com

On 10/21/2013 12:50 PM, Colton McInroy wrote:
> I just tested, yes, that does appear to be the case...
>
> blur (default)> mutate Program_sshd 1 1 event Date:20131013124823
> blur (default)> schema Program_sshd
> Schema(table:Program_sshd, 
> families:{event={Date=ColumnDefinition(family:event, columnName:Date, 
> subColumnName:null, fieldLessIndexed:false, fieldType:date, 
> properties:{dateFormat=yyyyMMddHHmmss})}})
> Program_sshd
> family : event
>         column   : Date
>                 fieldType : date
>                 prop      : dateFormat=yyyyMMddHHmmss
>
> Makes it kind of hard to look at the current schema before committing 
> to start pushing data into it.
>
> Thanks,
> Colton McInroy
>
>  * Director of Security Engineering
>
>
> Phone
> (Toll Free)
> _US_     (888)-818-1344 Press 2
> _UK_     0-800-635-0551 Press 2
>
> My Extension     101
> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com>
> Email     colton@dosarrest.com <mailto:colton@dosarrest.com>
> Website     http://www.dosarrest.com
>
> On 10/21/2013 12:28 PM, Aaron McCurry wrote:
>> Yes it might.  I will take a look this evening.
>>
>>
>> On Mon, Oct 21, 2013 at 3:20 PM, Colton McInroy 
>> <colton@dosarrest.com>wrote:
>>
>>> Hmmm... Is this expected? Does the schema only show up once I have 
>>> added
>>> something? Also, strictTypes doesn't appear to be getting changed to 
>>> true.
>>>
>>> blur (default)> create -c 10 -l 
>>> file:///tmp/blur/tables/**Program_sshd -t
>>> Program_sshd -s
>>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>>> yyyyMMddHHmmss
>>> blur (default)> schema Program_sshd
>>> Schema(table:Program_sshd, families:{})
>>> Program_sshd
>>> blur (default)> describe Program_sshd
>>> cluster                          : default
>>> name                             : Program_sshd
>>> enabled                          : true
>>> tableUri                         : 
>>> file:///tmp/blur/tables/**Program_sshd
>>> shardCount                       : 10
>>> readOnly                         : false
>>> columnPreCache                   : null
>>> blockCaching                     : true
>>> blockCachingFileTypes            : null
>>> tableProperties                  : null
>>> strictTypes                      : false
>>> defaultMissingFieldType          : text
>>> defaultMissingFieldLessIndexin**g  : true
>>> defaultMissingFieldProps         : null
>>> blur (default)> disable Program_sshd
>>> blur (default)> remove Program_sshd
>>> blur (default)> create -c 10 -l 
>>> file:///tmp/blur/tables/**Program_sshd -t
>>> Program_sshd -s -p strictTypes true
>>> blur (default)> describe Program_sshd
>>> cluster                          : default
>>> name                             : Program_sshd
>>> enabled                          : true
>>> tableUri                         : 
>>> file:///tmp/blur/tables/**Program_sshd
>>> shardCount                       : 10
>>> readOnly                         : false
>>> columnPreCache                   : null
>>> blockCaching                     : true
>>> blockCachingFileTypes            : null
>>> tableProperties                  : null
>>> strictTypes                      : false
>>> defaultMissingFieldType          : text
>>> defaultMissingFieldLessIndexin**g  : true
>>> defaultMissingFieldProps         : null
>>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>>> yyyyMMddHHmmss
>>> blur (default)> schema Program_sshd
>>> Schema(table:Program_sshd, families:{})
>>> Program_sshd
>>> blur (default)>
>>>
>>>
>>>
>>>
>>> Thanks,
>>> Colton McInroy
>>>
>>>   * Director of Security Engineering
>>>
>>>
>>> Phone
>>> (Toll Free)
>>> _US_    (888)-818-1344 Press 2
>>> _UK_    0-800-635-0551 Press 2
>>>
>>> My Extension    101
>>> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com>
>>> Email   colton@dosarrest.com <mailto:colton@dosarrest.com>
>>> Website         http://www.dosarrest.com
>>>
>>> On 10/21/2013 12:08 PM, Colton McInroy wrote:
>>>
>>>> Thanks,
>>>> Colton McInroy
>>>>
>>>>   * Director of Security Engineering
>>>>
>>>>
>>>> Phone
>>>> (Toll Free)
>>>> _US_     (888)-818-1344 Press 2
>>>> _UK_     0-800-635-0551 Press 2
>>>>
>>>> My Extension     101
>>>> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com>
>>>> Email     colton@dosarrest.com <mailto:colton@dosarrest.com>
>>>> Website     http://www.dosarrest.com
>>>>
>>>> On 10/21/2013 11:41 AM, Aaron McCurry wrote:
>>>>
>>>>> Yes, however with the API changes that have/are being discussed in 
>>>>> others
>>>>> threads (Document vs. Record, Document Collection vs.Row, etc) I 
>>>>> want to
>>>>> change the value portion of the Column to have a Value type that 
>>>>> will be
>>>>> a
>>>>> union in Thrift instead of a struct.  This would allow us to have 
>>>>> all the
>>>>> basic types be defined in separate fields.  This stringValue for 
>>>>> string
>>>>> types, textValue for text types, intValue for int types, etc that way
>>>>> when
>>>>> a table is not in strict mode it could better guess the correct type
>>>>> instead of blindly choosing text.
>>>>>
>>>> Makes sense, for my purposes this should work well  enough though. 
>>>> I will
>>>> want strictTypes=true and define column types for the column I know 
>>>> about,
>>>> and let automatic determination take care of the unspecified fields,
>>>> although that shouldn't be very many, since I should know all field 
>>>> types
>>>> as I define them in my interface. Kinda sucks that they cannot be 
>>>> changed
>>>> though, I'll have to build that into my GUI and make sure to take 
>>>> note of
>>>> that.
>>>>
>>>>> Ok so this code is to merely add the type to be available as a 
>>>>> type that
>>>>> can be used.  After this runs your new type will act just like 
>>>>> "text",
>>>>> "int", "date", or any other built in type.  After the type is 
>>>>> registered
>>>>> in
>>>>> the system, either by table or system wide you will still need to 
>>>>> call
>>>>> definecolumn to make use of the new type.
>>>>>
>>>>> So in the example.
>>>>>
>>>>> tableDescriptor.**putToTableProperties("blur.**fieldtype.customfield1",

>>>>>
>>>>> "org.apache.blur.analysis.**type.ExampleType1");
>>>>>
>>>>> "blur.fieldtype." is the important part for the loader. The prefix 
>>>>> tells
>>>>> the TableContext that this property is a new field type. So it 
>>>>> takes the
>>>>> value "org.apache.blur.analysis.**type.ExampleType1" for example and
>>>>> tries to
>>>>> load the class via the Class.forName method.  If successfully and if
>>>>> it's a
>>>>> FieldTypeDefinition it will register is in the BaseFieldManager.  
>>>>> Then
>>>>> the
>>>>> type is available for use.
>>>>>
>>>>> So "customfield1" is not even used.  It's only there to makes the
>>>>> property
>>>>> be a unique name.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>> Ahh... *sings* I can see clearly now the sun is out *sings*, I was
>>>> misunderstanding what you meant first, that makes a lot more sense, 
>>>> Thanks.
>>>>
>>>>
>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message