incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colton McInroy <col...@dosarrest.com>
Subject Re: Column Types
Date Tue, 05 Nov 2013 22:38:36 GMT
Hmm...

ipv6 cannot be converted to a long, so it is a bit more complicated, but 
I will start trying to figure it out right now because that is 
definately something I would like to see as well.

Thanks,
Colton McInroy

  * Director of Security Engineering

	
Phone
(Toll Free) 	
_US_ 	(888)-818-1344 Press 2
_UK_ 	0-800-635-0551 Press 2

My Extension 	101
24/7 Support 	support@dosarrest.com <mailto:support@dosarrest.com>
Email 	colton@dosarrest.com <mailto:colton@dosarrest.com>
Website 	http://www.dosarrest.com

On 11/5/2013 2:21 PM, Garrett Barton wrote:
> Awesome man,
>
> Would it be possible to add in ipv6 support?
>
>
>
>
> On Tue, Nov 5, 2013 at 5:16 PM, Colton McInroy <colton@dosarrest.com> wrote:
>
>>   Looking at the other field types I have come up with this IP field type.
>>
>> I am not sure what needs to be done to include it into the project though.
>> Are there any other files that need to be modified to add the field type
>> into the core or anything?
>>
>> This field type should convert IP addresses into a long which can then be
>> searched with a query specifying a range of IPs. Example...
>> IP:[192.168.0.1 TO 192.168.0.255]
>>
>>   Thanks,
>> Colton McInroy
>>
>>     - Director of Security Engineering
>>
>>     Phone
>> (Toll Free)   *US* (888)-818-1344 Press 2  *UK* 0-800-635-0551 Press 2    My
>> Extension  101  24/7 Support  support@dosarrest.com  Email
>> colton@dosarrest.com  Website http://www.dosarrest.com
>>     On 10/21/2013 5:43 PM, Aaron McCurry wrote:
>>
>> Possibly, I will add an issue to take a look at it.
>> https://issues.apache.org/jira/browse/BLUR-283
>>
>>
>> On Mon, Oct 21, 2013 at 3:53 PM, Colton McInroy <colton@dosarrest.com> <colton@dosarrest.com>wrote:
>>
>>
>>   Oh... strict is still showing up as false, even with -s -p strictTypes
>> true and adding a first mutate.
>>
>> Is this a bug?
>>
>>
>> Thanks,
>> Colton McInroy
>>
>>   * Director of Security Engineering
>>
>>
>> Phone
>> (Toll Free)
>> _US_    (888)-818-1344 Press 2
>> _UK_    0-800-635-0551 Press 2
>>
>> My Extension    101
>> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
>> Email   colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
>> Website         http://www.dosarrest.com
>>
>> On 10/21/2013 12:50 PM, Colton McInroy wrote:
>>
>>
>>   I just tested, yes, that does appear to be the case...
>>
>> blur (default)> mutate Program_sshd 1 1 event Date:20131013124823
>> blur (default)> schema Program_sshd
>> Schema(table:Program_sshd, families:{event={Date=**ColumnDefinition(family:event,
>> columnName:Date, subColumnName:null, fieldLessIndexed:false,
>> fieldType:date, properties:{dateFormat=**yyyyMMddHHmmss})}})
>> Program_sshd
>> family : event
>>          column   : Date
>>                  fieldType : date
>>                  prop      : dateFormat=yyyyMMddHHmmss
>>
>> Makes it kind of hard to look at the current schema before committing to
>> start pushing data into it.
>>
>> Thanks,
>> Colton McInroy
>>
>>   * Director of Security Engineering
>>
>>
>> Phone
>> (Toll Free)
>> _US_     (888)-818-1344 Press 2
>> _UK_     0-800-635-0551 Press 2
>>
>> My Extension     101
>> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
>> Email     colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
>> Website     http://www.dosarrest.com
>>
>> On 10/21/2013 12:28 PM, Aaron McCurry wrote:
>>
>>
>>   Yes it might.  I will take a look this evening.
>>
>>
>> On Mon, Oct 21, 2013 at 3:20 PM, Colton McInroy <colton@dosarrest.com
>>
>>   wrote:
>>
>>    Hmmm... Is this expected? Does the schema only show up once I have added
>>
>>   something? Also, strictTypes doesn't appear to be getting changed to
>> true.
>>
>> blur (default)> create -c 10 -l file:///tmp/blur/tables/****Program_sshd
>> -t
>> Program_sshd -s
>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>> yyyyMMddHHmmss
>> blur (default)> schema Program_sshd
>> Schema(table:Program_sshd, families:{})
>> Program_sshd
>> blur (default)> describe Program_sshd
>> cluster                          : default
>> name                             : Program_sshd
>> enabled                          : true
>> tableUri                         : file:///tmp/blur/tables/****
>> Program_sshd
>> shardCount                       : 10
>> readOnly                         : false
>> columnPreCache                   : null
>> blockCaching                     : true
>> blockCachingFileTypes            : null
>> tableProperties                  : null
>> strictTypes                      : false
>> defaultMissingFieldType          : text
>> defaultMissingFieldLessIndexin****g  : true
>> defaultMissingFieldProps         : null
>> blur (default)> disable Program_sshd
>> blur (default)> remove Program_sshd
>> blur (default)> create -c 10 -l file:///tmp/blur/tables/****Program_sshd
>> -t
>> Program_sshd -s -p strictTypes true
>> blur (default)> describe Program_sshd
>> cluster                          : default
>> name                             : Program_sshd
>> enabled                          : true
>> tableUri                         : file:///tmp/blur/tables/****
>> Program_sshd
>> shardCount                       : 10
>> readOnly                         : false
>> columnPreCache                   : null
>> blockCaching                     : true
>> blockCachingFileTypes            : null
>> tableProperties                  : null
>> strictTypes                      : false
>> defaultMissingFieldType          : text
>> defaultMissingFieldLessIndexin****g  : true
>> defaultMissingFieldProps         : null
>> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
>> yyyyMMddHHmmss
>> blur (default)> schema Program_sshd
>> Schema(table:Program_sshd, families:{})
>> Program_sshd
>> blur (default)>
>>
>>
>>
>>
>> Thanks,
>> Colton McInroy
>>
>>    * Director of Security Engineering
>>
>>
>> Phone
>> (Toll Free)
>> _US_    (888)-818-1344 Press 2
>> _UK_    0-800-635-0551 Press 2
>>
>> My Extension    101
>> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
>> Email   colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
>> Website         http://www.dosarrest.com
>>
>> On 10/21/2013 12:08 PM, Colton McInroy wrote:
>>
>>   Thanks,
>>
>>   Colton McInroy
>>
>>    * Director of Security Engineering
>>
>>
>> Phone
>> (Toll Free)
>> _US_     (888)-818-1344 Press 2
>> _UK_     0-800-635-0551 Press 2
>>
>> My Extension     101
>> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
>> Email     colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
>> Website     http://www.dosarrest.com
>>
>> On 10/21/2013 11:41 AM, Aaron McCurry wrote:
>>
>>   Yes, however with the API changes that have/are being discussed in
>>
>>   others
>> threads (Document vs. Record, Document Collection vs.Row, etc) I want
>> to
>> change the value portion of the Column to have a Value type that will
>> be
>> a
>> union in Thrift instead of a struct.  This would allow us to have all
>> the
>> basic types be defined in separate fields.  This stringValue for
>> string
>> types, textValue for text types, intValue for int types, etc that way
>> when
>> a table is not in strict mode it could better guess the correct type
>> instead of blindly choosing text.
>>
>>   Makes sense, for my purposes this should work well  enough though. I
>>
>>   will
>> want strictTypes=true and define column types for the column I know
>> about,
>> and let automatic determination take care of the unspecified fields,
>> although that shouldn't be very many, since I should know all field
>> types
>> as I define them in my interface. Kinda sucks that they cannot be
>> changed
>> though, I'll have to build that into my GUI and make sure to take note
>> of
>> that.
>>
>>   Ok so this code is to merely add the type to be available as a type
>>
>>   that
>> can be used.  After this runs your new type will act just like "text",
>> "int", "date", or any other built in type.  After the type is
>> registered
>> in
>> the system, either by table or system wide you will still need to call
>> definecolumn to make use of the new type.
>>
>> So in the example.
>>
>> tableDescriptor.****putToTableProperties("blur.****fieldtype.customfield1",
>>
>> "org.apache.blur.analysis.****type.ExampleType1");
>>
>> "blur.fieldtype." is the important part for the loader. The prefix
>> tells
>> the TableContext that this property is a new field type. So it takes
>> the
>> value "org.apache.blur.analysis.****type.ExampleType1" for example
>> and
>> tries to
>> load the class via the Class.forName method.  If successfully and if
>> it's a
>> FieldTypeDefinition it will register is in the BaseFieldManager.  Then
>> the
>> type is available for use.
>>
>> So "customfield1" is not even used.  It's only there to makes the
>> property
>> be a unique name.
>>
>> Hope this helps.
>>
>>   Ahh... *sings* I can see clearly now the sun is out *sings*, I was
>>
>>   misunderstanding what you meant first, that makes a lot more sense,
>> Thanks.
>>
>>
>>
>>
>>
>>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message