incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garrett Barton <garrett.bar...@gmail.com>
Subject Re: Column Types
Date Tue, 05 Nov 2013 22:21:05 GMT
Awesome man,

Would it be possible to add in ipv6 support?




On Tue, Nov 5, 2013 at 5:16 PM, Colton McInroy <colton@dosarrest.com> wrote:

>  Looking at the other field types I have come up with this IP field type.
>
> I am not sure what needs to be done to include it into the project though.
> Are there any other files that need to be modified to add the field type
> into the core or anything?
>
> This field type should convert IP addresses into a long which can then be
> searched with a query specifying a range of IPs. Example...
> IP:[192.168.0.1 TO 192.168.0.255]
>
>  Thanks,
> Colton McInroy
>
>    - Director of Security Engineering
>
>    Phone
> (Toll Free)   *US* (888)-818-1344 Press 2  *UK* 0-800-635-0551 Press 2    My
> Extension  101  24/7 Support  support@dosarrest.com  Email
> colton@dosarrest.com  Website http://www.dosarrest.com
>    On 10/21/2013 5:43 PM, Aaron McCurry wrote:
>
> Possibly, I will add an issue to take a look at it.
> https://issues.apache.org/jira/browse/BLUR-283
>
>
> On Mon, Oct 21, 2013 at 3:53 PM, Colton McInroy <colton@dosarrest.com> <colton@dosarrest.com>wrote:
>
>
>  Oh... strict is still showing up as false, even with -s -p strictTypes
> true and adding a first mutate.
>
> Is this a bug?
>
>
> Thanks,
> Colton McInroy
>
>  * Director of Security Engineering
>
>
> Phone
> (Toll Free)
> _US_    (888)-818-1344 Press 2
> _UK_    0-800-635-0551 Press 2
>
> My Extension    101
> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
> Email   colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
> Website         http://www.dosarrest.com
>
> On 10/21/2013 12:50 PM, Colton McInroy wrote:
>
>
>  I just tested, yes, that does appear to be the case...
>
> blur (default)> mutate Program_sshd 1 1 event Date:20131013124823
> blur (default)> schema Program_sshd
> Schema(table:Program_sshd, families:{event={Date=**ColumnDefinition(family:event,
> columnName:Date, subColumnName:null, fieldLessIndexed:false,
> fieldType:date, properties:{dateFormat=**yyyyMMddHHmmss})}})
> Program_sshd
> family : event
>         column   : Date
>                 fieldType : date
>                 prop      : dateFormat=yyyyMMddHHmmss
>
> Makes it kind of hard to look at the current schema before committing to
> start pushing data into it.
>
> Thanks,
> Colton McInroy
>
>  * Director of Security Engineering
>
>
> Phone
> (Toll Free)
> _US_     (888)-818-1344 Press 2
> _UK_     0-800-635-0551 Press 2
>
> My Extension     101
> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
> Email     colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
> Website     http://www.dosarrest.com
>
> On 10/21/2013 12:28 PM, Aaron McCurry wrote:
>
>
>  Yes it might.  I will take a look this evening.
>
>
> On Mon, Oct 21, 2013 at 3:20 PM, Colton McInroy <colton@dosarrest.com
>
>  wrote:
>
>   Hmmm... Is this expected? Does the schema only show up once I have added
>
>  something? Also, strictTypes doesn't appear to be getting changed to
> true.
>
> blur (default)> create -c 10 -l file:///tmp/blur/tables/****Program_sshd
> -t
> Program_sshd -s
> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
> yyyyMMddHHmmss
> blur (default)> schema Program_sshd
> Schema(table:Program_sshd, families:{})
> Program_sshd
> blur (default)> describe Program_sshd
> cluster                          : default
> name                             : Program_sshd
> enabled                          : true
> tableUri                         : file:///tmp/blur/tables/****
> Program_sshd
> shardCount                       : 10
> readOnly                         : false
> columnPreCache                   : null
> blockCaching                     : true
> blockCachingFileTypes            : null
> tableProperties                  : null
> strictTypes                      : false
> defaultMissingFieldType          : text
> defaultMissingFieldLessIndexin****g  : true
> defaultMissingFieldProps         : null
> blur (default)> disable Program_sshd
> blur (default)> remove Program_sshd
> blur (default)> create -c 10 -l file:///tmp/blur/tables/****Program_sshd
> -t
> Program_sshd -s -p strictTypes true
> blur (default)> describe Program_sshd
> cluster                          : default
> name                             : Program_sshd
> enabled                          : true
> tableUri                         : file:///tmp/blur/tables/****
> Program_sshd
> shardCount                       : 10
> readOnly                         : false
> columnPreCache                   : null
> blockCaching                     : true
> blockCachingFileTypes            : null
> tableProperties                  : null
> strictTypes                      : false
> defaultMissingFieldType          : text
> defaultMissingFieldLessIndexin****g  : true
> defaultMissingFieldProps         : null
> blur (default)> definecolumn Program_sshd event Date date -p dateFormat
> yyyyMMddHHmmss
> blur (default)> schema Program_sshd
> Schema(table:Program_sshd, families:{})
> Program_sshd
> blur (default)>
>
>
>
>
> Thanks,
> Colton McInroy
>
>   * Director of Security Engineering
>
>
> Phone
> (Toll Free)
> _US_    (888)-818-1344 Press 2
> _UK_    0-800-635-0551 Press 2
>
> My Extension    101
> 24/7 Support    support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
> Email   colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
> Website         http://www.dosarrest.com
>
> On 10/21/2013 12:08 PM, Colton McInroy wrote:
>
>  Thanks,
>
>  Colton McInroy
>
>   * Director of Security Engineering
>
>
> Phone
> (Toll Free)
> _US_     (888)-818-1344 Press 2
> _UK_     0-800-635-0551 Press 2
>
> My Extension     101
> 24/7 Support     support@dosarrest.com <mailto:support@dosarrest.com> <support@dosarrest.com>
> Email     colton@dosarrest.com <mailto:colton@dosarrest.com> <colton@dosarrest.com>
> Website     http://www.dosarrest.com
>
> On 10/21/2013 11:41 AM, Aaron McCurry wrote:
>
>  Yes, however with the API changes that have/are being discussed in
>
>  others
> threads (Document vs. Record, Document Collection vs.Row, etc) I want
> to
> change the value portion of the Column to have a Value type that will
> be
> a
> union in Thrift instead of a struct.  This would allow us to have all
> the
> basic types be defined in separate fields.  This stringValue for
> string
> types, textValue for text types, intValue for int types, etc that way
> when
> a table is not in strict mode it could better guess the correct type
> instead of blindly choosing text.
>
>  Makes sense, for my purposes this should work well  enough though. I
>
>  will
> want strictTypes=true and define column types for the column I know
> about,
> and let automatic determination take care of the unspecified fields,
> although that shouldn't be very many, since I should know all field
> types
> as I define them in my interface. Kinda sucks that they cannot be
> changed
> though, I'll have to build that into my GUI and make sure to take note
> of
> that.
>
>  Ok so this code is to merely add the type to be available as a type
>
>  that
> can be used.  After this runs your new type will act just like "text",
> "int", "date", or any other built in type.  After the type is
> registered
> in
> the system, either by table or system wide you will still need to call
> definecolumn to make use of the new type.
>
> So in the example.
>
> tableDescriptor.****putToTableProperties("blur.****fieldtype.customfield1",
>
> "org.apache.blur.analysis.****type.ExampleType1");
>
> "blur.fieldtype." is the important part for the loader. The prefix
> tells
> the TableContext that this property is a new field type. So it takes
> the
> value "org.apache.blur.analysis.****type.ExampleType1" for example
> and
> tries to
> load the class via the Class.forName method.  If successfully and if
> it's a
> FieldTypeDefinition it will register is in the BaseFieldManager.  Then
> the
> type is available for use.
>
> So "customfield1" is not even used.  It's only there to makes the
> property
> be a unique name.
>
> Hope this helps.
>
>  Ahh... *sings* I can see clearly now the sun is out *sings*, I was
>
>  misunderstanding what you meant first, that makes a lot more sense,
> Thanks.
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message