Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF3D41846B for ; Thu, 23 Jul 2015 18:09:19 +0000 (UTC) Received: (qmail 81433 invoked by uid 500); 23 Jul 2015 18:09:19 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 81372 invoked by uid 500); 23 Jul 2015 18:09:19 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 81362 invoked by uid 99); 23 Jul 2015 18:09:19 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Jul 2015 18:09:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E81AC1A7793 for ; Thu, 23 Jul 2015 18:09:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id spQJpqI4QQOo for ; Thu, 23 Jul 2015 18:09:17 +0000 (UTC) Received: from mail-wi0-f181.google.com (mail-wi0-f181.google.com [209.85.212.181]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 9D0F750612 for ; Thu, 23 Jul 2015 18:09:17 +0000 (UTC) Received: by wibud3 with SMTP id ud3so35638180wib.1 for ; Thu, 23 Jul 2015 11:09:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=nUrukJxcDgJI6Wxcqk1Qx83ePBfGempaHIVs33DOAXo=; b=Aj0sObVmDGdsmGk1V69f/lHPz2RspkVws/rSBVi45ofOF99rjGtUQb7NmB4kELRH2T Bm6Gqiim35oOE5AAszRK+KVN8hEF7Jy2dBuun7HNJJx1tqTy56XJzgF0TxFfOR+Fd4A6 7mSTTH39XtjHsDf9sLrfeQzGvL5J2ilLNEMQhQeBOlmH/UfOrUukc6saCDFiM2Q186er LJyDKIvJ2cRADQRA3J7aCyVVloPcxRYQ0SI25+Wi4PkVm7JXm5gvmv1GlNQkneY4Pdij JglJ7VhMd5DxsAwi8rOluzM7GWprwOrbR0vwG+yhB6EZ8c/sZFjSOSXuTfoMzq9AZ/K2 f6zQ== MIME-Version: 1.0 X-Received: by 10.195.13.113 with SMTP id ex17mr19567341wjd.17.1437674956773; Thu, 23 Jul 2015 11:09:16 -0700 (PDT) Received: by 10.28.176.199 with HTTP; Thu, 23 Jul 2015 11:09:16 -0700 (PDT) Date: Thu, 23 Jul 2015 19:09:16 +0100 Message-ID: Subject: Setting namespace for Avro schema before importing Twitter data From: Rich Midwinter To: user@flume.apache.org Content-Type: multipart/alternative; boundary=047d7bfcf00a19e12b051b8ec964 --047d7bfcf00a19e12b051b8ec964 Content-Type: text/plain; charset=UTF-8 Hi I'm using Flume to import Twitter data and if I generate an Avro schema from the data it looks like: { "type" : "record", "name" : "Doc", "doc" : "adoc", "fields" : [ { "name" : "id", "type" : "string" }, { "name" : "user_friends_count", "type" : [ "int", "null" ] }, { "name" : "user_location", "type" : [ "string", "null" ] }, { "name" : "user_description", "type" : [ "string", "null" ] }, { "name" : "user_statuses_count", "type" : [ "int", "null" ] }, { "name" : "user_followers_count", "type" : [ "int", "null" ] }, { "name" : "user_name", "type" : [ "string", "null" ] }, { "name" : "user_screen_name", "type" : [ "string", "null" ] }, { "name" : "created_at", "type" : [ "string", "null" ] }, { "name" : "text", "type" : [ "string", "null" ] }, { "name" : "retweet_count", "type" : [ "long", "null" ] }, { "name" : "retweeted", "type" : [ "boolean", "null" ] }, { "name" : "in_reply_to_user_id", "type" : [ "long", "null" ] }, { "name" : "source", "type" : [ "string", "null" ] }, { "name" : "in_reply_to_status_id", "type" : [ "long", "null" ] }, { "name" : "media_url_https", "type" : [ "string", "null" ] }, { "name" : "expanded_url", "type" : [ "string", "null" ] } ] } Unfortunately this doesn't have a namespace tag (although I'd also like to change the name value from Doc to something more relevant, like Tweet) and so generated Java code is in the default package and then I can't use it as a dependency. Does anyone know how I can set a namespace for Flume to use? Thanks Rich --047d7bfcf00a19e12b051b8ec964 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi

I'm using Flume to import Twitter data and if I generate an A= vro schema from the data it looks like:

{
"type" : &quo= t;record",
"name" : "Doc",
"doc" := "adoc",
"fields" : [ {
=C2=A0"name" := "id",
=C2=A0"type" : "string"
}, { =C2=A0"name" : "user_friends_count",
=C2=A0"= type" : [ "int", "null" ]
}, {
=C2=A0"= name" : "user_location",
=C2=A0"type" : [ &quo= t;string", "null" ]
}, {
=C2=A0"name" : &qu= ot;user_description",
=C2=A0"type" : [ "string"= ;, "null" ]
}, {
=C2=A0"name" : "user_statu= ses_count",
=C2=A0"type" : [ "int", "null= " ]
}, {
=C2=A0"name" : "user_followers_count&qu= ot;,
=C2=A0"type" : [ "int", "null" ]
= }, {
=C2=A0"name" : "user_name",
=C2=A0"ty= pe" : [ "string", "null" ]
}, {
=C2=A0"= ;name" : "user_screen_name",
=C2=A0"type" : [ = "string", "null" ]
}, {
=C2=A0"name" := "created_at",
=C2=A0"type" : [ "string",= "null" ]
}, {
=C2=A0"name" : "text",<= br> =C2=A0"type" : [ "string", "null" ]
},= {
=C2=A0"name" : "retweet_count",
=C2=A0"= type" : [ "long", "null" ]
}, {
=C2=A0"= ;name" : "retweeted",
=C2=A0"type" : [ "b= oolean", "null" ]
}, {
=C2=A0"name" : "= ;in_reply_to_user_id",
=C2=A0"type" : [ "long"= , "null" ]
}, {
=C2=A0"name" : "source"= ;,
=C2=A0"type" : [ "string", "null" ]}, {
=C2=A0"name" : "in_reply_to_status_id",
= =C2=A0"type" : [ "long", "null" ]
}, {
= =C2=A0"name" : "media_url_https",
=C2=A0"type= " : [ "string", "null" ]
}, {
=C2=A0"n= ame" : "expanded_url",
=C2=A0"type" : [ "= string", "null" ]
} ]
}

Unfortunately this does= n't have a namespace tag (although I'd also like to change the name= value from Doc to something more relevant, like Tweet) and so generated Ja= va code is in the default package and then I can't use it as a dependen= cy.

Does anyone know how I can set a namespace for Flume to use?
=
Thanks
Rich --047d7bfcf00a19e12b051b8ec964--