avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tl ...@rat.io>
Subject Re: Json.ObjectWriter - "Not the Json schema"
Date Wed, 24 Feb 2016 18:21:09 GMT

> On 24.02.2016, at 19:08, Prajwal Tuladhar <praj@infynyxx.com> wrote:
> 
> Can you paste your avro IDL schema?

See below. The full source is available at https://github.com/tomlurge/converTor

@namespace("converTor.torperf")
protocol torperf_descriptor {

  @aliases(["torperfResult"])
  record Torperf {
    string descriptor_type = "torperf 1.0";
    /** metrics-lib/TorperfResult: String getSource() */
    union { null, string } source;
    /** metrics-lib/TorperfResult: int getFileSize() */
    union { null, int } filesize;
    /** metrics-lib/TorperfResult: long getStartMillis() */
    union { null, long } start;
    /** metrics-lib/TorperfResult: long getSocketMillis() */
    union { null, long } socket;
    /** metrics-lib/TorperfResult: long getConnectMillis() */
    union { null, long } connect;
    /** metrics-lib/TorperfResult: long getNegotiateMillis() */
    union { null, long } negotiate;
    /** metrics-lib/TorperfResult: long getRequestMillis() */
    union { null, long } request;
    /** metrics-lib/TorperfResult: long getResponseMillis() */
    union { null, long } response;
    /** metrics-lib/TorperfResult: long getDataRequestMillis() */
    union { null, long } datarequest;
    /** metrics-lib/TorperfResult: long getDataResponseMillis() */
    union { null, long } dataresponse;
    /** metrics-lib/TorperfResult: long getDataCompleteMillis() */
    union { null, long } datacomplete;
    /** metrics-lib/TorperfResult: int getWriteBytes() */
    union { null, int } writebytes;
    /** metrics-lib/TorperfResult: int getReadBytes() */
    union { null, int } readbytes;
    /** metrics-lib/TorperfResult: Boolean didTimeout() */
    union { null, boolean } didtimeout;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(10)
*/
    union { null, long } dataperc10;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(20)
*/
    union { null, long } dataperc20;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(30)
*/
    union { null, long } dataperc30;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(40)
*/
    union { null, long } dataperc40;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(50)
*/
    union { null, long } dataperc50;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(60)
*/
    union { null, long } dataperc60;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(70)
*/
    union { null, long } dataperc70;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(80)
*/
    union { null, long } dataperc80;
    /** metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(90)
*/
    union { null, long } dataperc90;
    /** metrics-lib/TorperfResult: long getLaunchMillis()() */
    union { null, long } launch;
    /** metrics-lib/TorperfResult: long getUsedAtMillis() */
    union { null, long } used_at;
    /** metrics-lib/TorperfResult: List<String> getPath() */
    union { null, array<string> } path;
    /** metrics-lib/TorperfResult: List<Long> getBuildTimes() */
    union { null, array<long> } buildtimes;
    /** metrics-lib/TorperfResult: long getTimeout() */
    union { null, long } timeout;
    /** metrics-lib/TorperfResult: double getQuantile() */
    union { null, double } quantile;
    /** metrics-lib/TorperfResult: int getCircId() */
    union { null, int } circ_id;
    /** metrics-lib/TorperfResult: int getUsedBy() */
    union { null, int } used_by;
   }

}

> On Wed, Feb 24, 2016 at 7:46 AM, tl <tl@rat.io> wrote:
> Hi again,
> 
> I still haven’t found a solution to this problem. Does this look like some beginners
Java mistake (because that may well be…)? Is it okay to ask the same question on stackoverflow
or would that count as crossposting/spamming?
> 
> Cheers,
> Thomas
> 
> 
> > On 23.02.2016, at 02:22, tl <tl@rat.io> wrote:
> >
> > Hi,
> >
> >
> > I want to convert incoming data to Avro and JSON (and later Parquet). Avro conversion
is working okay, but JSON conversion throws the following error that I don’t understand:
> >
> > Exception in thread "main" java.lang.RuntimeException: Not the Json schema: {"type":"record","name":"Torperf","namespace":"converTor.torperf","fields":[{"name":"descriptor_type","type":"string","default":"torperf
1.0"},
> > [ … omitted for brevity …]
> > {"name":"circ_id","type":["null","int"],"doc":"metrics-lib/TorperfResult: int getCircId()"},{"name":"used_by","type":["null","int"],"doc":"metrics-lib/TorperfResult:
int getUsedBy()"}],"aliases":["torperfResult"]}
> >       at org.apache.avro.data.Json$ObjectWriter.setSchema(Json.java:117)
> >       at converTor.WriterObject.<init>(WriterObject.java:116)
> >       at converTor.TypeWriter.get(TypeWriter.java:31)
> >       at converTor.ConverTor.main(ConverTor.java:249)
> >       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >       at java.lang.reflect.Method.invoke(Method.java:606)
> >       at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
> >
> > … but that schema is indeed the schema that I want to use.
> >
> >
> > This is a snippet of my code:
> >
> > File schemaFile = new File("schema/jsonSchema.avsc");
> > Schema.Parser parser = new Schema.Parser();
> > Schema mySchema = parser.parse(schemaFile) ;
> >
> > Json.ObjectWriter jsonDatumWriter = new Json.ObjectWriter();
> > jsonDatumWriter.setSchema(mySchema);
> > OutputStream out = new FileOutputStream(outputFile);
> > Encoder encoder = EncoderFactory.get().jsonEncoder(mySchema, out);
> >
> >
> > Can somebody give me a hint?
> >
> >
> > Thanks,
> > Thomas
> 
> 
> 
> 
> 
> 
> 
> -- 
> --
> Cheers,
> Praj





< he not busy being born is busy dying >






Mime
View raw message