avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Willis <swil...@compete.com>
Subject How to handle schema dependencies
Date Tue, 11 Dec 2012 19:28:00 GMT

My company currently has one big repo that holds all our java code and avro schemas. I'm currently
splitting it up into one common repo and separate repos for each product. This has been easy
to do for our java code using maven and dependencies, however I can't find a way to do this
with our avro schemas. In the common repo I've got schemas that everything relies on. These
common schemas include some very domain specific stuff along with some pretty general use
schemas that we've defined like Date:

 {"name": "Date",
  "namespace": "com.compete.avro",
  "type": "record",
  "fields": [ { "name":"year", "type":"int" },
              { "name":"month", "type":"int" },
              { "name":"day", "type":"int" } ] }

I'd like to be able to use 'Date' (defined in the common repo) in schemas inside the product-xyz
repo. But when I try this, I get:

[INFO] ------------------------------------------------------------------------
[INFO] "Date" is not a defined name. The type of the "date" field must be a defined name or
a {"type": ...} expression.
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.avro.SchemaParseException: "Date" is not a defined name. The type of the "date"
field must be a defined name or a {"type": ...} expression.
	at org.apache.avro.Schema.parse(Schema.java:1094)
	at org.apache.avro.Schema.parse(Schema.java:1163)
	at org.apache.avro.Schema$Parser.parse(Schema.java:931)
	at org.apache.avro.Schema$Parser.parse(Schema.java:908)
	at org.apache.avro.compiler.specific.SpecificCompiler.compileSchema(SpecificCompiler.java:182)
	at org.apache.avro.compiler.specific.SpecificCompiler.compileSchema(SpecificCompiler.java:174)
	at org.apache.avro.mojo.SchemaMojo.doCompile(SchemaMojo.java:53)
	at org.apache.avro.mojo.AbstractAvroMojo.compileFiles(AbstractAvroMojo.java:129)
	at org.apache.avro.mojo.AbstractAvroMojo.execute(AbstractAvroMojo.java:99)
	at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
	at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
	at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
	at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
	at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:375)

The common jar which contains the compiled Date class is available in our maven repo... is
there some way to use that? I'm currently using the avro-maven-plugin to do the java code
generation, is there an option to this plugin to specify schemas or jars to include? It seems
like the only work around is to put all avro schemas that we might use in any product in the
common repo, or duplicate all the common schemas inside each product specific repo.

-Steven Willis

View raw message