atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0
Date Mon, 07 May 2018 21:30:28 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/#review202597
-----------------------------------------------------------


Ship it!




Ship It!

- Ashutosh Mestry


On May 7, 2018, 9:28 p.m., Pierre Padovani wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66064/
> -----------------------------------------------------------
> 
> (Updated May 7, 2018, 9:28 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2478
>     https://issues.apache.org/jira/browse/ATLAS-2478
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates documentation.
> 
> Included with this patch is an update to the berkley-elasticsearch profile to automatically
download and include elasticsearch as a side application much like solr is. Updates to the
start/stop/conf scripts are included as well.
> 
> NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java.
There are six constants that are incorrectly named with a '.' (dot). This is not supported
in Elasticsearch 5 and beyond when defining a mapping **UNLESS** the field names can be collectively
thought of as an object. In the case of the fields defined in the Constants.java file, 'type'
is defined as a string field, and 'type.name' is also defined as a string field. Elasticsearch
sees this as an error, since it cannot convert type to an object. The fix included simply
changes the field names from using a '.' (dot) to an '_' (underscore). This should NOT affect
compatibility with hbase/solr for new installs. For existing installations, a reindex will
be required as the field names will have changed.
> 
> **Query**: There is a way we can simplify integration/unit tests for the in-memory graph
store by using a maven plugin that will download and run an elasticsearch node. This is nothing
more than a maven change, and change to the atlas-application.properties to switch to elasticsearch
from solr. I did not implement this, but am curious if this change would be desired. If so,
this can be done with a separate ticket.
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 16a3aaa15 
>   distro/pom.xml 6431fd86d 
>   distro/src/bin/atlas_config.py 9062da649 
>   distro/src/bin/atlas_start.py 61d69eb21 
>   distro/src/bin/atlas_stop.py 94c3d6d46 
>   distro/src/conf/atlas-env.sh 053cbd500 
>   distro/src/main/assemblies/standalone-package.xml dc88add56 
>   docs/src/site/twiki/Configuration.twiki 63c3fce96 
>   docs/src/site/twiki/HighAvailability.twiki 4270d0974 
>   docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
>   graphdb/janus/pom.xml 016a09c33 
>   pom.xml 3469a393a 
>   webapp/pom.xml 03b84087f 
> 
> 
> Diff: https://reviews.apache.org/r/66064/diff/3/
> 
> 
> Testing
> -------
> 
> We currently use this fix with our Atlas setup in a fork of the Atlas code, and have
found no issues with it. Additionally, with the update to the berkley-elasticsearch profile,
I have extensively tested that profile to make sure that management of Elasticsearch functioned
correctly.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message