Mailing-List: contact dev-help@falcon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.apache.org
Date: Sat, 30 Jul 2016 05:22:20 +0000 (UTC)
From: "Venkat Ranganathan (JIRA)" <jira@apache.org>
To: dev@falcon.incubator.apache.org
Message-ID: <JIRA.12993640.1469825931000.176273.1469856140562@Atlassian.JIRA>
In-Reply-To: <JIRA.12993640.1469825931000@Atlassian.JIRA>
References: <JIRA.12993640.1469825931000@Atlassian.JIRA> <JIRA.12993640.1469825931085@arcas>
Subject: [jira] [Commented] (FALCON-2104) Loss of data in GraphDB when
 upgrading Falcon from 0.9 to 0.10.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Sat, 30 Jul 2016 05:22:27 -0000


    [ https://issues.apache.org/jira/browse/FALCON-2104?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D154=
00464#comment-15400464 ]=20

Venkat Ranganathan commented on FALCON-2104:
--------------------------------------------

From the titan upgrade [instructions|http://s3.thinkaurelius.com/docs/titan=
/0.5.4/upgrade.html]=20

{quote}
.5.1. From 0.4.x and previous

D.5.1.1. API Upgrade

Titan 0.5.0 has introduced a number of new features and seen significant ch=
anges to the API. Refer to the documentation for a detailed description of =
Titan 0.5.0.

Most importantly, Titan 0.5.0 introduces the management system which should=
 be used for schema creation, index building, and other administrative task=
s that affect the entire graph. The management system is accessed through g=
.getManagementSystem() which returns a management transaction that behaves =
like a normal transaction but provides additional features.

In a management transaction, edge labels and property keys are created with=
 the methods makeEdgeLabel(String) and makePropertyKey(String) respectively=
. Instead of multiple methods for specifying the multiplicity or cardinalit=
y, the builders returned by the respective methods now feature a multiplici=
ty method and a cardinality method, each of which expects an enum argument.=
 The names of the methods in Titan 0.4.x are virtually identical to the enu=
m constants used in Titan 0.5.0.

Note, that schema type definition and index building have been separated in=
 Titan 0.5.0. In older versions, one would call sortKey to build a vertex-c=
entric index for an edge label and indexed to build a global graph index fo=
r property keys. These methods have disappeared from the builder and indexe=
s are build separately using the management system. Refer to the relevant s=
ections of the documentation to learn more about building indexes.

\[Note\]=09Note
It is still possible to define types in an expicit TitanTransaction, howeve=
r, it is strongly encouraged to use this method only for those use cases wh=
ere schema type creation is part of normal user transactions. In all other =
cases Titan=E2=80=99s management system should be used.

Titan 0.5.0 introduces vertex labels as first class schema types. Many appl=
ications build on Titan have developed their own vertex labeling / typing s=
ystem through vertex properties or some other means. It is highly recommend=
ed to transition to the native vertex labels supported in Titan 0.5.0.

In previous versions of Titan, the entire graph database configuration had =
to be provided in a configuration file for each started Titan instance. Sta=
rting in 0.5.0, Titan has a central configuration system which stores all c=
onfiguration properties that must be coordinated across instances. These ar=
e initialized from the configuration file used to start the first Titan ins=
tance in a new cluster. After that, additional Titan instances only need a =
minimal configuration to connect to the cluster. Note, that changing global=
 configuration options can no longer be accomplished through changes to the=
 local configuration files. Such changes must now be made through Titan=E2=
=80=99s management system.

Some configuration options have been renamed and new options have been adde=
d. See the Chapter 12, Configuration Reference for an up-to-date listing of=
 config options.

FullDouble and FullFloat do no longer exist. Use Double and Float instead w=
hich are now serialized as 4 and 8 byte floating point numbers. In places w=
here Double or Float was used in sort keys (i.e. as the data type of the pr=
operty in a sort key), use Decimal and Precision instead, respectively, bec=
ause they have a fixd decimal range.

D.5.1.2. Data Upgrade

The data storage format of Titan 0.5.0 is incompatible with previous releas=
es. The 0.5.0 release does not yet include utilities to automatically conve=
rt data stored with previous releases. This is planned for the 0.5.1 releas=
e. If a data upgrade is desired before this release, it is encouraged to at=
tempt an export from the old version using the graphson format and import i=
t into a new Titan 0.5.0 graph using Faunus/Titan-Hadoop.
{quote}

Looking through the changes, it looks like the API updates are not affectin=
g our use case. - BTW, this was found  while [~bvellanki] was debugging the=
 QE rolling upgrade tests which have not shown other graphdb issues.

> Loss of data in GraphDB when upgrading Falcon from 0.9 to 0.10.=20
> ----------------------------------------------------------------
>
>                 Key: FALCON-2104
>                 URL: https://issues.apache.org/jira/browse/FALCON-2104
>             Project: Falcon
>          Issue Type: Bug
>    Affects Versions: 0.10
>            Reporter: Balu Vellanki
>            Assignee: Balu Vellanki
>            Priority: Critical
>             Fix For: trunk, 0.10
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> FALCON-1333 (Instance Search feature) requires Falcon to use titan-berkel=
eyje version 0.5.4 to support indexing. Up until version 0.9 - Falcon used =
  titan-berkeleyje-jre6 version 0.4.2.  GraphDB created by version 0.4.2 ca=
nnot be read by version 0.5.4. When attempting an upgrade, I realized that =
entity and instance lineage data created in 0.9 version could not be read b=
y Falcon in 0.10 version.=20
> The only solution seems to provide a tool to do the following
> - Use 0.4.2 version of titan-berkeleyje-jre6 to read the berkeleyDB based=
 graphDB and create a JSON file with all data.
> - shutdown falcon-0.9, upgrade Falcon.
> - use 0.5.4 version of titan-berkeleyje to read JSON file and repopulate =
berkeleyDB based graphDB.
> - restart falcon-0.10
> I will work on the tool, and update release-notes, upgrade instructions a=
ccordingly.=20


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)