[ https://issues.apache.org/jira/browse/MATH-639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073554#comment-13073554 ] Luc Maisonobe commented on MATH-639: ------------------------------------ The expected result quaternion computed to high precision manually and checked afterwards is: q0 = 0.62283703596082005783621150 q1 = 0.02577076214564987845778149 q2 = -0.00000000025030122695149900 q3 = -0.78192703908611094998656730 > numerical problems in rotation creation > --------------------------------------- > > Key: MATH-639 > URL: https://issues.apache.org/jira/browse/MATH-639 > Project: Commons Math > Issue Type: Bug > Affects Versions: 2.2 > Environment: Linux > Reporter: Luc Maisonobe > Assignee: Luc Maisonobe > Fix For: 3.0 > > > building a rotation from the following vector pairs leads to NaN: > u1 = -4921140.837095533, -2.1512094250440013E7, -890093.279426377 > u2 = -2.7238580938724895E9, -2.169664921341876E9, 6.749688708885301E10 > v1 = 1, 0, 0 > v2 = 0, 0, 1 > The constructor first changes the (v1, v2) pair into (v1', v2') ensuring the following scalar products hold: > == > == > == > Once the (v1', v2') pair has been computed, we compute the cross product: > k = (v1' - u1)^(v2' - u2) > and the scalar product: > c = > By construction, c is positive or null and the quaternion axis we want to build is q = k/[2*sqrt(c)]. > c should be null only if some of the vectors are aligned, and this is dealt with later in the algorithm. > However, there are numerical problems with the vector above with the way these computations are done, as shown > by the following comparisons, showing the result we get from our Java code and the result we get from manual > computation with the same formulas but with enhanced precision: > commons math: k = 38514476.5, -84., -1168590144 > high precision: k = 38514410.36093388..., -0.374075245201180409222711..., -1168590152.10599715208... > and it becomes worse when computing c because the vectors are almost orthogonal to each other, hence inducing additional cancellations. We get: > commons math c = -1.2397173627587605E20 > high precision: c = 558382746168463196.7079627... > We have lost ALL significant digits in cancellations, and even the sign is wrong! -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira