c - Calculating facing arc between two 3d rotational matrixes -

i working on space-ship simulator, , having trouble facing arcs between 2 space objects. each object has rotation matrix defined follows:

//top row rotation[0][0] = cos(pitch)*cos(yaw); rotation[0][1] = -sin(yaw)*cos(pitch); rotation[0][2] = sin(pitch); //middle row rotation[1][0] = cos(yaw)*sin(pitch)*sin(roll) + sin(yaw)*cos(roll); rotation[1][1] = -sin(yaw)*sin(pitch)*sin(roll) + cos(yaw)*cos(roll); rotation[1][2] = -cos(pitch)*sin(roll); //bottom row rotation[2][0] = -cos(yaw)*sin(pitch)*cos(roll) + sin(yaw)*sin(roll); rotation[2][1] = sin(yaw)*sin(pitch)*cos(roll) + cos(yaw)*sin(roll); rotation[2][2] = cos(pitch)*cos(roll); 

as having xyz coordinates.

using information, need able decide 90 degree arc facing between object1 , object1: forward, starboard, port, aft, dorsal (upper) , ventral (lower) trying use following formula:

arc = acos( sum(a*b) / ( sqrt(sum(a * a)) * sqrt(sum(b * b)) ) ) 

where |a| vector direction of |object2| - |object1| , |b| row1 (forward arc), row2 (starboard arc) or row3 (up arc). doing wrong because it's not behaving expected. asking find making mistake.

p.s. yaw, pitch , roll stored in degrees (0-360), , converted rads multiplying pi / 180.0.

i think you're approaching problem wrongly describing bearing in terms of roll, pitch , yaw. these euler angles describe movement of spaceship in terms of ship's current coordinate system itself. these angles not independent , cause problems when 1 of them ±90°. roll, pitch , yaw useful if want describe movement of ship.

i model case differently: each ship has position {p}, given in global cartesian x, y, z system. has bearing, given 3 orthonormal vectors u, v, w in global space. you'll have rotation matrix:

      | ux   vx   wx | [r] = | uy   vy   wy |       | uz   vz   wz | 

you can describe position of other object position s in local coordinate system of ship:

{s} = [r] · {s - p} 

notes:

• you can determine way ship see other object best determining maximum absolute component of {s} = {su, sv, sw}. example, if s = {-12, 8, 10}, object behind you. (or, perhaps more precisely, more behind in of other 5 directions; aft principal direction) if s = {2, -5, 8}, object below you.

• you use u forward, y starboard , z up. describes left-hand system. recommend using right-hand system, because that's cartesian systems conform to. eaxample, use u forwards, y starboard , w down.

• you can express cartesian vector {s} in spherical coordinates zenith in w direction. give distance of object r , 2 independent angles, azimuth in ship's (u, v) plane , zenith describes elevation of object plane.

• when ship moves, can update global bearing means of roll, pitch , yaw.

• you can still calculate arcs want formula above. use {s} = {su, sv, sw} in ship's coordinates , reference axes u = {1, 0, 0}, v = {0, 1, 0} , w = {0, 0, 1}:

  uarc = acos(su / r) varc = acos(sv / r) warc = acos(sw / r) 

  here, r distance, r² = {s}·{s}. arcs calculate not useful. tell angle between axis , object, these angles ambiguous. better calculate angles in planes, e.g.

  uvarc = atan2(sv, su) vwarc = atan2(sw, sv) wuarc = atan2(su, sw) 

  the zero-degree reference second axis in expression, i.e. u uvarc. gives angle of distance vector projected onto plane.