transform normal to eye space

Vertex position coordinates can be transformed from object space to eye space using model-view matrix. However, normals can not be transformed in that way.

The two pictures shows what happened if we transform normal using model-view matrix:

In eye space, tangent direction is correct, but normal is not perpendicular to tangent. Why is that happening?

The math equation can explain it easily. Assume T is tangent, MV is model-view matrix. P1, P2 are two vertices used to calculate tangent.

T = P2 - P1
T' = T * MV = (P2 - P1) * MV = P2 * MV - P1 * MV = P2' - P1'

T’ keeps tangent attributes, while normal does’t. You can find point Q1, Q2 to let N=Q2-Q1, after transformation, Q2’-Q1’ is not perpendicular to T’. The angle relationship doesn’t retain after object space to view space transformation.

We need to calculate normal transformation from object to eye space, after which the transformed normal is still perpendicular to transformed tangent.
Assume the normal transformation is G, model view transformation is V, normal is perpendicular to tangent, so

N'.T' = (GN).(VT) = 0

Vector dot product written as matrix multiplication:

(GN).(VT) = (GN)^T(VT) = (N^TG^T)(VT) = N^TG^TVT = 0

Notice that N^TT is 0(N is perpendicular to T, so N.T = 0, hence N^TT ＝ 0), if G^TV = I, above equation is true, so G=(V^-1)^T. G equals to V only when V(model-view matrix) is orthogonal.

normal matrix is the transpose of inverse model-view matrix, this is a very important conslusion that you should remember, because it is often used when calculating lighting.