Week 4: Learning about matrices.

· Please download and unzip the Week4.zip file in the Files/WeeklyExamples folder.

1. Access to Transform pipeline:

a. The “Tree” is just a piece of geometry for fun (Createà3D ObjectàTree)

i. Play around with it to get something

ii. Name as: 451Shader

b. Tree has 451Material as its material

c. 451Material has 451Shader as its shader

d. Custom shader

i. Create by: RMBàCreateàShaderàUnlit Shader (simplest entity we can work with)

ii. Rename to: 451Shader

e. Note on 451Shader

i. MyVert() and MyFrag()

f. MyFrag(): returns a constant blueish color, that’s why the tree is constant flat blueish. No fun L

g. MyVert()

i. v.vertex è Vertex of the GameObject (the Tree)

ii. Try this: o.vertex = mul(UNITY_MATRIX_MVP, v.vertex);

iii. We can control which of the matrices we want to respect

1. MVP: concatenated of Model View and Projection

a. Model: GameObject Transform TRS

b. View + Projection: Camera transform

iv. Try things: If only multiply with VP, Tree’s interactive Transform does not have any effects!!

Shading Engine Architecture: First pass. What are Vertex and Fragment shaders

2. Translation, Rotation, and Scaling:

a. XformLoader (Assets/Source/Model)

i. Attempts to “load” a matrix into the shader with: mMaterial.SetMatrix("MyXformMat", m);

ii. Attempts: that the shader has defined a MyXformMat

iii. NOTE: UI manipulator updates TRS values at same time (independent from what you change!!). E.g., changing rotation may result in changing position!? Why?

b. CSS451Shader:

i. Line-31: float4x4 MyXformMat;

ii. MyVert(): vertex program

1. Uses MyXformMat to transform incoming vertices

c. Tree has two special components: that works hand-in-hand (inter-dependent):

i. 451Materials (which has a 451Shader)

ii. XformLoader-script which knows how to load the MyXformMat in the shader

d. Sphere has 451Materials but not XformLoader

i. No way to communicate content of MyXformMat to the shader

ii. Cannot manipulate its transform

iii. Hide the Tree object (or move the tree object away when game runs) to see the blue default sphere not being transformed

e. Cylinder has the default shader with XformLoader

i. The GameObject tries to load MyXformMat into the shader, but nothing with that name is defined in the shader, so the attempt is ignored

ii. Can only manipulate the Cylinder as usual, XformLoader is useless.

f. Note: MyXformMat: is per-Material per-GameObject

i. Tree loads MyXformMat, Tree sees

ii. Sphere does not load MyXformMat, Sphere does not see

3. CPU-TRS: we can compute the results of TRS

a. Editor: ThePlane, Corners

i. ThePlane: the plane we can manipulate

ii. Corners: Given ThePlane transform, we can compute the location of its 4x corners, display the trees at these corners

b. Observe: run to see

i. The four trees are moved to the 4x corners of the plane, always pointing in the normal direction of ThePlane

ii. You can select and manipulate ThePlane to verify

c. Note: The plane is a Quad

i. Quad is originally defined on the X/Y plane

ii. A 90-degrees about X-axis rotation is performed on the Quad to rotate it the X/Z plane

d. Tree: material is Unity Default!!

i. This means, XformLoader is useless!!

ii. Try, replace Unity default material with 451 materials!

e. Math: all in TheWorld.cs::Update()

i. Compute m = TRS minus the original 90-degree rotation of the Quad

ii. Use the m to transform initial corners (-0.5 to 0.5) in X/Z

iii. Set the tree’s rotation (orientation) to that of the rotation corrected rotation

f. What we learned?

i. We can compute the effects of a Unity Transform in the CPU

1. This is IMPORTANT!! Often times, e.g., in MP4, we want to perform this operation to figure where things are.

ii. We delegate the computation to the GPU because GPUs are optimized to perform that operation.

4. About rotation matrices:

a. Editor: ThePlane, Tree, X/Y/Z-Axis

i. ThePlane: for sanity check orientation

ii. Tree: the object where we will track its X/Y/Z axis

iii. X/Y/Z-axis: three cylinders with RGB colors, we will draw these on the X/Y/Z axis of the tree

b. Observation: run and manipulate the Tree

i. Notice X/Y/Z axis showing up as RGB columns properly represents the axis of the Tree

c. Math: in TheWorld.cs

i. DrawCylinderAsVector(): Rotate cylinder’s up to align with V and move the cylinder so that it is on the side of the V direction

ii. Update()

1. m: Convert the Tree rotation quaternion into a matrix

2. Get the Column’s of m and display as X/Y/Z axis!?

d. NOTE: Rotation matrix (a matrix that describes rotation)

i. Column/Row: all of size 1

ii. Column are vectors that are perpendicular to each other

iii. Row are also vectors that are perpendicular to each other

iv. Since we are working with column-vectors,

1. Column-0 encodes the rotated X-axis [This is also Transform.right]

2. Column-1 encodes the rotated Y-axis [This is also Transform.up]

3. Column-2 encodes the rotated Z-axis [This is also Transform.forward]

v. We have been taking advantage of this fact for a while now!!

1. Cylinder.Transform.up (the Y-axis)

2. Week2.Math, Example 4 (Plane Normal), we took GetColumn(2) as the rotated Z-axis as normal!

e. What we learned?

i. We need to know is the initial orientation, e.g.,

1. initially Cylinder’s height is in the Y-direction, or

2. initially Quad’s normal is in the negative z-direction

ii. After transformation, we can find the current object’s orientation by looking at the column vectors of its rotation matrix, e.g., After some transformation,

1. Currently, the Cylinder’s height is in the Transform.up (or Column(1) of the rotation matrix) direction.

2. Currently, the Quad’s normal is in the negative Transform.forward (or Column(2) of the rotation matrix) direction.

5. Concatenating Matrices: TS vs ST

a. Editor: Tree

b. Note: still working with our own shader and Xformloader

i. Shader: 451Shader that looks for MyXformMat for model transform

ii. XformLoader.cs: Tree has this as its script, loads MyXformMat

c. Observation: Editor, with Tree selected, notice you can switch ST vs TS, try it!

i. Move and scale the Tree and observe what is going on!

d. Math: XformLoader.cs::Update() [attached to the Tree]

i. Build the sm (scaling matrix), and tm (translation matrix)

ii. Compute either one of the following:

1. cm = tm * sm

2. cm = sm * tm

e. Note:

i. Since we are working with Column vectors

ii. Result = cm * Vertex

iii. The matrix on the right is applied FIRST!

f. What we learned:

i. Matrices can be concatenated (“multiplied”).

ii. The effects concatenated matrix is the same as applying the individual matrices in the same order

iii. Order of matrix concatenation is important

g. TS (apply scale first the translation) seem much more intuitive to human

i. That’s why TRS (apply S first, followed by R, and lastly by T).

ii. No “law” says we have to do this, it just makes sense to a normal human

6. Inverse Transform: and

a. Prep: Color on 451Shader

i. has a MyColor variable that can be set

ii. Notice the “Properties” field, simple way to communicate to the Unity Editor

iii. Easy to create multiple 451-Materials with different shader color

1. Create material, drag 451Shader onto, and set color

b. Editor setup

i. Blue: reference tree, this is the one we will set transform

1. Green, Red, White: uses Blue’s transform to compute themselves

ii. Green: invert out T of blue reference tree

1. and then translate itself to positive-X position, for clarity

iii. Red: invert out both R and T of blue reference tree

1. And then translate itself to positive-X/Z position, for clarity

iv. White: invert out all of S, R and T of blue reference tree

1. And then translate itself to positive-X/Z position, for clarity

2. This is the original tree with no transform

c. Script: XformLoader.cs

i. With reference to BlueRefTree’s transform

ii. Able to show Blue’s TRS + invT, or invRT, or invSRT

iii. Blue: is set to TRS: so respect all transform

iv. Green with invT: so, does respect RS

v. Red with invRT: so only respect S

vi. White invSRT: does not respect any transform on BlueRefTree

vii. Algorithm: compute TRS and then apply inverse!

d. Try: manipulate Blue and notice:

i. Green: respect SR

ii. Red: respect S

iii. White: does not respect anything

e. What we learned:

i. We can invert TRS, by applying the inverse in inverse order:

ii. Inverse of TRS is

iii. So what? … we can “invert” the effect of an operation … so we can move object to some place, apply transform, and then move back … pivoted transform

7. Pivoted Scaling and Rotation

a. Editor: Tree, Pivot

i. Pivot: has a Sphere and three cylinder representing the axis.

ii. Pivot: is under control of the XformLoader.cs

b. Observe: Editor, select Tree (only object under control): UI (XformLoader.cs component)

i. DrawPivot: this is connected to the Pivot GameObject

ii. Pivot Offset: transformation’s pivot position (an offset from the localPosition)

1. Try changing this to move the Pivot position (e.g., (5, 0, 0))

iii. Enable ShowRotation and/or ShowScale

iv. Observe the pivoted transform

c. Math: Xformloader.cs

i. We are computing: -P * T * R * S * P

d. What we learned:

i. Matrices do concatenate in general

ii. Order of Matrix concatenation is important

iii. We can have rather fine control over how to transform objects

8. Rotate/Scale Cylinder: Apply/Try out what we learn

a. Problem: Cylinder’s reference (or Pivot) is in the middle in the height direction

b. Want: Pivot to be located at the bottom in the height direction

c. Editor: MyCylinder and DefaultCylinder

i. MyCylinder: IMPORANT

1. 451Material: to use 451Shader (using MyXformMat instead of the default Unity TRS)

2. Controls DefaultCylinder as a reference of default behavior

d. Observe: in Editor, select and rotate/scale MyCylinder to see

i. MyCylinder is transformed with respect to the bottom of the height direction

ii. DfaultCylinder: has EXACTLY the same Transform (except slight position difference): transformation is with respect to the center in the height direction

e. Math: nothing new AT ALL!! J

f. What we learned:

i. PivotOffset: is with respect to the object BEFORE the Scale operation!!

1. In this case, PivotOffset (0, -1, 0) does not change even when we scale the MyClinder in the Y direction by a large amount

ii. Be careful! VERY Careful …

1. MyCylinder’s transform must be derived from scratch (DO NOT look at the default Transform settings!!)

2. For example, with to create a cylinder of height 10, with bottom touching the X/Z plane, you want to consider:

a. A reference Cylinder, Pivot at (0, -1, 0), Height of 2. Bottom position by default also located at (0, -1, 0)

b. If we scale by (1, 5, 1), we know

i. Bottom position (the Pivot) is not going to move.

ii. Result is a cylinder of height 10, with bottom position at (0, -1, 0)

c. Now, translation by (0, 1, 0) è A cylinder of height-10, with bottom touch the X/Z plane

d. Transform setting for MyCylinder:

i. T(0, 1, 0)

ii. R(0, 0, 0)

iii. S(1, 5, 1)

iv. P(0, -1, 0) [to modify the default pivot of (0, 0, 0)]

iii. This stuff actually works! Cool?

9. Parent-Child Transform

a. Note: edit 451Shader to add in a new Uniform “MyColor” to provide color control to the GameObject

b. Editor: Tree (with TreePivot) and Base (and BasePivot), want to imagine as …

i. Tree is on top of Base

c. Observe:

i. Transforming the Base causes the Tree to follow (e.g., switch on show rotate/scale) on both the Base and Tree

ii. Origin position for the entire structure is at the base’s origin/pivot (0, -1, 0)

iii. Try: changing

1. Option 1:

a. Do set:

i. Tree Translation to (0, 0, 0)

ii. Rotate tree: now tree not on top of base (not surprisingly)

b. See that:

i. Tree translation is achieving the effect of it being in the hierarchy (on top of the base)

ii. Since this translation is used, there is no way to move the tree around a little without affecting its relationship to the Base.

2. Option 2:

a. Do set:

i. With translation at (0, 0, 0)

ii. Tree Pivot (0, 1, 0): rotate the tree, see that, NOT achieving the same effect as moving the translation position of the tree.

b. See that: translation and pivot are doing two different things, translation _MOVES_ the object, pivot sets the rotate/scale center (no actual movements!!)

d. Math: XformLoader.cs

i. Sets Color into shader “MyColor”

ii. Now has a Parent reference (can be null)

iii. If Parent xform is defined, transform the GameObject by: p * m

1. p: parent transform

2. m: our own transform

iv. Note, PivotPosition, must be transform by parent’s transformation

1. Note the Matrix.MultiplePoint() function! J

e. What we learned:

i. Pivot position of the child must also obey parent transform,

1. In our case, since Pivot is NOT a 451 material, we must compute the transform by ourselves.

ii. Parent and Child transform are accomplishing two tasks

1. Transform the shape (the cylinder or the tree) into what we want

2. A transformation for the kid è We need two separate transform here!!

a. One to change the shape

b. One to act as a parent for the kid

iii. Something is lacking: Having the base origin at (0, -1, 0) is not convenient.

1. This is governed by the cylinder’s need

2. BUT … Two serious limitations:

a. Cannot change the cylinder without affecting the Tree

b. Cannot add more elements at the “Base” level, how to add siblings to Base?