Tensorflow in 3D
TensorFlow is not just a deep learning library - it is a library for performing manipulations on numbers, and as such it can perform tasks that many other libraries can. In this lesson, we take a look at using TensorFlow to perform manipulations on 3D objects.
A 3D object can be modelled as a series of triangles in three-dimensional space, that we normally refer to as (x, y, z). These names are not required, but are commonly used. A triangle is created from three of these 3D points. A point itself can be represented as a vector with size (3,). An array of these is a matrix of size (n, 3), where n is the number of points we have. Let’s dive in and see a basic cube. We will need this functionality later, so let’s create a function that plots out a basic shape:
If you are using Jupyter Notebooks, I recommend running this line of code, which gives you an awesome interactive 3D plot. Left-click-and-drag moves around, right-click-and-drag zooms in or out.
Now let’s create a shape. The function below will return the six points that make up a cube. If you go back to the previous function, you will see the Delaunay line, which turns these points into triangles, so that we may render them.
Now let’s put those pieces together and see what it looks like:
I’m just showing an image here, but you can see the cube, and that is has been turned into triangles by our code and coloured differently (depending on the z-value). This is great, but now let’s use TensorFlow to do some operations on this.
A translation is a simple move: up/down, left/right, forward/backwards, or some combination of these. It is created by simply adding a vector to each point. If you add the same vector to all points, then the whole object will move in unison. Check out our lesson on broadcasting to understand what happens when we add our translation vector with size (3,) to our points matrix of size (n, 3).
A rotation is formed by creating the dot product or a rotating matrix and the original points. Rotating an object first requires you to determine which axis you are rotating over. To rotate around a particular axis, set that axis’ value to zeros, with a 1 in the related axis. There are three matrices you need:
Rotating around the x-axis
Rotating around the y-axis
Rotating around the z-axis
In the above, theta is the number of degrees to rotate the object. You take a point, compute the dot product with one of these matrices, and then the point is rotated around the relevant axis. We can also compute the dot product of one of these (3, 3) matrices against our (n, 3) points matrix. However, for a dot product to work, you need to have the inner dimensions matrix (and n is not necessarily 3). Therefore, you need to put the points matrix first - then it becomes a dot product of a (n, 3) matrix with a (3, 3) matrix.
With simple matrix manipulations like these, but combined and at scale, you can create a whole range of transformations to 3D objects like this. Shearing, scaling, intersections and more are possible using this concept. GPUs are very good at doing these transformations, which happen to relate to the same types of transformations needed for data analytic work such as deep learning. Due to this, TensorFlow works nicely with GPUs, which works nicely with 3D objects as well as for deep learning tasks.
- Create a different object, such as a pyramid or hexagonal prism. If you aren’t sure how to start, start with a prism and create it in 2D first.
- Rotate the object around the x and y axis.
- You can combine rotations into a single transformation matrix. To do this, simply compute the dot product of the rotations.
- Does order matter for the question 3?
- A shear matrix is the identity matrix with a value in the off diagonal. An example is below. Create a shearing matrix and test different values.