186. 3D Animation Editor

So now I’ve begun building built a 3D animation editor.

I had many challenges to overcome, as you will see.

The two approaches to rotating joints

The main problem with animation is, of course, rotating the moving parts.

There are two obvious approaches.

Both of them share the concept that there is a set of joints inside the mesh, specifically, for toes, ankles, knees, hips, shoulders, elbows, hands, and head. I will need more to control movements of the body itself (ie chest, stomach etc), but this will get me started.

These joints will be used to build bones. So the lower leg bone will be a straight line between the knee joint and ankle joint, and so on. The straight line itself is not important, but the angle is.

When I move a joint, Codea will calculate the new angle between the affected bones, and rotate the affected mesh vectors by the change in angle from the original position.

1 Rotate individual vertices

Using this approach, each vertex in the mesh is rotated individually. Most of them rotate in line with the bones around them, but vertices near the joints of two bones need to be influenced by the rotation of both bones, so the mesh bends smoothly at the joints. This is how major animation programs do it, and there are usually extra vertex layers around joints so the mesh can stretch smoothly.

This is the ideal way to do it, but the problem in Codea is that if you have to adjust every vertex individually in each frame, it slows to a crawl. It may be possible to do it using a shader, but I haven’t explored that.

2 Rotate whole body parts

This requires breaking the mesh into separate pieces, one for each bone. Then each mesh rotates along with its own bone, as joints are moved around.

This is simpler than the first option, and will run fast, but has the problem that as joints move, gaps may appear in the mesh, as body parts rotate away from each other. This is one reason why in many games, characters have big shoulder pads and belts, to cover any “holes” caused by moving arms and legs!

Anyway, this is the way I did it.

Choosing and preparing a model

You can see I chose a big chunky robot as my first model. This was mainly because it had big shoulders and hips that would rotate without leaving holes, and clearly defined arms and legs that should be easy to split off. (I would have preferred Batman, but his costume is too tight, and his cape would be difficult to work with).

It took me several hours to find the robot, because free 3D models are plentiful, but most have one problem or another that makes them unsuitable.

Then I had to break up the robot into all its parts. I used Blender for this, splitting the model into named parts, then exporting to OBJ format, which my program can read. Splitting the model was more difficult than it looked, because it had a lot of extra vertices inside it, probably to help with animation, but they just got in the way. I got it nearly right, but I did miss a few vertices, which you may see floating in mid air when I rotate a limb. That can be cleaned up with a bit of effort.

Editor design

I borrowed some of the editor design from Blender.

There are four standard views of the model – front, left side, right side, and above. You can also use a circular fingerpad to rotate the view in any direction.

The joints are drawn as little red squares inside the model. I positioned them by hand, ie by moving their positions until they looked right. They need to be visible, so I made the model transparent so the joints show through it (actually, I have a slider so you can vary the transparency). This transparency is provided by a very simple shader.

You select a joint by touching it.

Once a joint is selected, you can drag on the set of red/green/blue lines (bottom right corner) to rotate the joint in any direction (subject to any restrictions, eg knees can’t rotate sideways).

I also provided the ability to rotate and/or the whole model in any direction, using two more red/green/blue controllers.

Finally, I provided the ability to save the current joint settings, which enables me to create a set of frames with different body positions that can be used to make an animation.

Selecting a joint

It is not easy to touch a 3D object via a 2D surface – think of standing in front of a window, looking out at the garden, and selecting a tree outside by touching the window pane – in fact all you are doing is choosing a direction, not a particular spot in the garden, and it is not clear what you have selected. Additionally, when you draw on the screen in Codea, you end up with a mess of coloured pixels, but no objects, so Codea has no way of knowing what object you touched.

Fortunately, I’ve solved this problem previously. When you touch the screen, I draw the next frame to an image in memory, and I only draw the joints (in case you were wondering how I manage to touch a joint that is inside a solid model), and each joint is coloured with an individual colour so I can identify it. Then I look up the pixel colour for the spot I touched, and from that, I get the joint.

Rotate or drag joints?

For some reason, when I first built the editor, I selected a joint, then dragged its position along the x, y or z axis. This caused problems because bones (which connect two joints) have a fixed length, and dragging joint positions makes bones longer or shorter, so I had to use special formulae to figure out what position would keep the length the same.

It was a pretty stupid way to do it.

Eventually, with some help from a graphics book, I realised I should simply rotate the selected joint (and also rotate any other joints further down, so a shoulder rotation also affects the elbow and wrist). I could also restrict rotations so that limbs could not rotate in unrealistic directions, eg knees can only rotate backwards, not sideways or forwards.

This works much better, and most of the time, I don’t get gaps opening up when joints rotate, thanks mainly to choosing a robotic model with big shoulders and hips.

Rotations

This editor is full of rotations. And I hate rotations. I really hate rotations.  (Did I tell you I hate rotations?)

Each joint can be rotated individually, and if a joint further up the limb is rotated, it needs to have that rotation as well (eg a rotation to the shoulder has to be passed on to the elbow and wrist joints as well). Then of course, the entire model can be rotated as well. Then there is the camera view, which is another rotation, which can be further rotated manually.

That is a big stack of rotations (each of which can be any combination of x,y and z rotation), and I spent many days sorting them out. I used quaternions, which are marvellous, if totally incomprehensible. I am not going to detail my approach, because I doubt anyone is interested, but you can contact me if you want more detail.

 Final result

Below is my first animation. It is only 5 frames long, and Codea interpolates between them to keep it smooth. It runs at a respectable 45 frames per second on my iPad 3.

(If it looks a bit jerky as it rotates around, that’s because I was rotating manually, not because the animation is jerky).

How long did it take? Probably upwards of 100 hours.

Why did I do it? Because I’m m-m-m-mad, probably. But I did want to know what it would take to produce genuine 3D animations in Codea. I think I’ve proved it can be done, despite the lack of 3D animation tools in Codea (which is not a criticism), but it is very, very, very hard, and you need a lot of skills to achieve it.

Will I share the code? Not at this stage. It’s too messy at the moment, for a start. And I need to figure out where I go with this.

 

 

Advertisement