Pardon me for posting, but had a semi-technical question…a few games (mostly console games) these days use a technique called “Cel rendering,” which makes 3D sprites look as if they were animated by the old “ink-and-paint” method, like a cartoon. My question is: Is it any easier or any harder on a computer’s software and hardware to animate a game like this, as opposed to conventional 3D rendering?
Well, thanks for your time,
Ranchoth
It doesn’t look it, but believe it or not it’s quite a bit harder.
There are two components to the cel shading, the most obvious one is the flat coloring, where the object appears as a flat color, possibly with highlight and shadow areas. This part is actually pretty easy. You take the angle between the light and the “surface normal” (a vector that is perpendicular to the surface of the object) and look it up in a table to decided which color (bright, normal, dark) to give it. This is pretty easy to do.
The harder bit is drawing thick black outlines around all the model. I’ve seen a few ways to do this. One way that is pretty efficient, but leaves some ugly artifacts is:
Take the model and make it, oh, 5% bigger.
Take all the polygons that face the camera and throw them out
Paint the other ones flat black
Then draw your original model
This technique works ok, but its not as nice as this next one…
Find all the “sillouhette edges” in the model (any edge that seperates one triangle which faces the camera from one that doesn’t face the camera), and paint these with a thick black line
Then draw your original model
Finding the sillouhette edges is the hard part of this technique, and I’ve seen a few papers and discussions about how to make this fast.
OK, so thats the traditional techniques, and they are mostly hard because you are trying to subvert the normal way the graphics hardware works. It just wasn’t designed for toon shading, or other drawing techniques generally referred to as “nonphotorealistic rendering”. In the future though, you’ll hopefully be seeing a lot more of this stuff, because hardware manufacturers are undergoing something of a paradigm shift, over to programmable graphics hardware. With the new hardware, you send it little programs for how you want your triangles or your pixels manipulated, and it will do whatever you want. So something like the toon shading can be done much more efficently than before.
That sounds interesting. Can you give an example of a mainstream game that makes use of this cel rendering?
I’m a software engineer, not a Graphic Artist or Software Graphic Artist.
While I could find no hard-and-fast answer to your question spelled out, I perused the site http://www.cc.gatech.edu/classes/AY2002/cs4451_spring/groups/group2/index.html and then read the paper at http://www.cs.unc.edu/~harrism/npar/npar2000_lake_et_al.pdf in depth.
In my opinion, when it comes to ‘real-time’ rendering, it ought to take less time (CPU power, whatever) to (using the words in the paper) find a transition boundary and paint solid colors to either side, instead of smoothly interpolating shading across a model.
And, on preview, in a jocular response to Hunsecker about thick black lines around the model, you can get around that by working on Samurai Jack 
…which in case you didn’t get the joke has broken some ground with a distinctive style of not outlining the animations. Then again they watercolor their cels, IIRC.
AmbushBug
[sub]i am a constant irritant to the Graphics people where i work, take that as you will…[/sub]
Well the original dragons lair space ace was cell shaded… but what started the current trend is the jet set radio games
But theres other games out there as its coming back as a trend
Jet Grind Radio (DC) was the first cel-shaded game I saw, but there’s also cel shading in Cel Damage (GC?), Frequency (PS2), Klonoa 2 (PS2), and my favorite example, The Legend of Zelda (GC).
I’ve also admired Marvel vs. Capcom 2 in the arcade, which appears to be cel shaded, though I can’t really tell how the background graphics are produced, or if they’re even 3D models at all.
True, if you were to do toon shading on your CPU it would be less work. You don’t have to deal with textures or lighting, so you’re interpolating fewer values across a triangle. (Sillhouette edge detection is still a problem, but like you say, you could go with the Samurai Jack look)
The problem is that graphics hardware was not built with the flexibility that toon shading requires. For toon shading the dot product of the light and the surface normal are interpolated across the triangle, and this value is used as a texture coordinate. Using the standard 3d rendering pipelines (like OpenGL, D3D) there was no easy way to use the interpolated dot product as a texture coordinate, so you had to resort to slower methods.
As the UNC paper you cited says: “a hardware pathway between the lighting and texturing subsystems and an associated automatic texture-coordinate generation mode would be beneficial for cartoon shading.”
Like I mentioned in my post, new graphics processors are doing away with the idea of a “fixed function” rendering pipeline, and letting the programmer take control of how each vertex gets transformed and how each pixel gets rendered. So hopefully with this new power you’ll see a lot more of these techniques in the future, and computer graphics will become more expressive.
I’m still waiting for a game done using the pencil sketching effect. 
For pictures of the pencil sketching effect, and Non-Photorealistic Rendering in general, go here: Stylized Depiction: Non-Photorealistic, Painterly and 'Toon Rendering
I’m out of my depth, anyway. Luckily, I leave this sort of stuff to the Maya- and PhotoShop- trained people at my work.
Does this qualify? Replace the OpenGL driver with a pencil-rendering engine, “rendering” Quake in pencil. Or, potentially other OpenGL things. There are some other sections to that webpage also.
AmbushBug
The next Zelda game form GC, I believe, is to be cel shaded.
Yeah, I played with NPR quake before, and it was pretty amazing, mostly because of the way they just stuck the NPR inbetween the game and the graphics card. So true, I guess there has been a pencil sketched game. There was also another game that I believe won an Independent Game Developers award either this year or last which was a lot like doom, but everything was hand drawn.
The more recent pencil sketching algorithms have a real subtlely to them that the above two games lack (for example, they can more accurately simulate shading by adding more strokes), and I’ve seen demos of them running in real time, but nothing beyond proof-of-concept yet.
Also, the game I saw that makes best use of the cel shading so far was Cel Damage. It was a driving combat game, much like twisted metal, but set in a cartoon world, so weapons were things like giant hammers held by big cartoon hands, or a big axe.