When I start with a new feature I like to start with simple simple data and figure out what things do before I start to apply that knowledge to more complicated projects. to that end we have this scene here:

it's not a fantastic image no but it does have what we need: data.

lets look at the image as if it had taken a long time to render, and something we don't want to have to spend a lot of time re-rendering again, lets presume that Bob down at Bob's house of Floating Sphere's has us on a really tight deadline, we've been working late nights and some how we accidentally pushed our fill light from 20% to 200% we set it up to render and went to sleep.

when we woke up out white spot on the sphere was completely blown out and our shadow was not dark enough, and did we really mean to make the key light that color? but oh no we don't have the time to re-render this for bob!

Luckily we set up our project to render out to OpenEXR files with light buffers in them so we can make some adjustments. How did we set up our renders this way? So glad you asked!

Render Setup

getting buffers out a render is as easy as one two three
1) set the render type to OpenEXR Sequence
2) set the Buffer for Light Buffers to On and change it to Each light in it's own buffer (we'll save the other options for another day )



3) Turn Multi-Pass rendering on.



With these settings A:M will break out the diffuse, specular, and shadow elements of each light into separate buffers (think of them like layers in photoshop) each of which we can put to work in A:M Composite.

Starting a New Composite

A:M Composite is a special image type that lets us put images together in ways other than the standard RGBA we can use Post Effects to change aspects of any image we put into a composite, allowing us to make the adjustments that will make Bob happy without the time it takes to re-render the whole scene.

It is recommended that you composite in a new project file so after you finish rendering start a new project. In order to use composite the we need to load in the sequence of exr files we rendered out, this works just like any sequence of images in A:M. Right-Click (control click on mac) on the Images folder and choose Import->Image Sequence. You'll see new composite in there as well but ignore that for now.

when you have the sequence in the project take a look at what we have as far as data goes:

here we see all the buffers that A:M split out for us:
Color Alpha
Diffuse
Ambience
LightDiffuse1
LightDiffuse2
LightDiffuse3
LightSpecular3
LightShadow3

The first buffer, Color Alpha, is actually the image as A:M would output it to any format it has the data from all the lights built right in, we don't use this much in compositing, though it does come in handy as a reference to what you started from.

Diffuse Contains all the color data from the scene without any shading factored in (this is pretty much the same as a flat shade render out of A:M)

Ambience Contains all the ambient light data from the scene. any objects that have ambience set as a surface property will have that show up in this buffer

Then each light gets a buffer for the options on it that were enabled. In this scene Light 1 and Light 2 have diffuse properties only while Light 3 has diffuse, specular and shadow properties.

Now we could go in and laboriously build up a composite for these elements to allow us to start tweaking elements by choosing new -> composite from the image folder's contextual menu. Or we could just do it the easy way. Ok easy way it is. Bring up the contextual menu for the EXR sequence in the PWS and choose build composite and A:M will automatically put together a composite for you based on the data in the sequence.

What you get is a series of 'Mix' post effects each of which has two children which are 'mixed' together to produce a final image. you can adjust how much a particular buffer contributes to the final image by simply changing the value of it's Mix percentage.

Ok I need to go get some work done (that bob is a task master! ) so I hope this will tide you guys over for now, and I'll be back with a continuation of this composite later.

-David

I think I'm starting to understand compositing now! I can only imagine how many images you're going to have when it comes to animating with it!
I think a feature request of mine will be to group, for example, the spec/shadow/diffuse for a light under the light name. Otherwise the PWS will go on forever.....I think.

Thanks for the info David!

I've experimented with compositing in v12 but I think I need someone to hold me by the hand a little in this new territory. I keep trying to use it in ways I don't think it was intended to be used. I think my brain is trying to use it as a linear film editor.

I just need to know what success looks like I guess and your efforts here are sure to help.

Thanks again.

Thanks for the intro to compositing, David. Though I don't have Version 12 yet, I can certainly see how useful this will be. I just hope the version of Shake that I have will be able to import the files. I'm looking forward to the next tuts you put together.

Sterling

David, thanks alot for the compositing tut. I really look forward for this am feature.
I do have a question though.

Ok I think i might have time for one more step:

Identifying your needs

before you get too far along with any composite you need to analyze your image and determine what you need to be tweaking, while you can just load things and and go at it trial and error to see where you wind up (JohnL you out there? ) with interesting results if you want Bob to be happy you need to identify the issues he is going to have with your render and fix those.

so back to this image


we know right off the bat that the white spot on that sphere is way too blown out, we did mention that the shadow could be darker, and now that I look at the work order... bob really dislikes yellow lights, he said "I'm a cool guy i want my patented floating sphere to be lit with cool colors!" oops we will need to fix that.

those three things are probably enough to start with.

Once we have our needs laid out it's time to jump into the composite and find out what we have to work with.



This is what A:M built for us when we asked it to build a composite. it contains all the data needed to produce the image we already have above. so let's find the things we want to change in the data. first we need to open a window larger than the little PWS preview. Double click the Composite object in the PWS. The new image window that opens shows the result of our current composite. If, however you look at the PWS you will see that after each element in the composite is a small circular icon, this icon shows three possible states for each element: Green is the currently viewed element including any children of the tree that goes into making that element (so for now you have the root composite item selected to view and all it's children are in place.) Dark Grey/Blue indicates that a particular element contributes to the currently viewed element. Empty indicates that the element is neither being viewed nor does it contribute to the current view.

We can navigate through the layers that produce the composite by clicking on the circle to the right of it's name. The first time you click a circle it brings that element to focus making any of it's children dark grey and all other aspects empty. If you click on it again the view is toggled to the full composite. This allows us to easily go back and forth to examine what a given branch is contributing to the composite tree.

So for instance when we click on the circle next to the first mix it will make that circle green and all the following circles dark grey, indicating that the first mix is the final compositing step. Given that information I would rename the mix: final composite mix. This is done just as you would rename any element in the pws, by hitting f2 and typing in a new name. So our next step is to look at each element in the composite and rename them all so that we know what each branch is doing.



Note that I have renamed all the items under the first mix but not the element named start simple 0Composite RGBA that is because that element doesn't contribute to the final comp, rather is is the 'before' snap shot, which allows us to easily see what our changes have done to the image by showing us the original. It is this original that the post effects are acting upon to create the new composite. if you like you can rename this image to 'original'

Ok Bob's breathing down my neck again so I need to get some work done here I'll be back to continue this with an exploration of the mix branches to tackle that first lighting issue.

TTFN,
-David

This is cool!

At first I was unsure about the whole thing. I was worried about file format support in other applications...

Now... I don't think I will have to upgrade anything. Just use A:M for all my compositing needs!

Looking forward to more... like... how far will the shrapnel fly when my computer explodes if I open an animation sequence with all of those "layers"?



Vernon "BOOM!" Zehr

Wow. This is indepth stuff. I can't quite follow without getting v12. But this is really compelling me to get it real soon. This has got to be a first for a 3D app...no?

Question: Is anti-aliasing going on in the layers? The ball looks abit pixelated....

Now that we have identified our needs and the contents and effect of each Branch lets go in and fix at least one of our problems.

Adjusting Mix Levels

Now there are a few different ways in which we can handle changing the intensity of the light for the scene, but I like to work with what I have in place before I start tossing in changes. So the first thing we are going to do is change the values of our mix post effects.

I'm going to start at the all lights mixed post effect. I want to look at just this level of the tree so I click the circle next to it in the PWS here in this mix we have all the light data fed into two channels, the first is the Key light in the scene and the second is the fill light in the scene. I am pretty sure that the fill light is the source of most of my woes, so I just bump the mix percentage down for input 2. I drop it to 50% and lo and behold it's actually not so bad!



so I pop back to the root of our composite tree and see this:



huh not quite there is it? So what's going on? I changed the lights and it looked OK didn't it? well then there must be something between the lights and the base of the composite that is causing us trouble, which (in this case anyhow) leaves just one thing: the Ambience on the ball is pushing the light on the ball to be too bright. We can adjust it at the first Mix branch of our composite. So on Full Composite Mix I change input 1 (the ambience level) down to 10% and I get this:



Looks like Problem 1 is solved.

Next time: Shadows

-David

Is this in effect like bringing an alpha image into Photoshop and adjusting the brightness? Or does it allow more control than that? (Apart from the control of the shadows/diffuse/etc)

Can't wait till you get to the shadow bit!

Cool. I know I'm jumping the gun here, but if this was an animation, would that change "propagate" through every frame of the animation?

Yes.

which is why you wouldn't want to do this kind of thing in photoshop

-david

Hold Onto Your Pants!

Breaking out Shadows

Ok so the levels of the light looks good, but the shadow still looks a bit paltry down there... Bob won't like it I can tell you that right now (bob is a picky man ).

so we look at our data and see where the shadow might live... this element I renamed Light 3 Diffuse with shadow might be just the trick! lets look at it:



yep there's that shadow... but hmm looking at the composite here it's all bundled in with the diffuse elements of this element... I just want to darken the shadow here... I don't want to darken the sphere. egad! uproar! consternation!

But wait perhaps there is a way I can separate out the shadow... well if I look up here at the composite.



there is a buffer split out for shadows... it's not anywhere to be found in our composite though.. why? well it's not needed to build the image since the shadow is also right there in the diffuse element of light 3. so all we need to do is... pull the shadow out of light 3 diffuse w shadow and then put it back in with our desired adjustments to the darkness. Well we are in luck there is a simple way to do exactly that.

First we need to get those pesky shadows out of there. This is done by simply right clicking (mac users be like Obnomauk and get a multi-button mouse they really are worth the extra buttons and all, but until then you can control click) the light 3 diffuse with shadows element and choose Insert Post Effect->Hash Inc. -> Divide

This will add a branch directly above the item you right clicked on and place the light 3 diffuse with shadows element as the first input this new branch. I went ahead and renamed it Light 3 diffuse no Shadow. Now we need a second element to this branch before it will function correctly. What it does is Divide the pixels from one element from another, which if you know that shadow layers are multiplied in to adjust a level makes perfect sense but might seem weird if you don't. This is all computer graphics talk for how the image from one layer is combined with an image from another. If you are familiar with photoshop you are likely already familiar with this concept which is called 'blend mode' there. All you need to know for this (right now anyway) is that those shadows you see there are 'multiplied' into the diffuse image, and divide (being the opposite of multiply) will get em right out. So for our second element simply drag and drop the LightShadow3 buffer onto the Divide and...



HOLY CATS! look at that then, neat.

so just for reference your composite now looks like this:



See how I renamed the Divide? note that I kept that DIVIDE in the name (knowing what it does is important too no?)


Ok... we do need to get those shadows back in there, but all on their own... so how would we do that? yes it's a quiz... hope you brought your number 2 pencil...


ok pencils down.

If you said: Multiply it back in give yourself a gold star!

right we need to multiply the shadow back into the now shadowless diffuse element for this light. And it needs to come after we have divided it out (wouldn't do much good beforehand). So we want yet another branch in our composite. This time right click on the divide branch and choose Insert Post Effect->Hash Inc.->Multiply. Now drag and drop the shadow element onto this new branch to complete it's input and... hey we are right back where we started! with one HUGE difference: Now we can adjust the darkness (and other aspects) of the shadow without effecting the diffuse elements of this layer.



So this is what our Composite looks like now:



again I have renamed the multiply branch.

Ok so now we just need to adjust the darkness of the shadow.... I need to sleep for now but i'll wake up soon!

-David

I feel dizzy. Does that white line below the sphere have any significance?

Okay, I have had to Change My PANTS!

TWICE!

This feature is going to revolutionize A:M. You have only scratched the surface and already the possibilities are spinning in my head.

During the early discussions... I don't think I fully grasped its potential. I had some doubts to be honest...

Those doubts are in the laundry basket with my pants.

Vernon "!" Zehr

terrific stuff - I have another probably out-of-order question - at some point ...please, could you post a screen shot or list of all the options available for the layer compositing methods (ie the "right click list" that includes multiply, divide for Hash post effects). I'm curious what else is available, if anything?. Very interesting. Very well done instructions. Thanks.

Better still. Maybe for this and cloth we'll get a tech talk video during it's "big push"?

ahh nothing like a nap and some coffee!

Adjusting shadow darkness

RIght so now we want to go into the shadow buffer and change the darkness of that shadow so that it is more pronounced on the ground. This is actually very easy (i could have done it before I went to bed if I didn't have to type up a description here ).

We're familiar with the concept of adding a branch to our composite tree, so for our shadow we just need to find the right branch to add. So what are our options here?

• AutoLevels
• Clamp
• Divide
• Exposure
• HSE
• MIx
• Multiply
• Over
• Tint

each of these, of course does something different and we are already familiar with at least three of them (Mix, Multiply and Divide) We just have to look at the remainder and determine which will do the best job (note that if you were thinking you could multiply two copies of the shadow layer together to produce a darker shadow you were thinking correctly but there is more than one way to skin a cat and I want just a hair more control over things than that.) A lot of learning how to make Composite sing is in figuring out each Post effect and what it does. In our case I think Exposure will do the job.

Why?

Well lets look at what Exposure Does. It has the following Properties: Active, Exposure, Brightness, Contrast.

Hey... brightness? Contrast? that sounds just like what the doctor ordered. Active, of course, just shuts the effect off and lets the elements past the branch just filter through. Exposure... what's that? It has to do with the HDR aspects of our image, it allows us to push highlights past white and shadows under black. If we had an underlit scene, an exposure adjustment would allow us to bring the render back into the range where we can see what was going on. In our case we will start with Brightness and Contrast.

we want the shadow to be darker so drop the brightness from 50% (normal/default) to 30%

we get this:



The readily apparent issue is that this will cast a pallor on our entire scene rather than just darkening the shadow area (which it has as well.) what we need to do is to push the light grey to white and allow the shadows to stay black. This is exactly what contrast does. I push the contrast up to 90% and I get this:



Hey that looks a lot darker to me!




so what's left? just changing the color of the lights in the shot, that's all. everyone stay tuned more to come!

-David

David,

The steps for getting at the shadow part of the image so it can be adjusted seems a few steps more involved than your first example. Is this also something that only needs to be done once for a whole series of TGA pictures?

Maybe a better question is to ask, is there anything that could be done to a composite that doesn't propogate through all images in a sequence?

thanks!

-Jim

Love it! Such control.

How does this compare to other compositors? Are there similar terms to Mix/Multiply/Devide?

I just want to share what I envisioned when I heard compositing was coming to AM. Maybe in the future it will be possible. Or maybe it wouldn't give as much control/wouldn't work:

You render out your animation to exr format.
Then re-import the frames into the same AM project.
Now the controls of the light would dynamically effect the images on screen.
For example, to lower the brightness, you would lower the the light's intensity as you would normally do it in 3D.
In effect, the exr images replace your 3D scene but allow the dynamic control of the final render.

David,

Wow!

I'm just blown away by how much control this is going to give us. I eagerly await your next examples!

-Jim

Bob and his need for spheres kept me away yesterday, but today I will make up for that by showing you all the secret to Dividing and conquering light color!

Light Color and Adjustment

We last we left we had just finished adjusting the shadow darkness for the shadows and just had the color of the light left to change.

In order to truly adjust the color of a light in a scene you need to adjust the actual light and leave the underlying surface properties of the geometry alone... Which, if you know anything about how light works in the real world can be pretty confusing. In the real world the color of a surface is actually determined by light, the surface will bounce back certain wavelengths of light and absorb others (speaking in a very simplified way) the light leaving that surface is then picked up by our eyes and bounced around our rods and cones to determine how bright that light is and what color we see the surface as being. In 3D we cheat this quite a bit (gasp! not a cheat in 3d! ) each surface has it's own color regardless of what's going on in the scene and then we shoot light rays at that surface and the color of the light alters the color of the surface by... you guessed it: Multiplication! So when we want to change the color of a light we need to be able to pull just the information that the light is contributing to a surface regardless of the color that surface might have.

So how do we separate out the light data? The same way we separated out the shadow data. First we look at our available buffers for what we need:



The buffer that we are looking at here is the Diffuse buffer, not the diffuse buffer for any of our three lights mind you, those contain the diffuse color of a surface and the effect of lighting, no we need the Diffuse buffer that contains only the diffuse surface colors for the objects in the scene which is the one right there at the top: start simple 0 Diffuse.



This is what we call a flat shaded version of the scene. it is what the scene looks like without any lights added in at all, it's not much to look at but it does give us something to divide our diffuse light elements with to get to just the light data.

So we go into our composite and add a branch to the Light 3 diffuse with shadow element, we want to add a Divide branch just as we did to remove the shadow but this time we drag the global diffuse buffer to the branch to complete it. Again, just like we did for the shadow we need to multiply the diffuse elements of the scene back into the composite after we have doe the Divide. I like to do this right now while I'm thinking of it, and it's pretty much IMNSHO a good idea to multiply when you divide and visa versa, it keeps you from forgetting about it later.

Rename your new branches and you might have a composite that looks like this:


if you toggle between the multiply and divide branches you probably won't detect a huge difference, but that's because our surface colors aren't really set, they are all default white, if you had any variation off that default white in the scene you'd see pretty vast differences.

Ok now that we have the light separated we can see that it is indeed yellow in hue, and Bob hates yellow. we need to make this light a cool color. There are a couple ways that we can go about this, we can either use the tint post effect (see that list I made a few posts back ) or we can use the HSE post effect... decisions decisions.... well I tell you what we have another light down there we can change the color on, lets do tint here and HSE there.

Tint is a post effect that takes the range of color in an image and tosses it out the window, what it leaves is the brightness of each pixel, which we can then map to two colors, one will suplant the dark tones in the image and the other will alter the light tones. In the print world (and in photoshop) this is called a duo tone. It is used to give images that old timey sepia tone look to them or to simply take a color image and make it black and white.

We go ahead and add the Tint branch to our Composite tree. Right-click on the Divide Branch we just added in and add a new Tint post effect. The new brach will default to making your image black and white. There are a number of other presets available to us: Colbalt will make the darks of the image black and the lights a rich deep blue, Sepia will turn the image into a sepia tone, where the darks are a dark rich burnt umber color and the lights are a light brown, X-Ray will make the dark tones white and the light tones black. I think i'm going to use a custom set of colors:



The darks are a very dark blue grey and the lights are a brighter version of the same color.


much better no more of that yellow in there and we have a nice cool light

The last little bugbear is the yellow/orange fill light down there on light 1 diffuse. We tackle this just like we did light 3. first we divide out and multiply back in the Diffuse for the entire scene, then we add a branch in to change the color of the separated out light data and as I promised, this time we are going to use the HSE effect to do it!

HSE stands for Hue Saturation and Exposure. The simple way to think of this is: Hue determines the color, Saturation determines how vivid or washed out the colors are and Exposure determines how bright or dark the colors are. In the way of computers, seemingly simple concepts are actually pretty darn complex if you keep digging, but this base understanding will be enough for us now.

So at the point where we have the light data broken out of our composite (i.e. on the divide branch) right click to add a new HSE effect. This effect will now allow us to shift the hue from the yellow range that it is in now to a more Bob-friendly color range. To understand what the hue shift will do think of a color wheel and think of our color as being a fixed point in space not really a color but an coordinate that points to a color on the wheel. When we adjust the HUE we rotate the wheel without changing the location of the point of our color so that now it points to a different color entirely. The upshot is that we can change an entire color range without effecting the variations across a surface (as with tint.) Hue takes a value from -180 (rotate the wheel counter clock wise 180 degrees) to 180 (rotate the wheel clockwise 180 degrees.) With a little experimentation you will be able to find a color that you like, for this case I give the wheel a value of 50 (shifting the color wheel 50 degrees clock wise moves the green range of color under our point)


so at this point we have a full composite that looks like this:


which is a whole world of difference from where we started:



The actual time to build and adjust this composite was actually less than 20 minutes (which if you are talking about an animation with hundreds of frames that you want to adjust is a HUGE time savings on re-rendering.) Of course typing it up took a lot longer than that

and just for your reference here is my final Composite:


If this is what we want, it's a simple matter to commit to this: right click the composite in the PWS and choose Save As Animation... and your composited image will be saved to disk in the format of your choice. Now if we send this to bob and he decides he wants the fill purple instead of green... hey no problem we just change the HUE (-150 would probably do the trick) and save as animation... total re-rendering time? 0 seconds. let that sink in... you just changed something that effects the look of every frame of your animation and the total time to render that change to file is 0 seconds. Suddenly the render is a re-usable asset that can be used again and again changed slightly and output with no real hit to your time.... we'll wait for vern to change his pants before we sum up this little exploration.

The thing to remember here is that Composites aren't just for fixing problems, heck i'd say that's the least of the cool factor. Mostly they are about giving you the freedom to try different things all at full resolution all over the full length of an animation all without having to render a scene dozens of times.

This simple introduction was just intended to give everyone a basic feel for how the feature works and encourage you guys to jump in and start playing with your own renders. Soon I will be starting up a thread that goes into a real scene from an actual animation and go over some of the more advanced ways in which we can use Composite and our EXR layers. Lots and lots of good stuff to come!

-David

I'm actually anxiously awaiting your next installment, David... I'm hooked on compositing.

Actually... fixing "problems" was the last thing I thought of....

The first thing I thought of... will sound odd....

Imagine.... blending.... layers from "different" animations? I was thinking you could produce some wicked bizarre effects using the rendered frames from several different animations.

Or you could go in and create interesting effects by changing the numbering and/or the order of the rendered frames.

Like... some cool... "bullet motion" thing... create several "copies" of a rendered sequence... off-set the frame numbering and use these new features to blend them over time.

..or...

Take a rendered sequence from a completely different animation... and apply it to another in some sort of "ghosted" overlay... thingy... that fades and changes... the mind boggles.

...or...

Create some interesting images in a paint program to overlay. You could create all kinds of things... like... fading... flashing... computer screen overlays... like in Terminator... or screen noise... create a few "pseudo" frames of noise in photoshop... animate the blend effects over time... MY GOD! It's almost like having After Effects built in to A:M! AND THE PRICE DIDN'T GO UP!

....DAMN!....

...I have to change my pants again...

One more than I quit....

Could you do a post blur when you render? Imagine having two versions of a sequence.... one is blurred... you could "replace" certain layers with the blurred layers... giving it a whole new "feel".


Vernon "!" Zehr

This is unbelievable stuff. I am a fairly clueless newbie and am sure to screw this up more often than I get it right, but it sure seems to make sense. This has got me salivating to upgrade.

Just so that I can say that I fully understand it I like to parrot stuff back. If I understand correctly one could complete work on an animation, set up the basic lighting and render it out. He could then adjust lighting by working with the composite and fix it without rerendering. Is this correct? Render once fix multiple times? Doesn't this potentially move rendering to a very different place in the work pipeline?

I've got to reread this thread about five more times to continue to let this sink in.

That's cool!

Scott

David, I was just reading your book and you covered compositing in it too....the hard way. What a difference now!

Some questions:

You covered depth of field in the book. Can we do that the easy way yet?

Practically speaking, do exrs take up alot of resources? Realistically, how many frames could you "composite" without overloading your system. (On a good spec machine).

Finally, I've heard that light is just the start of the compositing abilities of AM. What other things could be useful in compositing. DoF I know. Anything else?

Oh Baby!

(Shaun rifles through cabinet for another pair of pants)

Render layers looks more versatile than I dreamed!

Great tutes David. Really valuable stuff.

Great feature Hash Inc. Really valuable stuff.

I love it. I love it. I love it. I love it, etc, etc, etc.....

EDIT: oh, forgot to say - I really, really love it.

Gread Stuff David, you are absolutely the guy for this job.

I just rendered out my animation in v12 for the soft shadows but I am now kicking myself for not rendering them using EXR format I could be having so much fun tweaking away right now and instead I just have a bunch of alphas. What is more, there is a sequence at the beginning where I fade different things in and out that render layers would have been perfect for.

If I had seen this tutorial before hand I would be in a different position now.

Oh well, I will know for next time.

Another question which wouldn't be necessary if I had v12:

Do EXRs take longer to render to than say TGAs?