When Adobe introduced the Rotobrush into CS5, I was pleasantly suprised by how much time it shaved off the usually tedious and annoying task of rotoscoping. But the part that I found to be the most helpful was the matte refining tools that are added to the rotoscoped footage. I thought it was pretty freaking cool that it would add motion blur back into the edges of a shot without bluring the core of the material.
So, to the point at hand. Recently we shot a bunch of greenscreen material for the local Addy Awards show. It was a limited budget so we shot it on our HVX on a pretty small screen. It was always our goal for it to have a Lo-Fi cheese factor, so we accepted the fact that we might not have time to really treat the edges of the key and it would probably look like poorly keyed stuff on really cool painterly backgrounds. When I sat down and started keying, however, I stumbled onto an effect that I hadn't noticed quite yet. Turns out Adobe brilliantly took the Refine Matte tools from the rotobrush and added it as a separate effect under the Matte menu.
Now I'm never one to say that anything in motion graphics or VFX is a push button solution...or that the same process will work for every situation...but this thing has just saved me a ton of grief with just a little tweaking. My process was pretty straight foward: garbage matte, core matte, and edge matte. Then I applied Refine Matte to the edge matte only. With a couple little tweaks I was getting results that looked a ton better than anything I'd ever gotten from a straight key on HVX footage. The reason it looks so good is because it decontaminates the edges by extending the color information out passed the edge of the alpha, then smoothes out any chatter in the alpha. Of course you can't go too extreme with it, otherwise your matte looks TOO smooth.
After that, I just did a really soft key for the hair detail and tucked that under the core matte.
I'll try to get some examples up soon, but if you've got some suspect green screen stuff and your edges are giving you fits...give Refine Matte a shot...could save you a headache.