Blender Git Loki

Cycles: Experiment with shadow catcher

**DISCLAIMER**: This is more a code dump of a local branch, not somewhat really finished or so. Underlying math is the subject for rework since it's not quite physically based at all.

Publishing to start collaboration with other Cycles developers who are looking into solving this puzzle.

=== What do we consider a shadow catcher? ===

That's a good question actually, and there's no single formulation of what it exactly is and mathematically it's a bit malformed in the constraints we're working on. Ideally shadow catcher is a difference between image rendered without artificial objects and with them. Such approach gives best ever shadows, but takes 2x more time to render. So for good usability we need to get some assumptions, make system a bit more biased but give artists an useful tool.

Shadow catcher is mainly used by VFX artists to inject artificial objects into real footage. At least that definition we'll stick to
in Blender. Hence here's what shadow catcher should be capable of doing:

- Receive shadows from other objects: be totally transparent when there's no shadows cast on it, be more opaque in shaded areas.

- Ignore self-shadowing and shading. Shadows caused by occlusion with itself already exists in the footage. Same applies to the
shading -- all shading caused by material itself are also in the footage already.

- Interact with other objects in the scene. This sounds a bit tricky but makes sense actually. Consider situation when one needs to put sharp glossy object into the footage: you'll want objects from a real scene to be reflected in the artificial object. And often you'll want the object on which shadow is to be cast to be reflected in such situations. Surely you can escape with copying object and playing with ray visibility, but that's complicated scene setup instead of making it simpler.

- Be affected with indirect light. Cycles is the GI render engine after all!

=== How to use the shadow catcher? ===

1. Create an object on which you want to receive shadow.
2. Create some basic material setup which is close to a real object.
3. Enable "Shadow Catcher" in Object buttons -> Cycles Settings.
4. Be happy! (hopefully, once we've debugged all the code)

=== What this patch actually contains? ===

It contains all the bits which tries to implement definition of shadow catcher above. It is trying to implement it all in a way so we don't need to make big changes in the ray integration loop, hence it has some tricky magic to deduct what was the received shadow from the light passes and will fail in certain situations, mainly when there is no direct lighting of the object at all. It is totally tweakable to become more artists friendly, i just didn't have enough time to try all the ideas and used whatever latest semi-working formula was.

Major changes are in fact made around shadow_blocked() to exclude shading from self. This part is based on an older patch which tried to expose it to an user. That exposing settings are somewhat malformed and shouldn't really be used. In fact, we should remove those settings from the interface.

=== Some pictures? ===

Sure, here's one from a hackish patch:

{F282085}

(This is a glossy monkey on a checker board floor, floor is makred as a catcher, Here's .blend file {F282088})

Reviewers: lukasstockner97, juicyfruit, brecht

Subscribers: jensverwiebe, Nikos_prinio, brecht, lukasstockner97, borisdonalds, aliasguru, YAFU, forest-house, uli_k, aditiapratama, hype, davidandrade, printerkiller, jta, Davd, johnroper100, poor, lowercase, juang3d, GiantCowFIlms, iklsr, gandalf3, sasa42, saphires, duarteframos, madog, Lapineige, railla, zuggamasta, plasmasolutions, jesterking

Differential Revision: https://developer.blender.org/D1788