Blender Git Loki

USD: Introducing USD support

This commit introduces an exporter to Pixar's Universal Scene
Description (USD) format.

- The USD libraries are expected to be in /opt/usd/, not yet built by
install_deps.sh. I'll work on that while this patch is under review.
- Only experimental support for instancing; by default all duplicated
objects are made real in the USD file. This is fine for exporting a
linked-in posed character, not so much for thousands of pebbles etc.
- This patch contains LazyDodo's fixes for building on Windows in D5359.

== Meshes ==

USD seems to support neither per-material nor per-face-group
double-sidedness, so we just use the flag from the first non-empty
material slot. If there is no material we default to double-sidedness.

Each UV map is stored on the mesh in a separate primvar. Materials can
refer to these UV maps, but this is not yet exported by Blender. The
primvar name is the same as the UV Map name. This is to allow the
standard name "st" for texture coordinates by naming the UV Map as such,
without having to guess which UV Map is the "standard" one.

Face-varying mesh normals are written to USD. When the mesh has custom
loop normals those are written. Otherwise the poly flag `ME_SMOOTH` is
inspected to determine the normals.

The UV maps and mesh normals take up a significant amount of space, so
exporting them is optional. They're still enabled by default, though.
For comparison: a shot of Spring (03_035_A) is 1.2 GiB when exported
with UVs and normals, and 262 MiB without. We probably have room for
optimisation of written UVs and normals.

The mesh subdivision scheme isn't using the default value 'Catmull
Clark', but uses 'None', indicating we're exporting a polygonal mesh.
This is necessary for USD to understand our normals; otherwise the mesh
is always rendered smooth. In the future we may want to expose this
choice of subdivision scheme to the user, or auto-detect it when we
actually support exporting pre-subdivision meshes.

A possible optimisation could be to inspect whether all polygons are
smooth or flat, and mark the USD mesh as such. This can be added when
needed.

== Animation ==

Mesh and transform animation are now written when passing
`animation=True` to the export operator. There is no inspection of
whether an object is actually animated or not; USD can handle
deduplication of static values for us.

The administration of which timecode to use for the export is left to
the file-format-specific concrete subclasses of
`AbstractHierarchyIterator`; the abstract iterator itself doesn't know
anything about the passage of time. This will allow subclasses for the
frame-based USD format and time-based Alembic format.

== support for simple preview materials ==

Very simple versions of the materials are now exported, using only the
viewport diffuse RGB, metallic, and roughness.

When there are multiple materials, the mesh faces are stored as geometry
subset and each material is assigned to the appropriate subset. If there
is only one material this is skipped.

The first material if any) is always applied to the mesh itself
(regardless of the existence of geometry subsets), because the Hydra
viewport doesn't support materials on subsets. See
https://github.com/PixarAnimationStudios/USD/issues/542 for more info.

Note that the geometry subsets are not yet time-sampled, so it may break
when an animated mesh changes topology.

== Hair ==

Only the parent strands are exported, and only with a constant colour.
No UV coordinates, no information about the normals.

== Camera ==

Only perspective cameras are supported for now.

== Particles ==

Particles are only written when they are alive, which means that they
are always visible (there is currently no code that deals with marking
them as invisible outside their lifespan).

Particle-system-instanced objects are exported by suffixing the object
name with the particle's persistent ID, giving each particle XForm a
unique name.

== Instancing/referencing ==

This exporter has experimental support for instancing/referencing.

Dupli-object meshes are now written to USD as references to the original
mesh. This is still very limited in correctness, as there are issues
referencing to materials from a referenced mesh.

I am still committing this, as it gives us a place to start when
continuing the quest for proper instancing in USD.

== Lights ==

USD does not directly support spot lights, so those aren't exported yet.
It's possible to add this in the future via the UsdLuxShapingAPI. The
units used for the light intensity are also still a bit of a mystery.

== Fluid vertex velocities ==

Currently only fluid simulations (not meshes in general) have explicit
vertex velocities. This is the most important case for exporting
velocities, though, as the baked mesh changes topology all the time, and
thus computing the velocities at import time in a post-processing step
is hard.

Tests: Blendfile-loading test class

This new test class minimally sets up Blender so that it can load blend
files and construct a depsgraph without crashing.

Note that it hasn't been tested on very complex blend files, so it may
still crash when the loaded blend file references/requires uninitialised
data structures.

The test will certainly crash with Blend files created with Blender
older than 2.80, as the versioning code requires space types to be
registered. This is normally done by initialising the window manager,
which is not done in this test. The WM requires Python to run, which in
turn requires that Blender finds the release directory in the same
directory that contains the running executable, which is not the case
for GTest tests (they are written to `bin/tests/executablename`.

Reviewed By: sergey, mont29

Differential Revision: https://developer.blender.org/D6246

Fix T71558: Hair particles: Brush effect not occluded by emitter geometry

Fix crash if OptiX context creation fails in Cycles

When encountering an error during context creation, the "OptiXDevice" constructor aborts early.
This means the "cuda_stream" vector is never resized and the destructor iterated over non-existent data.

GPencil: Remove Emboss por Sliders

GPencil: Add N Panel in Dopesheet

GPencil: Fix icon for mask layer in Dopesheet

Merge branch 'master' into undo-experiments

Fix (unreported) broken python resgistrable classes checks logic.

Logic for registering and checking properties of registrable classes was
broken, allowing to ignore some errors.

Recent fix rBeb798de101a `broke` the result of the
pyapi_idprop_datablock test, because previously that test would fail
(i.e. suceed, as it is an 'expected to break test') for a reason it was
not designed to check.

This is the problem with that kind of tests - you cannot really check
that they are failing on the expected reason(s)...

Console: remove shortcuts from console splash

Menu items can be used to find shortcuts instead of keeping
keymap items here.

EEVEE: RenderLayers

Small change do not draw an unsupported renderpass in the viewport

USD: renamed `should_export_object` to `should_visit_object`

GPencil: Add layer operators to Dopesheet Header

USD: Revamped building & bundling process

- USD is now built as monolithic library, instead of 25 smaller
libraries. We were linking all of them as 'whole archive' anyway, so
this doesn't affect the final file size. It does, however, make life
easier with respect to linking order, and handling upstream changes.
- The JSON files required by USD are installed into bin/usd; they are
required on every platform.
- USD is patched so that it finds the aforementioned JSON files in a path
relative to the running executable. When left unpatched, the only way
to influence the search path for those files is to set an environment
variable BEFORE Blender starts (USD uses static initialisers, so by
the time the USD exporter runs, it has already unsuccessfully tried
loading the JSON files, and won't re-investigate that environment
variable ever again).
- USD is patched to have a `PXR_BUILD_USD_TOOLS` CMake option to disable
building the tools in its `bin` directory. This is sent as a pull
request at https://github.com/PixarAnimationStudios/USD/pull/1048

Fix assert in Cycles memory statistics when using OptiX on multiple GPUs

The acceleration structure built by OptiX may be different between GPUs, so cannot assume the memory size is the same for all.
This fixes that by moving the memory management for all OptiX acceleration structures into the responsibility of each device (was already the case for BLAS previously, now for TLAS too).

USD: Avoid crash when USD JSON files cannot be found

USD ships with a set of JSON files that define which formats can be written
and which plugins are required. This is used even when statically building
and only using the default USDA and USDC formats.

Fully WIP code.

Don't have proper solution yet for issue described in comment in
readfile.h, still investigating...

Overlay Engine: Fix issue with facing overlay

Render using the default view instead of the jittered view.

Cleanup: Missing bracket.

Buildbot: Migrate package archive format for Linux from tar.bz2 to tar.xz

xz compresses 25% better than bz2, reducing download times and server load.
The numbers:

blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz: 96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)

blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz: 133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)

xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).

Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.

Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.

Differential Revision: https://developer.blender.org/D6138