Blender Git Loki

undoexp: add a way to prevent re-using data from old bmain for memfile undo.

There are cases where we need a full 'clean' re-read of stored memfile
data. This include some (rather uncommon, but possible) ID rename
situations, and also likely some switching between memfileundo and
edit-data undo systems.

Merge branch 'undo-experiments' into undo-experiments-idnames

undoexp: Fix crash on NULL depsgraph in storage/restoring code.

There are some cases where a depsgraph has 'no time' to be built (like
multi-steps undo's).

Plus I guess in multi-viewlayers/scenes cases...

Change message in case of pointer collision to always print.

It is very difficult to test that code, as pointer collisions are
expected to be rather rare... So for now systematically print when we
detect one, could be at least a useful hint in bug report.

And some more comments.

Merge branch 'undo-experiments' into undo-experiments-idnames

Merge branch 'master' into undo-experiments

undoexp: much better handling of idname-based re-using old IDs.

This commit adds obvious usage of a proper mapping to find local IDs
in old Main based on their names.

It also adds handling of potential pointer collisions between old Main
realm and read .blend memfile realm. That last part is theoritical
currently though, testing it actually works won't be easy (pointer
collisions will happen, but they are not very likely to happen...).

We are still missing some way to prevent all that new code to be ran
over an undo step that contains ID renaming (ID creation/deletion should
be safe, but idname-based search cannot work whith renaming).

Merge branch 'undo-experiments' into undo-experiments-idnames

Merge branch 'master' into undo-experiments

undoexp: Initial, *VERY* basic code using ID names.

This comes from the fact that we are now handling IDs from potentially
many different 'memory realms' (at the very least, two of them, those
read from memfile [new IDs], and those from previous Main [old IDs]).

The main consequence is that using pointers (aka memory addresses) as
'uids' is not working anymore: we'd need to keep some sort of 'history'
of all pointers a given data-block has had to get it working, and that
would likely lead to 'pointer collisions' [1] at some point or another.

So instead we need to use the only kind of uid we have for datablocks:
their names.

That patch is very basic, but it is enough to check undo/redo on basic
object and pose-bone-of-rigged-mesh dummies, not get it crashing, and
demonstrate huges speed-up in some artificial 'worst test case' scenario.

There are still several things to do from there:
* IDname handling:
* Obviously, use a ghash instead of dummy linear list search! Most
likely use BKE_library_idmap code to generate it.
* Check and handle pointers collisions. [1]
* Testing, testing and moar testing! These changes are highly dangerous
and can destroy a blend file completely, so we'll need to be 100% sure
they are working perfectly before puting them in master!

[1] Pointer collision: the issue here is with 'old' memory addresses
stored/used as uid in .blend file. When one read a .blend file as a
whole this is not an issue, since all old data-blocks where in a single
'memory realm' (the old Main valid at time of file writing), and all new
data-blocks are also in a single realm (the newly read file). Since we
only remap each pointer once, this is fine.

But when we start mixing those 'memory realms' by re-using IDs from old
Main in this undo project, we can end up with the same 'uuid' old
pointer value having to be remapped to two different new addresses.

undoexp: fix crashes with even trivial relationships (like parent ones).

Even if we re-used an old ID, it may have some pointers to other IDs
remapped to newly read data, e.g. the `parent` of an object...

So CoW/evaluated data in re-used despgraph needs to be flushed in those
cases.

undoexp: Add new 'UNDO_OLD_ID_REUSED' ID tag.

We need to be able to distinguish between reused IDs and newly read
ones, and 'NEW' tag is not enough here, as it gets cleared in read code.

This will be used in next commit to inform depsgraph that some re-used
old IDs need at least a COW refresh, as they are using ID pointers to
newly read data.

Undoexp: do not tag re-used IDs with LIB_TAG_NEW.

Merge branch 'master' into undo-experiments

undoexp: do not clear icon_id of re-used IDs.

Those icon_id's remain valid, and allocated in the preview system,
clearing them here will lead to asserts...

Merge branch 'master' into undo-experiments

Merge branch 'master' into undo-experiments

Fix stupid misplacement of depsgraphs extraction/restoration.

Merge branch 'master' into undo-experiments

Undo: Re-use existing IDs from old bmain: do not search by name.

This is actually a fairly bad idea, since IDs can be renamed...

Only search by pointer value now, this *should* work in all expected
cases once undo/redo fully reuses unchanged data-blocks.