[Gc] Safe to call GC_finalize() BEFORE world starts?

[Niklas Therning]

Hi,

In RoboVM (Java on iOS) we use GC_REGISTER_FINALIZER_NO_ORDER() to
implement SoftReferences (i.e. weak references). When an instance of a Java
object (the referent) is wrapped in a SoftReference we use
GC_REGISTER_FINALIZER_NO_ORDER() to get a callback when the referent is no
longer strongly reachable and then we clear all SoftReferences referring to
it.

We're now facing a bug which seems to be related to the way finalizable
objects are marked in GC_finalize(). GC_finalize() happens after the world
is started and the referent is still weakly reachable until the callback
has been called to clear the SoftReferences. So there is a small window
where other threads can mutate the referent in such a way that some other
instance it refers to never gets marked. E.g. by nulling out a field in the
referent before GC_finalize() runs and then putting back the same value
after GC_finalize() has run. This can lead to crashes later on when that
other instance is reclaimed by the collector and reused in another
allocation.

I have now patched GC_stopped_mark() in alloc.c to call GC_finalize() just
before START_WORLD() is called (not if the stop function stops the marking)
and it seems to fix the bug we've been seeing. But the question is whether
this is safe? The finalizer callbacks are still called after the world is
started.

Regards,
Niklas