This is an excerpt from our internal documentation describing an issue with drawing in NSView
s on macOS Big Sur.
In macOS Big Sur (probably starting with β9), Apple changed the default contents format for backing layers of NSView
s. Instead of an explicit CALayerContentsFormat.RGBA8Uint
value, an „Automatic“ value is now used. Even though it also resolves into „RGBA8“ in our testing, it has some serious implications as it breaks assumptions our code relies on.
I first stumbled upon this issue in this tweet by Frank. It links to a thread on Apple Forums by Mark that contains valuable information as well as ideas for workarounds. The changed behavior was also confirmed by Marcin in this tweet.
The issue affects Diagrams in the following ways:
- As a canvas-based app that renders contents using the old-fashioned drawing approach via
NSView.draw(_:)
, we heavily rely on drawing optimization. Only the invalidated rectangles inCanvasView
are considered for inspection, generation of render operations, and rendering. Since the usage of automatic backing stores prevents us from getting information about rendered rectangles (NSView.getRectsBeingDrawn(_:count:)
), we’re out of luck for any optimizations. With the new behavior, we only receive the rectangle for the whole tile (that equals to the dirty rectangle). - A much bigger issue is the difference in handling the clipping areas. It seems like that drawing outside of the invalidated rectangles is forbidden, which leads to missing parts of shadows in cases where the render operation defining the shadow (shape) doesn’t intersect the invalidated rectangles. We run into a similar issue with clipped shadows that reached into neighbored tiles in the past, but the current issue looks slightly different as it’s not tied to tiles.
- We assumed a bitmap
CGContext
with the typekCGContextTypeBitmap
would be provided toNSView.draw(_:)
, which is no longer the case. You now get aCGContext
with the typekCGContextTypeCoreAnimationAutomatic
. The recently introduced custom tile pattern rendering in CanvasKit leveraged the possibility of getting the height of the context. A non-bitmap context throws an exception when callingCGContext.height
.
The issue is only present when linked against the new SDK. The latest version of Diagrams in the MAS (1.0.4) is not affected.
In order to understand the issue, I went quite deep with reverse engineering and found out a few interesting bits.
It seems like NSWindow
s hold a flag telling all CALayer
s which contentsFormat
to use when they’re created. This is the case for layers backing NSView
s as well. On Catalina, it seems that „RGBA8“ was the default value, but on Big Sur it changed to „Automatic“. To be more precise, this value is configured by consulting the NSViewUsesAutomaticLayerBackingStores
user defaults flag that has been present already on Catalina, but its default value was false
, whereas it’s true
on Big Sur.
This is the relevant part of the call stack that accesses the user defaults flag:
UserDefaults.object(forKey:) // arg="NSViewUsesAutomaticLayerBackingStores"
_NSGetBoolAppConfig()
_NSSetWindowDepth()
NSWindow.setDepthLimit(_:)
NSWindow._commonAwake()
There is also a new property NSCGSWindow.usesAutomaticContentsFormat
that wasn’t present on Catalina. It seems to store the value, but from my testing, it isn’t consulted after being set.
None of these details are relevant, though. The crucial thing is that overriding the value of the flag indeed reverts the behavior for the whole application and everything gets back to normal. The downside is that it affects layers in all views within all application windows, which is probably not what we want. Furthermore, telling from the experience with a similar approach we used for circumventing new scrolling behavior in NSScrollView
some time ago, it’s very likely that this user defaults flag will be removed in the future.
I prefer to keep the change as local as possible. The workaround presented in the forum thread does just that, as it changes the contentsFormat
of an individual layer in NSView.viewWillDraw()
. According to my tests, this works fine with layer-backed views that have only a flat layer, e.g. in Marcin’s simplified example.
_NSViewBackingLayer (contents=_NSBackingLayerContents)
Although this approach solves the issue with our preview view (getting the height of the CGContext
), it is not applicable to the canvas. The CanvasView
uses the NSScrollView
architecture and typically has a large size. AppKit splits the backing layer into multiple tile layers that come and go.
_NSViewBackingLayer (contents=_NSBackingLayerContents)
└ _NSTiledLayer (contents=_NSTiledLayerContents)
├ NSTileLayer (contents=CABackingStore)
├ NSTileLayer (contents=CABackingStore)
├ NSTileLayer (contents=CABackingStore)
└ ...
Unfortunately, setting the contentsFormat
on the parent layer doesn’t affect sublayers. I tried to set it recursively and even reached to the superlayers (clip view, scroll view), but never got the right result. I haven’t managed to reliably change the format and NSView.getRectsBeingDrawn(_:count:)
always reported the size of the whole tile. It felt like there is too much going on underneath that is not meant to be touched. So no local solution found yet.
This response from Mark is also very worrying. According to him and the message from Apple he got, this is now the preferred way, and developers who need drawing optimizations should „roll their own system“. I mean, we now have Apple Silicon, so there’s no need for optimizations anymore, right?
As the workaround for setting the contentsFormat
doesn’t cover all of our cases, I decided to go with overriding the value for the user defaults flag NSViewUsesAutomaticLayerBackingStores
and set it to false
. This is an immediate but temporary fix, which will be revisited later.
To ensure that CanvasKitMac is always used in the correct way, I added a precondition that checks that the passed-in CGContext
is a bitmap context. The nice side effect is that we can safely remove the recently introduced workaround for rendering tile patterns on Big Sur.
If we find a way to apply the local workaround to CanvasView
, we should then remove the global one and add this logic to CanvasKitMac.ComponentView.viewWillDraw()
that covers both the preview view as well as Xcode previews.
🆕 Follow-up from WWDC 2021
[The AppKit team is] aware of this issue and they confirmed that it’s not intended for
NSView.getRectsBeingDraw(_:count:)
to return the whole bounds or tiles in the case ofNSScrollView
. It’s a bug they didn’t manage to fix in macOS Big Sur, but they’ll do their best to fix it in a later release of macOS Monterey.Furthermore, they confirmed that our current solution of setting a user defaults flag is appropriate. First, it’s safe as it doesn’t have negative implications on other parts of the app as it’s an app-wide override (other than a higher memory footprint which is no big deal). Should they change its internal behavior, the flag would become a noop which also shouldn’t do any harm. Second, it’ll be the way to go for macOS Big Sur anyway, as the fix most likely won’t be deployed back to Big Sur.
This means that there’s no need for rolling out our own system for tracking the rectangles. Ufff… 😅
With that being said, we haven’t yet run tests on macOS Big Sur 11.6 nor macOS Monterey beta. If you happen to know how it behaves on these system, please leave a comment.