Skip to content

Instantly share code, notes, and snippets.

@darwin
Last active April 9, 2024 22:30
Show Gist options
  • Save darwin/3c92ac089cf99beb54f1108b2e8b4b9f to your computer and use it in GitHub Desktop.
Save darwin/3c92ac089cf99beb54f1108b2e8b4b9f to your computer and use it in GitHub Desktop.
APFS Container cloning/replicating under Catalina (with a bootable system)

Today I wanted to move existing APFS-resident macOS Catalina installation to a new disk. I upgraded my late 2014 Mac Mini with a shiny new 1TB SSD. This took way too many hours of my life I will never get back. Hope this saves some time to you.

Good news:

  1. it is possible to create a DMG image from existing APFS container with macOS Catalina installation including metadata needed for complete restore (the DMG contains OS, OS Data, Preboot, Recovery and VM volumes)
  2. it is possible to restore this DMG image into empty APFS container and get a bootable copy of the original system

This information is relevant for Catalina (I'm currently running macOS 10.15.1).

There are major tricks how to make it work, the second one quite recent and third one is its consequence:

Trick 1

Source of restore operation must be a synthetised disk mounted from the DMG. Not the DMG file itself!

Trick 2

"APFS Volume Groups", a new feature introduced in Catalina does not play well with current Disk Utility.app, so you have to drop to command-line and force using legacy method of restoring the container.

Trick 3

Source DMG must not contain any APFS Volume with APFS snapshots.


Let's create the source DMG

This can be done comfortably from Disk Utility.app. You must be booted into some other system than your main system. In the left sidebar of Disk Utility.app switch to "Show All Devices". And then right-click the whole APFS container containing your system. It should have a name like "container diskN" and none of its volumes should be currently mounted. In the context menu you should see an available option to "Image from container diskN".

Wait, hold your horses. Before making image you might consider deleting all existing snapshots in all APFS Volumes in the container. It will save you headaches later. Better finish reading the article first.

Can I boot into RecoveryOS to do this DMG image?

No. RecoveryOS resides in one of APFS Volumes in the container you want to image so this is not possible, I think (not tested).

Oh, what to do if I don't have a secondary system?

I would recommend using a recent macOS system installed on an external drive and boot from there. Alternatively, you can downsize the existing APFS container, create a new APFS container on your main disk and install a secondary macOS there and boot into it. This is quite easily doable just by clicking in Disk Utility.app if you have enough space on the main drive and a macOS installer at hand.

Clone (restore) the source container

Disk Utility.app seems to be using asr command under the hood. You can man asr to read the docs, especially the sections about APFS restore.

When restoring you specify source and target. asr supports different ways how to specify source and target to cover scenarios related to previous filesystems. Source could be a DMG image file path or some existing disk in the form of /dev/diskN.

One might expect that specifying a DMG image file as source and some existing empty APFS container as target should do the right thing. No! You get cryptic errors. The trick is to:

  1. first, mount your DMG file => it creates a synthetised disk
  2. second, figure out which /dev/diskN corresponds to the synthetised disk created in step 1
  3. finally, use this synthetised disk as source for your asr restore operation

Make sense?

Let's be more concrete with my case. My DMG file mounted as disk9 which created synthetised disk10. As a target I created an empty APFS container in disk0 with single empty APFS volume via Disk Utility.app. The synthetised target container had identifier /dev/disk1:

> diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.0 TB     disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                 Apple_APFS Container disk1         1000.0 GB  disk0s2
/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +1000.0 GB  disk1
                                 Physical Store disk0s2
   1:                APFS Volume NewEmptyOS
...
/dev/disk9 (disk image):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   +499.2 GB   disk9
/dev/disk10 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +499.2 GB   disk10
                                 Physical Store disk9
   1:                APFS Volume MinimeOS - Data         311.1 GB   disk10s1
   2:                APFS Volume Preboot                 82.2 MB    disk10s2
   3:                APFS Volume Recovery                528.5 MB   disk10s3
   4:                APFS Volume VM                      4.3 GB     disk10s4
   5:                APFS Volume MinimeOS                10.9 GB    disk10s5

To execute restore I had to use asr command from command-line:

> sudo asr restore --source /dev/disk10 --target /dev/disk1
...
Volume replication failed - Read-only file system

The --useInverter is the tricky option here. Replication functionality which is enabled by default is quite recent and it does not play well with APFS Volume Groups. It fails when trying to do something with read-only system volume:

I consider this to be a bug and to be resolved in some future macOS version. Unfortunately Disk Utilty.app seem to be using asr without the --useInverter flag and gives me the same error in the GUI when trying to restore mounted DMG into existing container.

So I ran it and got:

> sudo asr restore --source /dev/disk10 --target /dev/disk1 --useInverter
...
APFS inverter failed to invert the volume - Invalid argument

Uh, oh!? Trying to run the restore again with --verbose and --debug flag gave me a hint:

> sudo asr restore --source /dev/disk10 --target /dev/disk1 --useInverter --verbose --debug
...
*** Mounting inner volume (ContainerToInvert)...
...
mount_inner_volume:884: Inner volume has snapshots

APFS inverter failed to invert the volume - Invalid argument

Ah, now I remember that the man page stated that --useInverter does not work with snapshots. The error does not tell me which volume has this issue but sure I can figure this out:

> diskutil apfs listSnapshots disk10s1
Snapshots for disk10s1 (20 found)
|
+-- 9801FBE4-E581-4B95-A055-8A94A9C2ABE9
|   Name:        com.apple.TimeMachine.2019-11-13-000654.local
|   XID:         2200904
|   Purgeable:   Yes
|   NOTE:        This snapshot limits the minimum size of APFS Container disk10
|
...
|
+-- 46C315AB-6014-4275-AF94-B284E476718A
|   Name:        com.apple.TimeMachine.2019-11-13-220854.local
|   XID:         2211035
|   Purgeable:   Yes
|
+-- 72FE9F27-A358-42F0-91CE-CBC0E8140B46
    Name:        com.apple.TimeMachine.2019-11-13-231202.local
    XID:         2212680
    Purgeable:   Yes

> diskutil apfs listSnapshots disk10s5
Snapshots for disk10s5 (3 found)
|
+-- B07AE72B-1EE9-4145-827F-CB31E06635BB
|   Name:        com.apple.TimeMachine.2019-11-13-211042.local
|   XID:         2210200
|   Purgeable:   Yes
|   NOTE:        This snapshot limits the minimum size of APFS Container disk10
|
+-- D124E17B-B061-4143-B69D-6626E5E8B41A
|   Name:        com.apple.TimeMachine.2019-11-13-220854.local
|   XID:         2211035
|   Purgeable:   Yes
|
+-- 1430DB4E-6527-4FA3-B226-77006055CA99
    Name:        com.apple.TimeMachine.2019-11-13-231202.local
    XID:         2212676
    Purgeable:   Yes

> diskutil apfs listSnapshots disk10s4
No snapshots for disk10s4

> diskutil apfs listSnapshots disk10s3
No snapshots for disk10s3

> diskutil apfs listSnapshots disk10s2
No snapshots for disk10s2

So the Data volume disk10s1 and OS volume disk10s5 seem to have some snapshots created by TimeMachine.

My task now is to delete them and try to run restore again. The problem is that my disk10 disks are read-only because they were mounted from read-only DMG.

... few hours of banging my head into a wall

Ok, I will spare you and won't describe all my trial and errors here.

Finally I found a way how to strip all snapshots from disk10s1:

> hdiutil attach -owners on /Volumes/W1/backups/mos.dmg -shadow
expected CRC32 $0A545D85
/dev/disk11         	                               	
/dev/disk12         	EF57347C-0000-11AA-AA11-0030654	
/dev/disk12s1       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS - Data 1
/dev/disk12s2       	41504653-0000-11AA-AA11-0030654	/Volumes/Preboot 1
/dev/disk12s3       	41504653-0000-11AA-AA11-0030654	/Volumes/Recovery 1
/dev/disk12s4       	41504653-0000-11AA-AA11-0030654	/Volumes/VM 1
/dev/disk12s5       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS 1

The magical -shadow option mounts the image as read-write and keeps diff of changes in /Volumes/W1/backups/mos.dmg.shadow file. So if you try this at home make sure you have enough space available for the shadow file. Not sure if -owners on is important but this worked for me after some previous problems. Also note that an alternative solution would be to use hdiutil convert -format UDRW ... and continue with converted read-write image. See the Q/A section below.

Anyways, having mounted writable disk allowed me to enumerate all snapshots and delete them:

# this is just a dry-run to see all snapshot GUIDs to be deleted
> diskutil apfs listsnapshots /dev/disk12s1 | grep "+--" | cut -d" " -f2
9801FBE4-E581-4B95-A055-8A94A9C2ABE9
DA44B969-EBA6-4EFE-ADCA-B4BB43FF6B37
8AEFF1F4-BC35-4C05-A21C-3122A97569B4
0149B1B9-9095-4896-B883-CD976802E419
A6E252E6-2007-4DD5-9313-EE2156C0C813
F3C38242-8AF7-42A8-9EC6-CF6381F647C0
D6391D9B-4ADB-44D4-9F05-C58FABC730AD
9281E5D8-867D-423D-A95F-7C4E3B4F34E2
82EA6CA1-FB41-40DC-8F80-B8C34311E491
EEE95161-ABFA-4E99-B893-2887DF4EC317
779FAA27-DACD-41C2-A408-E731DC6E96A1
36E7F342-9E5E-43F4-BA45-0441F9067244
8359CA22-79A6-43DF-A265-35BEB8D4FC7E
70105B62-19B0-4F14-9FFE-E73C6E32C74A
3C581209-FC4B-4D14-9491-95AE656B3335
068E2DD8-B6CF-454E-A5CC-8E68261EC9A6
422788AF-2A6E-40D6-976D-CC3F93A925CA
B647F1C2-E4CF-4F90-8A77-3C1CB2B992C9
# delete them all
> diskutil apfs listsnapshots /dev/disk12s1 | grep "+--" | cut -d" " -f2 | xargs -I{} sudo diskutil apfs deleteSnapshot disk12s1 -uuid {}
Deleting APFS Snapshot DA44B969-EBA6-4EFE-ADCA-B4BB43FF6B37 "com.apple.TimeMachine.2019-11-13-011547.local" from APFS Volume disk10s1
Started APFS operation
Finished APFS operation
Deleting APFS Snapshot 8AEFF1F4-BC35-4C05-A21C-3122A97569B4 "com.apple.TimeMachine.2019-11-13-030651.local" from APFS Volume disk10s1
Started APFS operation
Finished APFS operation
...
Deleting APFS Snapshot B647F1C2-E4CF-4F90-8A77-3C1CB2B992C9 "com.apple.TimeMachine.2019-11-13-211042.local" from APFS Volume disk10s1
Started APFS operation
Finished APFS operation

To double-check the result, I ran:

> diskutil apfs listsnapshots /dev/disk12s1
No snapshots for disk12s1

I repeated the same snapshot purging with disk12s5 and confirmed:

> diskutil apfs listsnapshots /dev/disk12s5
No snapshots for disk12s5

Ok, hopefully there are no snapshots in any volume in whole container and I can run the restore again using the massaged disk12:

> sudo asr restore --source /dev/disk10 --target /dev/disk1 --useInverter
	Validating target...done
	Validating source...done
	Validating sizes...done
	Restoring  ....10....20....30....40....50....60....70....80....90....100
	Verifying  ....10....20....30....40....50....60....70....80....90....100
	Inverting target volume...done
	Restoring  ....10....20....30....40....50....60....70....80....90....100
	Verifying  ....10....20....30....40....50....60....70....80....90....100
	Inverting target volume...done
	Restored target device is /dev/disk1s2.

The disk is bootable and seems to be 1:1 copy of the original. Sure not really a device-level block-by-block copy, but good enough.

It was easy, any questions?

How do I mount the DMG?

Case 1: if your DMG does not contain any snapshots:

Option 1: via Finder.app

Simply double-click the DMG file and it should mount. Then use diskutil list or Disk Utility.app to figure out which synthetised disk it was mounted as. It should have "APFS Container Scheme".

/dev/disk9 (disk image):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   +499.2 GB   disk9
/dev/disk10 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +499.2 GB   disk10
                                 Physical Store disk9
   1:                APFS Volume MinimeOS - Data         311.1 GB   disk10s1
   2:                APFS Volume Preboot                 82.2 MB    disk10s2
   3:                APFS Volume Recovery                528.5 MB   disk10s3
   4:                APFS Volume VM                      4.3 GB     disk10s4
   5:                APFS Volume MinimeOS                10.9 GB    disk10s5
Option 2: via command-line
> hdiutil attach /Volumes/W1/backups/mos.dmg 
Checksumming whole disk (Apple_APFS : 0)…
....................................................................................................................
         whole disk (Apple_APFS : 0): verified CRC32 $09A8F547
verified CRC32 $0A545D85
/dev/disk10         	EF57347C-0000-11AA-AA11-0030654	
/dev/disk10s1       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS - Data
/dev/disk10s2       	41504653-0000-11AA-AA11-0030654	
/dev/disk10s3       	41504653-0000-11AA-AA11-0030654	
/dev/disk10s4       	41504653-0000-11AA-AA11-0030654	
/dev/disk10s5       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS
/dev/disk9          	                    

Case 2: if your DMG has some snapshots, you must mount it as read-write with a shadow file

> hdiutil attach -owners on /Volumes/W1/backups/mos.dmg -shadow
expected CRC32 $0A545D85
/dev/disk11         	                               	
/dev/disk12         	EF57347C-0000-11AA-AA11-0030654	
/dev/disk12s1       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS - Data
/dev/disk12s2       	41504653-0000-11AA-AA11-0030654	/Volumes/Preboot
/dev/disk12s3       	41504653-0000-11AA-AA11-0030654	/Volumes/Recovery
/dev/disk12s4       	41504653-0000-11AA-AA11-0030654	/Volumes/VM
/dev/disk12s5       	41504653-0000-11AA-AA11-0030654	/Volumes/MinimeOS

How to convert read-only DMG to writable DMG?

This is what worked for me:

> hdiutil convert /Volumes/W1/backups/mos.dmg -format UDRW -o /Volumes/W1/backups/mos-rw.dmg
nx_kernel_mount:1387: : reloading after unclean unmount, checkpoint xid 2213158, superblock xid 2213151
Reading whole disk (Apple_APFS : 0)…
....................................................................................................
Elapsed Time:  3h 14m 56.431s
Speed: 40.7Mbytes/sec
Savings: 0.0%
created: /Volumes/W1/backups/mos-rw.dmg

You can then follow case 1 to mount /Volumes/W1/backups/mos-rw.dmg instead of /Volumes/W1/backups/mos.dmg.

As you can see from the output this conversion can take quite some time if the file is on external/slower disk, so it is preferable to use -shadow mounting as described in the article.

I've launched hdiutil/asr and it does seem to be stuck, what now?

With larger images/disks it might take up to several hours for operation to complete. The progress indicator in terminal output is not granular enough to display any action for several minutes. I would recommend to open Activity Monito.App switch to Disk view and observe most busy reader/writer processes. This should give you sense if your command is doing any work or is slacking. It could give you also rough lower bound estimate how far it is in progress because it usually needs to read or write its source or target data at least once. So if you have 300GB DMG and see reading speed 1GB/s you can get an idea if you can take a nap or not. Also better disable power-saving because your Mac could hibernate unexpectedly while working.

@kyledecot
Copy link

Thank you so much for documenting this process–I recently needed to migrate the data on my old MBP to a new one and this helped greatly.

My experience was a little different from yours however–I did not have to remove all of the snapshots nor did I have to use the --useInverter option. I also had to call diskutil unlockVolume ... as my volumes are encrypted w/ FileVault but other than that went very smoothly!

@ibrahim-malvanwala
Copy link

ibrahim-malvanwala commented Feb 6, 2020

Hey @darwin - this is a really interesting article.
I am facing a hard time to be able to image the container i.e. "Image from container diskN". the option is greyed out for me.
I am a newbee to Mac world and couldn't find out why it is not available to me.

Would you know how to make it available?

@nazgul33
Copy link

nazgul33 commented Feb 18, 2020

you should give full disk access to disk utility to make an image from APFS volume group. (I'm on 10.15.3)
SystemPreferences > Security & Privacy > Privacy > Full Disk Access > + > "Browse to Disk Utils -> Applications/Utility/" > ensure disk utility checked

@ibrahim-malvanwala
Copy link

@nazgul33 - I have tried that as well. I have given full disk access to both terminal and disk utility. Even after i unmount all volumes in the container (sudo diskutil unmountdisk diskX), it still doesn't enable me to take 'Image from container diskX'

@tschloeter
Copy link

tschloeter commented Mar 7, 2020

Thank you for sharing that process. I faced the same problem trying to restore my Mac from the GUI and your tutorial made me try it from the console.

However my solution was a bit easier although I don‘t know if this will work for everyone:

I just used the —erase parameter for asr to have the tool copy just blocks instead of contents.

  1. I booted the mac with internet restore mode (cmd + option + R on my 2012 MBPR)

  2. I mounted my backup image of the APFS container that I had previously created from the GUI:

hdiutil attach image.dmg -noverify -nomount

  1. I used diskutil list to find the synthesized devices.

  2. I used asr to block-copy the container to where it came from:

asr restore —source /dev/disk(synthesized-image) —target /dev/disk(synthesized-SSD-partition) —erase

That did the job for me.

Before I was able to reboot I had to let the computer idle for several hours because some system process was blocking one of the volumes and I did not want to interrupt it. I just selected to reboot from the GUI and after some hours I found the computer showing my login screen from the successfully restored system.

@siarsky
Copy link

siarsky commented Apr 17, 2020

Yesterday I've cloned a SATA HDD internal disk (HDD) with Mac Catalina (10.15.4) on it to a new SATA SSD disk (SSD) with Disk Utility and Terminal in rescue Mode. My Mac is MacBook Pro Mid 2012. No tricks needed, no DMG creation, no UUID partition renaming, all partitions (preboot and Recovery) were copied by Disk Utility:

  1. connect a new SSD drive over USB (I've used a USB 2.0 to 2.5” SATA Hard Drive Adapter) to your Mac
  2. reboot into a rescue mode (press Command+R during start, wait till progress bar appears, then release the keys)
  3. start a graphical "Disk Utility" => Set View (upper left corner) to Show All Devices
  4. erase SSD with these options: APFS with a GUID partition scheme, give it a name Backup : only one APFS volume "Backup" will be created
  5. unmount the newly formated APFS volume Backup, make sure the source (drive to be copied) is unmounted as well
  6. click on unmouted "Backup" volume and then press "Restore", select as a source the "Macintosh HD"
  7. wait (for me 500 GB over USB2 took 2.5 hours)
  8. after the restore the new drive will have two volumes, one mounted ("Macintosh HD") and one unmouted (if you mount this volume, you will see "Macintosh HD Data")
  9. Be sure the newly restored APFS "Macintosh HD" volume is mounted, then close "Disk Utility"
  10. Start a "Terminal" application from the rescue upper menu
  11. mark the new cloned volume as bootable:
    bless -folder /Volumes/Macintosh\ HD\ 1/System/Library/CoreServices
    DON'T FORGET the 1 in "Macintosh HD 1" !!!!
  12. test it: reboot Mac and press Option (Alt) key : the new SSD appears as a bootable volume
  13. shutdown your Mac, open it and replace HDD with SSD (use iFixit.com manual for your model)

Remarks:

  1. the VM partition (Swap partition) will not be cloned, but was created by the system afterwards
  2. even only one APFS volume is selected for restore both volumes ("Macintosh HD" and "Macintosh HD Data") will be correctly cloned
  3. Those steps above work with previous non-APFS file systems also, I used them already 5-6 times
  4. Important steps again:
  • unmount before restore: otherwise you will get an error that source/target is not compatible
  • select the volume (not the disk, not the APFS container) for restore : otherwise you will get an error
  • mount the volume after restore, otherwise you will not be able to run "bless" correctly

@yangx2000
Copy link

yangx2000 commented May 16, 2020

I have followed @siarsky's instruction, and have successfully cloned the internal disk on my late 2014 Mac Mini running Catalina (10.15.4) to the solid state drive (SSD). Here are the steps that I took:

  1. Login my account, Unmount and unplug all external disks that are currently connected to the Mac.
  2. Connect the new 1TB SSD drive (https://www.amazon.com/gp/product/B073SBQMCX/ref=ppx_yo_dt_b_asin_title_o08_s01?ie=UTF8&psc=1) to the Mac over USB-3 through a disk enclosure (https://www.amazon.com/gp/product/B07NF6JQQ4/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8&psc=1); from now on, this is the ONLY external disk being connected to the Mac
  3. Start "Disk Utility" (LaunchPad > Other > Disk Utility); then set View (upper left corner) to Show All Devices; Select the external SSD disk in the device list on the left.
  4. Erase and partition the SSD (APFS with a GUID partition scheme), and leave the default name as "Untitled"
  5. If you are following these steps, please make sure there were no errors at step 4; otherwise you should stop, and fix the error first; For now on, assume there were no disk errors or other hardware problems.
  6. Reboot the Mac into Recovery mode (press Command+R during start, wait till progress bar appears, then release the keys). Wired keyboard is preferred, but wireless keyboards work too. I am using Logitech K750 (https://www.amazon.com/Logitech-K750-Wireless-Solar-Keyboard/dp/B005L38VRU?ref_=ast_sto_dp)
  7. Start "Disk Utility" from the "MacOS Utilities" list, click "Continue" button
  8. Set View to "Show All Devices"; Unmount "Macintosh HD" and "Macintosh HD -Data" under the internal disk, and also the "Untitled" under the external disk; make sure you unmount the "Volume" not the "container disk".
  9. Select "Untitled" volume (which should be grey, because it is unmounted), and then select  "Restore" (at the top of the screen), select as a source the "Macintosh HD" ,accept the warning etc. and go on.
  10. Now, the restore/cloning process has started; I have a 1TB mechanic internal disk, and I had to leave it running for overnight. Do not trust the progress bar for the progress. In my case, the progress bar reached 90% in few minutes, then stuck there overnight.
  11. After the restore process completes, I have an exact clone of the internal disk on the SSD; On the device list, there are "Macintosh HD" and "Macintosh HD - Data" volumes under both the internal disk and the external SSD disk. My case is slightly different from @siarsky's post: all of the 4 volumes are unmounted (grey). If they are mounted in your case, you should unmount them first. Then I mount only the "Macintosh HD" volume on the external SSD; From now on, This is the only mounted volume.
  12. Selected the mounted "Macintosh HD" volume on the external SSD, write down the mounting point path at "Mount Point:" in the list on the right side of the screen. My screen says, "/Volumes/Macintosh HD", without "1" at the end. Again, it is different from @siarsky's post, may be because "Macintosh HD" is the only mounted volume (see step 11).
  13. Exit "Disk Utility", and back to the "MacOS utilities" screen under the Recovery mode.
  14. Start a "Terminal" application from the top menu bar;
  15. Follow siarsky's instruction to make the new cloned volume as bootable, type:
bless -folder [the mounting path you wrote down at step 12]/System/Library/CoreServices
  16. Exit "Terminal"; Leave the SSD disk connected to the Mac; select "Restart" at the top menu.
  17. Press Option (Alt) key, while the Mac is restarting; I see both the internal disk and the external SSD are listed as bootable volumes; and I select the external SSD, then continue;
  18. The Mac boots as it normally does; Login my account and it seems everything is exactly as it was before the cloning;
  19. To verify the MacOS is running off the SSD. I start "Disk Utility"; and I see both internal and external SSD are listed under the device list; and under the external SSD disk, click "Macintosh HD" I see the "smiley face" on the disk icon at the top right, and click "Macintosh HD -Data" I see the "yellow house" on the icon;  This indicates the Mac is currently booted from the external SSD. I have also double checked the two volumes under the internal disk, and I don't see the "smiley face" and the "house" when I select them. This confirms the Mac is running from the external SSD, and everything works.
  20. Shutdown the Mac, and disconnect the external SSD; I am done with cloning.

 
I had the disk installed this morning, by following the instructions on ifixit.com; no surprises, no crises ; the Mac booted fine after the installation.
 
Once again, I would like to thank @siarsky and everyone for the detailed instructions on the cloning process. I would not have dared to get the project started without these valuable instructions.

@willmo
Copy link

willmo commented May 27, 2020

Re: @darwin's error Volume replication failed - Read-only file system, I found that this happened whenever the source was read-only. Like @darwin, and unlike most later commenters, I was using a disk image as the source. In my case it was already a writable type of image (a raw image that I had created directly from my old computer's APFS partition with dd), but I had used hdiutil attach -readonly to avoid accidentally corrupting it. The new default replication-based restore always failed with that error until I attached it read-write. Then I could use the Disk Utility UI to do the restore as described by the last two commenters. (I did this from Recovery mode on 10.15.5.)

No idea why the source would need to be writable; seems like a bug to me.

@maxscience
Copy link

maxscience commented May 29, 2020

@siarsky and @yangx2000 steps are not working for me. I followed those steps to clone a MacBook Pro 15" 2019 with 10.15.5 to another brand new MacBook Pro 15" 2019 connected via an Apple USBC thunderbolt cable using Target Disk Mode.
When I boot the new Mac up, the spinning globe is showing up as it cannot recognize a bootable drive.

@IvanExpert
Copy link

IvanExpert commented Aug 21, 2020

This was super, super, super helpful in terms of understanding APFS' operations. Thank you. For this particular use case, I would just use Carbon Copy Cloner, but I wanted a lower level understanding. After reading your piece, and closely persuing "man asr", it appears to me that restoring all the volumes in a multi-volume container requires a one-volume-at-a-time process, which just seems crazy to me. Even if Apple's not going to provide a block-copy of the whole container, you'd think they could automate serial multiple restore from a single container.

Though I also don't know why they couldn't provide a block copy if a container is your target, and then fix things up at the destination; it practically does that anyway during the inversion, but that process only lets you operate on one source volume, even after it's already block-copied over the whole container. (A true block-copy manual process for keeping all volumes is laid out here: http://blog.tempel.org/2019/05/cloning-apfs-volumes-containers-apfs.html, though it seems best suited to unencrypted disks, and perhaps only those on non-T2 Macs.)

@tschloeter I think while you got a mostly-block-copy, you also got only a single volume out of your source. That's fine if you only had a single volume anyway, but I think you would have had problems with more. If I understand the asr man page correctly, it no longer copies at the file level anyway, and if you specify --erase with the whole APFS container as a target, all it does is delete the container before making a new blank empty volume to receive the container copy, then inverts the one volume it finds inside that copy. If you'd had more than one, you would have had to manually specify which one you wanted, and then subsequent volumes would have been copied (inefficiently) via the same process, but without --erase, in which case it adds a new volume without deleting the container or the previous volume.

@maxscience I wonder if this has anything to do with T2 Macs storing all data encrypted, whether or not you elect to use FileVault.

@mrpriceisright
Copy link

very good stuff. I don't know if I got lucky or they did something in diskutil (although it says 2019).
I was struggling with this in the process of cloning a macbook catalina disk (HD) to ssd (after several attempts).
In the end I installed the ssd, put the HD in a usb case and booted from it (usb) in recovery mode.
Ran diskutil.
Initialized the ssd as APFS (encrypted, just like the HD).
Had to mount the hd (macintosh HD),
The clicked on the ssd and restore,
picked the MACINTOSH HD from the list.
took quite a while to copy BUT it created both MACINTOSH HD and MAC HD-DATA APFS volumes on the SSD.
during the copy it said "replicating"
when it said done, I had to mount the SSD before I could select it as a boot device.
then reboot. Appears to have everything.

@yogesch
Copy link

yogesch commented Sep 24, 2020

So my situation is somewhat similar to @siarsky and @yangx2000 but I have a 1TB HDD with two equal partitions (one bootable and another for data) and a 500GB SSD. I have a couple of questions it would be great to have answers to.

  1. Should it suffice to restore just the bootable partition onto the container of the SSD?

  2. What about the recovery partition? How do I get a recovery partition on the SSD? @mrpriceisright in your approach of placing the new SSD internally, does a recovery partition also get created automatically?

@darwin
Copy link
Author

darwin commented Nov 16, 2020

@Xenthio
Copy link

Xenthio commented Jul 14, 2021

prohibitory sign shows when i try to restore a big sur dmg using catalina

@oakwellconsulting
Copy link

oakwellconsulting commented Oct 25, 2021

Thank you so much for posting this. I don't know what I would have done had I not found this page.

I had a little bit of head banging against the walls on my own. Moving from a 2013 MBP running Catalina to a 2015 machine with a larger disk, I first created the DMG image. I was careful to remove any snapshots beforehand but still had the APFS inverter failed to invert the volume - Invalid argument error. In my case, the error occurred trying to restore the VM volume (something that probably should not even have been attempted by 'asr' in the first place).

In any case, after a few trials and errors, I decided to use a different external HDD and start again from scratch. When I re-created the image, I made it read/write rather than read-only and voilà! Everything restored without the '--useInverter' flag and it went as smooth as silk.

Thanks again for sharing your experience here!

@FlexMcMurphy
Copy link

FlexMcMurphy commented Nov 19, 2021

Following the instructions above I made a backup image... and first time I went to restore it to a drive it worked fine.

I then tried a second time to another drive but now when I use the command to attach it as read/write:
hdiutil attach -owners on [path to .dmg file] -shadow

In Terminal I get: "expected CRC32 $0A545D85" and the image attaches as read only instead of "Checksumming whole disk... verified" and the image attaching as read/write.

Why doesn't my backup image attach as read/write anymore?

Thank you all for this amazing help... surprisingly difficult to find this anywhere on the web.

Update
I discovered that you don't need to use the -owners on and -shadow options to attach/mount the disk image file.
You only need those options to mount it as read/write because you need to e.g: remove snapshots from volumes in the disk image.

I used this command which attached the disk image in about 10 seconds instead of several minutes:
hdiutil attach "/Volumes/USBDrive/macOSCatalinabackup.dmg"
The disk image gets mounted as read/only but that is fine.
Now you can use the asr restore command to restore the synthesized disk image to the target drive.

Cheers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment