The "each Leaflet object as a Svelte component" thing wasn't the first I tried. My first attempt manages the state manually with a process similar to what I described here: https://imfeld.dev/writing/leaflet_with_svelte#syncing-local-state-to-the-map-through-reactive-statements. Essentially I had some code blocks that keep track of which objects I had created in Leaflet, and manually reconciled that with the declarative state from the reactive statements, adding or removing markers/lines/etc. I had a code block like this for each type of object that the app used.
// A Map containing all the markers that we have created.
let markers = new Map();
let allLocations = {...};
// Create the markers.
for (let [id, data] of Object.entries(allLocations)) {
let marker = L.marker(data.location, { icon: data.icon });