Cloud and Local Storage

RSS for tag

Store data locally or in the cloud.

Posts under Cloud and Local Storage tag

46 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Safari ios 18.2 download problems
iPhone 15pro iOS 18.2 Downloaded files cannot be located anywhere in Files, only by accessing Downloads in Safari. I have tried setting download folder to various locations, iCloud, Phone, Google Disk, but nothing is stored. Has an invisible cache or temp folder been introduced? If so, it is a total fail: When press-holding any file in Safari download, the normal file action options (Quick Look, share, store to Files, etc) are not available. When clicking any file it opens any of several apps that has this file type associated with it, and there is no way to change the default app or disable the forced opening of an app. I tried deleting the app opening .csv (in this case OneDrive), and another irrelevant app opened. There seems to be a hierarchy of apps-file types, and it has no logic to it. in Chrome behaviour is as expected. Chrome vs. Safari screen recordings: https://shorturl.at/my3Oy
1
0
92
2h
Avoiding deletion of referenced records
Let's say I have a CloudKit database schema where I have records of type Author that are referenced by multiple records of type Article. I want to delete an Author record if no Article is referencing it. Now consider the following conflict: device A deleted the last Article referencing Author #42 device B uploads a new Article referencing Author #42 at the same time The result should be that Author #42 is not deleted after both operations are finished. But both device don't know from each other changes. So either device B could miss that device A deleted the author. Or device A could have missed that a new Article was uploaded and therefore the Author #42 was deleted right after the upload of device B. I though about using a reference count first. But this won't work if the ref count is part of the Author record. This is because deletions do not use the changeTag to detect lost updates: If device A found a reference count 0 and decides to delete the Author, it might miss that device B incremented the count meanwhile. I currently see two alternatives: Using a second record that outlives the Author to keep the reference count and using an atomic operation to update and delete it. So if the update fails, the delete would fail either. Always adding a new child record to the Author whenever a reference is made. We could call it ReferenceToken. Since child records may not become dangling, CloudKit would stop a deletion, if a new ReferenceToken sets the parent reference to the Author. Are there any better ways doing this?
0
0
94
1w
Data Persistence of AVAssets
Hey, I am fairly new to working with AVFoundation etc. As far as I could research on my own, if I want to get metadata from let's say a .m4a audio file, I have to get the data and then create an AVAsset. My files are all on local servers and therefore I would not be able to just pass in the URL. The extraction of the metadata works fine - however those AVAssets create a huge overhead in storage consumption. To my knowledge the data instances of each audio file and AVAsset should only live inside the function I call to extract the metadata, however those data/AVAsset instances still live on on storage as I can clearly see that the app's file size increases by multiple Gigabytes (equal to the library size I test with). However, the only data that I purposefully save with SwiftData is the album artwork. Is this normal behavior for AVAssets or am I missing some detail? PS. If I forgot to mention something important, please ask. This is my first ever post, so I'm not too sure what is worth mentioning. Thank you in advance! Denis
1
0
167
2w
Iframe LocalStorage is Lost when opening a new window
I have a homepage with an iframe containing my app LocalStorage, but when opening my app page the LocalStorage is lost. structuration of the page: homepage : domain.com iframe and app : app.domain.com the localstorage is set up directly within the iframe and postmessage between iframe and parent works. We see in the parent page that the LocalStorage for the sub-domain is set but opening the sub-domain on a new window it disappears : The problem is detected On Safari 18 only.
0
0
283
Oct ’24
Any specific event that can be monitored for tracking file upload via fileproviderd/extension?
Hi Team, I am trying to explore ESF events specifically generated by cloudsync extensions built on File Provider framework. Brief: I have high-level understanding of how various cloud vendors have provided their extensions to sync data from cloud/remote storage to local filesystem (and vice-versa). e.g.iCloudDriveFileProvider (icloud), DFSFileProviderExtension (google drive). There are 2 ESF AUTH events for file provider I can see namely: ES_EVENT_TYPE_AUTH_FILE_PROVIDER_MATERIALIZE , ES_EVENT_TYPE_AUTH_FILE_PROVIDER_UPDATE. and respectively their NOTIFY events. Observation: Observed that these events are generally triggered by fileproviderd process during download scenario i.e. syncing files from cloud/remote storage to local file system. i.e. 'materialize' for new file creation and 'update' for updating existing file. Question/Problem: Is there a correct way to find which cloud provider has triggered this download event? i.e. weather it is iCloudDriveFileProvider or DFSFileProviderExtension (there is this instigator field in Materialize event struct, but could not find similar for Update event. Are there similar ESF events for upload scenario? (I have fair understanding of how file-to-upload is copied to temp location and then uploaded by respective extensions to remote storage, but then they work with original files clone created in their temp location, so the AUTH events generated by this extension will wont reveal the original file name even if I am able to get the Fileprovider name) To Summarize: Basically I am looking for ESF event that will be triggered during upload scenario that can also let me know original file name as well the cloudprovider extension process name. As of now 'fileproviderd' process name is obtained from filesystem ESF events like AUTH_OPEN etc.
4
0
391
Sep ’24
handling files offloaded to iCloud?
My app saves its document files by default into ~/Documents. It does some important domain-specific stuff when a document is deleted. I monitor for deletion using https://github.com/eonist/FileWatcher Unfortunately several users have noticed my app doing this cleanup work even when they have not deleted the corresponding document. We've traced it through and realised it's the iCloud "Optimise Mac Storage" feature, or "Store in iCloud > Desktop and Documents". I'm not sure which because I don't use these features of macOS at all, and also they seem to have been renamed or changed in Sonoma. Either way, I'm wondering: a) how I can tell in Swift whether a file has actually been deleted, or whether it's been "offloaded" to iCloud by macOS. b) how can I test this? My research is pointing at urlubiquitousitemdownloadingstatus but it's hard to play with it without knowing how to test it.
2
0
365
Sep ’24
How to retrieve previously set title for a CKShare?
I create a CKShare and then I set the title: share[CKShare.SystemFieldKey.title] = title I even call: if let cloudStore = coreDatabase.cloudStore { try await persistentContainer.persistUpdatedShare(share, in: cloudStore) } but when I retrieve this share again using persistentContainer.fetchShares(matching: [id]) the title is not set. I even checked CloudKit console and I can't see there title either... How can I retrieve the previously set title for a share?
1
0
323
Aug ’24
How to remove a single record from a CKShare?
It is possible to append a record to a CKShare using NSPersistentCloudKitContainer.share(objects, to: share) but how can I reverse this operation and remove the object from share? The workaround would be to delete and recreate the object, but is there any SDK function to do it right? The more I work with CoreData+CloudKit the more it seems like everything there is a workaround or hack or bug... This SDK is still in Alpha at best.
0
0
274
Aug ’24
CoreData sharing/collaboration feature is broken
On top of unstable CloudKit sync, we've got also extremely unstable sharing/collaboration functionality. I mean, it's quite impressive how easy it is to enable sharing, but this feature should not be released at this point. It feels like using software in the alpha version. Let's take NSPersistentCloudKitContainer.share(_:to:) (https://developer.apple.com/documentation/coredata/nspersistentcloudkitcontainer/3746834-share), the documentation says: Sharing fails if any of the following conditions apply: Any objects in managedObjects, or those the traversal finds, belong to > an existing share record. However, it's wrong... if you pass any object from the existing share, it will return the same share... It never fails in this case. Things are getting even weirder if you experiment a little bit with shares. So let's assume you share a couple of objects: persistentContainer.share([A, B, C, D], to: nil) Now you stop sharing those via UICloudSharingController and you want to share again but just C. So you call: persistentContainer.share([C], to: nil) and surprise, surprise, you get a new share URL, but you share all previously shared objects (A, B, C, D), not as you would have expected only C... On top of that, you keep getting some weird errors like: error: CoreData+CloudKit: -[NSCloudKitMirroringDelegate _requestAbortedNotInitialized:](2190): <NSCloudKitMirroringDelegate: 0x3029b84b0> - Never successfully initialized and cannot execute request '<NSCloudKitMirroringExportRequest: 0x30359f1b0>123123123123' due to error: <CKError 0x3018699b0: "Partial Failure" (2/1011); "Failed to modify some record zones"; uuid = 12312312312312; container ID = "iCloud.some.id"; partial errors: { com.apple.coredata.cloudkit.share.123123123123123:__defaultOwner__ = <CKError 0x30186a3d0: "Invalid Arguments" (12/2006); server message = "Only shared zones can be accessed in the shared DB"; op = 12312312312312; uuid = 123123123123> }> Even though the only thing I use is persistentContainer.share and UICloudSharingController(share: share, container: cloudKitContainer) And the cherry on the top, from time to time if you play a little with sharing multiple objects at once, it happens that it randomly wipes out some records from that share... I don't even know where to start reporting issues and if its worth it, because every ticket will be dismissed anyway because "we need thousands of your logs, we require you to do all the job for us".
1
0
384
Aug ’24
The recommended way of handling CoreData+CloudKit errors
What is the recommended way of intercepting and processing errors? As far as I know, there are 4 main areas: iCloud account status - this can be checked and intercepted via notifications exceptions from fetch/execute/save - it can be a simple do..catch, but what exceptions can we expect here, what should be handled, and how? there could be some asynchronous issues with synchronization. How should we intercept them and how should they be handled? issues with iCloud storage - quota exceeded, etc. How to intercept & handle those? I'm trying to achieve production-ready implementation, but there are many pitfalls and hidden issues that are not well documented. Could you provide some advice on how to handle properly all these situations?
3
0
397
Aug ’24
Prevent wiping out all records when user disables iCloud sync for the app
Hello, I'm using NSPersistentCloudKitContainer. If the user disables iCloud sync for my app in the system settings and opens the app, all records are immediately wiped out, even if there are unsynced changes (like records added offline). Disabling iCloud sync doesn't even show any warning, so the user may lose all data (if it's not already synced to Cloud). Is it possible to intercept that the store will be wiped out when the app is launching? I would copy all records to the local storage then to avoid losing data by the user.
1
0
379
Aug ’24
CoreData + CloudKit synchronization is very unstable & unreliable
CloudKit sync is very unstable. Sometimes it just stops syncing for no reason, other times it works almost instantly. The core issue with synchronization is that CoreData relies mostly on two things: silent push notifications, which are by design unreliable and can be throttled user interactions, I noticed that the local database is updated most likely periodically and also based on some app events like entering the foreground. Unfortunately, there is no SDK function that allows us to force sync with CloudKit, which basically prevents us from providing some features to recover if a user encounters problems. After thousands of tests, I finally discovered what was wrong and how to make the synchronization stable. Basically, I noticed that at some point CoreData decides that it won't synchronize data unless you deactivate and activate the application, which is crazy. It's getting even worse if we talk about extensions like the keyboard extension on iOS. The same happens on all platforms. Therefore, knowing that I implemented a trick that happened to work perfectly. The workaround requires to periodically sending an event pretending that the app is going foreground. macOS: var cancellable = Set<AnyCancellable>() // ... Timer.publish(every: 20.0, on: RunLoop.main, in: .common) .autoconnect() .sink { _ in NotificationCenter.default.post(.init(name: NSApplication.willBecomeActiveNotification)) } .store(in: &cancellable) iOS: var cancellable = Set<AnyCancellable>() // ... Timer.publish(every: 20.0, on: RunLoop.main, in: .common) .autoconnect() .sink { _ in NotificationCenter.default.post(.init(name: UIApplication.didBecomeActiveNotification)) } .store(in: &cancellable) After that, everything works perfectly. Pitty that the solution mostly meant for enterprise is so unstable and there is not even a single SDK function to recover from that (force sync). Any plans to fix CoreData+CloudKit? I also created a ticket: #FB14531806.
2
0
638
Aug ’24
CloudKit is not accessible from iOS extension targets
Hi! I'm using CoreData + CloudKit. It works well both on macOS and iOS, however, I can't make it work with extensions (share, action, keyboard). I get Invalid bundle ID for container error: <CKSchedulerActivity: 0x3029f4d20; identifier=com.apple.coredata.cloudkit.activity.export.A65D5B7A-18AA-400A-B25F-F042E46646F6, priority=2, container=iCloud.com.org.app.dev:Sandbox, relatedApplications=( "com.org.App.dev.App-Keyboard" ), xpcActivityCriteriaOverrides={ ActivityGroupName = "com.apple.coredata.cloudkit.App Keyboard.A65D5B7A-18AA-400A-B25F-F042E46646F6"; Delay = 0; Priority = Utility; }> error: CoreData+CloudKit: -[NSCloudKitMirroringDelegate _recoverFromPartialError:forStore:inMonitor:](2812): <NSCloudKitMirroringDelegate: 0x303fd82d0>: Error recovery failed because the following fatal errors were found: { "<CKRecordZoneID: 0x300ef9bc0; zoneName=com.apple.coredata.cloudkit.zone, ownerName=__defaultOwner__>" = "<CKError 0x300efa5e0: \"Permission Failure\" (10/2007); server message = \"Invalid bundle ID for container\"; op = xxxxxxx; uuid = zzzzz-xxxxx; container ID = \"iCloud.com.org.app.dev\">"; } I checked everything 10x: profiles, bundle ids, entitlements, etc. I even removed all local provisioning profiles and recreated them, I also tried setting different CloudKit container, but nothing helps. I tested it on a real device. My setup: main app bundle id: com.org.App.dev keyboard bundle id: com.org.App.dev.App-Keyboard action extension bundle id: com.org.App.dev.Action-Extension CloudKit container id: iCloud.com.org.app.dev I keep the CoreData database in the app group container, but I also tried locally and it doesn't really matter. This is how I setup my CoreData: self.persistentContainer = NSPersistentCloudKitContainer(name: "AppCoreModel") persistentContainer.persistentStoreDescriptions = [createCloudStoreDescription()] persistentContainer.loadPersistentStores { [self] _, error in if let error { logError("Could not load Core Data store \(error)") } else { persistentContainer.viewContext.automaticallyMergesChangesFromParent = true persistentContainer.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy logDebug(persistentContainer.persistentStoreDescriptions.first?.url?.absoluteString ?? "") logDebug("Core Data store loaded") } } private func createCloudStoreDescription() -> NSPersistentStoreDescription { let cloudStoreOptions = NSPersistentCloudKitContainerOptions( containerIdentifier: "iCloud.com.org.app.dev" ) cloudStoreOptions.databaseScope = .private let documentsUrl = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: AppConstants.appGroupId)! let cloudStoreDescription = NSPersistentStoreDescription( url: documentsUrl.appendingPathComponent("cloud-database.sqlite") ) cloudStoreDescription.type = NSSQLiteStoreType cloudStoreDescription.cloudKitContainerOptions = cloudStoreOptions cloudStoreDescription.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey) cloudStoreDescription.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey) return cloudStoreDescription } Any help would be highly appreciated. It seems like iOS bug, because everything seems to be configured properly. I even checked app identifiers if containers are properly assigned. Similar issue when using CloudKit directly (unresolved): https://developer.apple.com/forums/thread/665280
3
0
491
Sep ’24
How to ensure fast synchronization between devices using iCloud Database?
For a long time, I've been using just a shared file via iCloud to ensure synchronization between devices. It works very well, everything is synchronized within seconds. However, I started encountering issues with race conditions, especially if there are more than 2 devices updating data. It's easy then to override the database accidentally. That's why I decided to try with the tool that is designed for that: iCloud Database. However, I quickly realized that the performance was extremely low. As I understand, the recommended approach to listen to changes in the database is to create a subscription, and I did that. However, during my tests, I discovered that it can take even 10 or more minutes to get that notification. It's unacceptable, some user could be waiting for new changes on their device. I could of course spam with fetch every 10 seconds (using changeToken), but I'm not sure if it's a good idea. I could also provide manual fetch like pull to refresh or something like that, but automatic synchronization would be much better for the user.
2
0
401
Jul ’24
Is History Tracking in Cloudkit shared database needed?
I’ve setup the Cloudkit persistent container with private and shared database (see code below). I’ve enabled NSPersistentHistoryTrackingKey to true also for .shared database. I’ve noticed in the example from Apple that the History Tracking is only enabled in .private but not for .shared. Questions: For a CloudKit setup to sync (a) between owners’ own devices (only private database), and (b) between multiple iCloud Users through .private and .shared databases, Do I need to enable history tracking for .shared database if I want to check the remote changes in the .shared database (or is the history tracking of the .private database of the owner also accessible in the .shared database)? ======================== let APP_BUNDLE_IDENTIFIER = Bundle.main.bundleIdentifier! let APP_GROUP_IDENTIFIER = "group." + APP_BUNDLE_IDENTIFIER private func setupPersistentContainer(_ container: NSPersistentCloudKitContainer? = nil, isStartup: Bool = true) -> NSPersistentCloudKitContainer { let container = container ?? getCloudKitContainer(name: CORE_DATA_DATA_MODEL_NAME) let defaultDirectoryURL: URL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: APP_GROUP_IDENTIFIER) ?? NSPersistentCloudKitContainer.defaultDirectoryURL() let privateDataStoreURL = defaultDirectoryURL.appendingPathComponent("PrivateDataStore.store") let sharedDataStoreURL = defaultDirectoryURL.appendingPathComponent("SharedDS.store") // MARK: Private Store configuration let privateDataStoreDescription = NSPersistentStoreDescription(url: privateDataStoreURL) privateDataStoreDescription.configuration = "PrivateDataStore" // Enable lightweight migration privateDataStoreDescription.shouldInferMappingModelAutomatically = true privateDataStoreDescription.shouldMigrateStoreAutomatically = true // Turn History Tracking privateDataStoreDescription.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey) let logOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: CLOUDKIT_LOG_CONTAINER_ID) logOptions.databaseScope = .private privateDataStoreDescription.cloudKitContainerOptions = logOptions // turn on remote change notifications privateDataStoreDescription.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey) container.persistentStoreDescriptions = [privateDataStoreDescription] // MARK: Share Store configuration let sharedDataStoreDescription = NSPersistentStoreDescription(url: sharedDataStoreURL) sharedDataStoreDescription.configuration = "SharedDS" // MARK: Enable lightweight migration sharedDataStoreDescription.shouldInferMappingModelAutomatically = true sharedDataStoreDescription.shouldMigrateStoreAutomatically = true sharedDataStoreDescription.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey) let sharedOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: CLOUDKIT_LOG_CONTAINER_ID) sharedOptions.databaseScope = .shared sharedDataStoreDescription.cloudKitContainerOptions = sharedOptions // turn on remote change notifications sharedDataStoreDescription.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey) container.persistentStoreDescriptions.append(sharedDataStoreDescription) self.stores = [StoreType : NSPersistentStore]() container.loadPersistentStores(completionHandler: { [self] (storeDescription, error) in if let error = error as NSError? { print(error) } if let cloudKitContainerOptions = storeDescription.cloudKitContainerOptions { if cloudKitContainerOptions.databaseScope == .private { self.stores[.privateStore] = container.persistentStoreCoordinator.persistentStore(for: storeDescription.url ?? privateDataStoreURL) } else if cloudKitContainerOptions.databaseScope == .shared { self.stores[.sharedStore] = container.persistentStoreCoordinator.persistentStore(for: storeDescription.url ?? sharedDataStoreURL) } } else { self.stores[.privateStore] = container.persistentStoreCoordinator.persistentStore(for: storeDescription.url ?? privateDataStoreURL) } }) /// Automatically merge changes in background context into View Context /// Since we always use background context to save and viewContext to read only. The store values should always trump container.viewContext.automaticallyMergesChangesFromParent = true container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy // Create separate context for read and write container.viewContext.name = VIEW_CONTEXT_NAME container.viewContext.transactionAuthor = self.contextAuthor self.setQueryGeneration(context: container.viewContext, from: .current) return container }
1
0
413
Jul ’24
Download a portion of a (large) file from iCloud drive
I have a large file saved in iCloud drive. I need just a portion of that file. func ubiquitousData(file: URL, offset: UInt64, size: UInt64) async -> Data { // downloads just the portion of the ubiquitous `file`. } FileManager already has an api that downloads the full file. Is there a way to download just a portion of the file?
0
0
358
Jul ’24
Public and Private CK sync failure, macOS mistake = add data before indexing
I have a Multiplatform app for iOS and macOS targets. I am using CloudKit with CoreData and have successfully established a private and public database. The app has successfully synced private and public data for months between macOS (dev machine), an iPhone 13 Pro and an iPad Pro 12.9inch 2nd gen. The public data also syncs perfectly to simulator instances running under other iCloud accounts. Recently I added a new entity in the public DB and here is where I seemed to have made a mistake. I entered data into the new public database via my developer UI built into the macOS app running on my MBP before I indexed the necessary fields. Side note - I find it necessary to index the following for each Entity to ensure iCloud sync works as expected on all devices... modifiedTimestamp - Queryable modifiedTimestamp - Sortable recordName - Queryable Realising my mistake, I indexed the above CKRecord fields for the new Entity. Since then, the macOS target has remained in some way "frozen" (for want of a better term). I can add new public or private records in the macOS app but they do not propagate to the public or private stores in iCloud. I have attempted many fixed, some summarised below: clean build folder from Xcode; remove all files from the folder /Users//Library/Containers/, place in recycle bin, empty recycle bin, then build and run; build and run on iPhone and iPad targets to ensure all apps are current dev version, then repeat above processes. I've read through the console logging when I build and run the macOS app many many times to see whether I can find any hint. The closest thing I can find is... BOOL _NSPersistentUIDeleteItemAtFileURL(NSURL *const __strong) Failed to stat item: file:///Users/<me>/Library/Containers/com.me.AppName/Data/Library/Saved%20Application%20State/com.me.AppName.savedState/restorecount.plist but my research on this and other forums suggests this is not relevant. Through this, the app still functions as expected on iOS devices and both private and public database additions and modifications propagate to iCloud stores and to other devices. I expect that removing the macOS app entirely from my dev machine would trigger a complete sync with all existing data. Imagine I bought a new macOS device and chose to install my app where before I had run this only on my iOS devices. My current problem suggests that I could not do this, but I know that this is not the intended behaviour. This scenario makes me think there is a setting file for my macOS app that I'm not aware of and that this impeding the sync of all existing app data back to the fresh install of the macOS app? But that is a wild guess. Running public releases (no betas) Xcode 15.4 (15F31d) macOS Sonoma 14.5 physical iOS devices running iOS 17.5.1 Any words of wisdom on how I might go about trying to solve this problem please?
1
0
543
Jul ’24