obtaining file creation & modified time & size failing 5-10% of time within .onOpenURL when file shared with app

When users share a file with my app I am having trouble 5-10% of the time obtaining the file meta data, specifically creation and modified time and size.

Using SwiftUI with the code below..

.onOpenURL { url in

var fileSize: Int64 = 0
var creationTime: Date = Date(timeIntervalSince1970: 0)
var modificationTime: Date = Date(timeIntervalSince1970: 0)

do {
     let fileAttributes = try FileManager.default.attributesOfItem(atPath: url.path)

     fileSize = fileAttributes[FileAttributeKey.size] as? Int64 ?? 0

     creationTime = fileAttributes[FileAttributeKey.creationDate] as? Date ?? Date(timeIntervalSince1970: 0)
                
     modificationTime = fileAttributes[FileAttributeKey.modificationDate] as? Date ?? Date(timeIntervalSince1970: 0)

		<SNIPPED CODE no other tries though and not involving above variables>


} catch {
// quite confident I am ending up here because variables after the above code aren’t being set and there are no other try blocks, 
// so  FileManager.default.attributesOfItem(atPath: url.path) must be throwing….
} 
                       
<SNIPPED CODE>

To attempt to resolve this, I added in a 0.5 second wait cycle if creationTime == 0 and modificationTime == 0 , so if obtaining both metadata fails, wait 0.5 seconds and try again, try this a max of 3 times and then give up. I don’t know how often I am entering this code (didn’t instrument the app for it), but am still getting times when metadata comes back blank which means this code wasn’t successful after 3 tries.

I assume the file would only become visible and sharable with my app after it has completed being written by the original app/process. Perhaps it hasn’t finalized yet? Is there a way to detect this so I can tell the user in my share screen to wait and try again?

I am assuming that the file has finished writing though since when I read the data from the file contents, it’s good data and complete even when metadata failed.

I will be instrumenting the above code in my next app version, just hoping to fix it right now since users are emailing saying my app is broken.

Thanks!

First off, I need do comment on a pet concern of mine:

     let fileAttributes = try FileManager.default.attributesOfItem(atPath: url.path)
     

String based paths are a long standing problem in our entire API. Their are valid file system configuration that system should be able to work with, but that fail catastrophically because of how string based paths manipulate unicode normalization. I don't know if we will ever be able to fully resolve those issue, but doing so is going to require using NSURL for ALL file references.

In terms of "selfish" reasons, you should be aware that the "path" and "URL" variants of the "same API" can have WILDLY different performance characteristics. For example, "enumeratorAtPath" is ~3x SLOWER than "enumeratorAtURL", due to a combination of different API semantics (it's design forces a additional "stat" call for attributes you may never look at) and because it's not the primary API.

Moving on to the issue itself:

When users share a file with my app I am having trouble 5-10% of the time obtaining the file meta data, specifically creation and modified time and size.

A few questions:

  • Do you know what error it returned?

  • Do you know anything about "where" the file came from? Both in terms of how the user "gave" it to your app and what the underlying file source might have been?

Having said that, I don't know enough about the semantics of your particular app, but this:

I assume the file would only become visible and sharable with my app after it has completed being written by the original app/process. Perhaps it hasn’t finalized yet? Is there a way to detect this so I can tell the user in my share screen to wait and try again?

...is not a valid assumption in the general case. As the simplest example, it's entire possible for the following to occur:

  • An app writes a file to an smb share.

  • The app passes that file to you.

  • At the same moment, the permissions on the item are changed such that it's no longer accessible to "you".

That's a somewhat contrived example, but the reality is that you can't assume the file you receive will always be "valid". File system don't work that way.

Related to that point, what metadata are you actually "failing" this on? And how how are you checking the dates? For example, if you were to assume that any date earlier than 1970 is invalid, then I think there are lots of file "in the wild" that will fail that test. HFS+ starts it's year in 1904 and we shipped a number of laptops that would lose their clock if/when the battery completely died, at which point you'd start getting weirdly "early" files. I think we may have capped the date later than 1904, but I believe it was earlier than 1970. NFTS actually uses 1601 for it's epoch.

I am assuming that the file has finished writing though since when I read the data from the file contents, it’s good data and complete even when metadata failed.

How are you reading the files contents? Unless there's a significant time gap between you metadata call and your data read, I don't think timing is likely to be a factor her. Keep in mind that data doesn't actually have to reach the disk for your app to be able to read it. However, what might make a difference here are the specific APIs used to access the data, particularly if you're using UIDocument or (possibly) NSFileWrapper, both of which implement file coordination.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Hi Kevin,

Thanks for responding! Just realized I forgot to specify this is for iOS and iPadOS. Has happened on multiple versions of iOS 17.

I do not currently know what the error is but am going to add some code that sends my server the error message when it occurs.

The shared files are typically from the Files app, if not there, then at least from the local device, not iCloud or any cloud/network storage.

They arrive as "file://Documents/Inbox/filename", then the file attributes are obtained, then the file is moved to the application support directory. I just realized as I wrote this that there is then a pause as the user has to hit upload before the file is read using

try Data(contentsOf: )

For most users the time delay is minimal, share file, select our app from the list, see our app's share screen and hit upload button immediately, but perhaps this delay has something to do with it. All of this code is synchronous so the move should finish before the user is even shown the button.

I'm checking if creationTime: Date and modificationTime: Date == Date(timeIntervalSince1970: 0), what I set the values to beforehand, to check for failure.

Thanks!

Colin

The shared files are typically from the Files app, if not there, then at least from the local device, not iCloud or any cloud/network storage.

I'm not sure what you mean here. Files.app will let you share from basically "any" of those sources, so how do you know they're NOT from those other possibilities?

Honestly, the ONLY way what you're describing would make any sense is that the files are specifically NOT from the local device (at least not originally). APFS file cloning means that duplicating a file within APFS is about as fast as an operation can be. You're certainly not going to be able to "catch" the file in any weird/intermediate state.

My guess is that what's actually going on here is that the file(s) are actually coming from a some kind of cloud storage provider who's file provider isn't handling things correctly. The provider is supposed to have this kind of metadata immediately "at hand" so that the data is available to apps that are "looking" at the file but haven't actually retrieved the file data. That would also explain why this matters:

I just realized as I wrote this that there is then a pause as the user has to hit upload before the file is read using

try Data(contentsOf: )

What matters here isn't the time, it's that getting the files data will (implicitly) force the file to fully "materialize". Related to that point, let me jump back to here:

Is there a way to detect this so I can tell the user in my share screen to wait and try again?

What makes this question difficult to answer is that how the system handles all of this has changed a lot over time. In the "original" system design, cloud storage basically operated "above" the file system itself, which was why the file coordination system existed- failing to use file coordination could mean that you didn't actually get the data you were looking for. However, over time the implementation has been moved much "lower" in the file system, specifically so that you couldn't "bypass" the cloud layer and accidentally get the wrong data. FYI, we actually documented part of how that works in "TN3150: Getting ready for dataless files".

In any case, I think the simplest solution here would be to read the data first, then try and retrieve the metadata. If you need to get the size first then grab that, but don't bother with the other metadata until after you've grabbed the data itself.

One comment here:

I'm checking if creationTime: Date and modificationTime: Date == Date(timeIntervalSince1970: 0), what I set the values to beforehand, to check for failure.

I'll be honest, this check makes me nervous. It's obviously not a common case, but I think there may be cases where the file system will end up "correctly" returning "0/1970".

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

obtaining file creation &amp; modified time &amp; size failing 5-10% of time within .onOpenURL when file shared with app
 
 
Q