How can I copy files inside SMB volumes as fast as the Finder?

The copyfile() method of the macOS API does not seem to use SMB server-side copying, while copying in Finder does appear to use this.

Answered by DTS Engineer in 795660022

The copyfile() method of the macOS API does not seem to use SMB server-side copying, while copying in Finder does appear to use this.

First off, please file a bug on this and then post the bug number back here.

The only API option I've found that will do this are the long deprecated Carbon File Manager copy APIs. The Carbon File Manager was originally implemented using the same support library the Finder uses and, while the Finder's overall implementation has changed considerably, this particular case is still going through exactly the same code*.

*For the curious, "server side copying" was a LONG standing feature (pre-Mac OS X) of AFP (Apple Filing Protocol). VFS Support for server side copying was originally implemented for AFP and was otherwise unused. MANY years later, SMB then implemented server side copying using the same VFS hook AFP implemented... at which point the Finder (and Carbon File Manager) suddenly support server side copying on SMB.

The Carbon File Manager API can look pretty strange to modern eyes, so here is what that copy looks like in code:

#import <CoreServices/CoreServices.h>
#import <Foundation/Foundation.h>

bool CopyFileCarbon(NSURL* source, NSURL* destFolder, NSString* newfileName)
{
    bool retVal = false;
    OSStatus err = noErr;
    FSRef file1Ref;
    BOOL isDir;
    err = FSPathMakeRef(source.path.fileSystemRepresentation, &file1Ref, &isDir);
    if(err == noErr)
    {
        FSRef file2Ref;
        err = FSPathMakeRef(destFolder.path.fileSystemRepresentation, &file2Ref, &isDir);
        if(err == noErr)
        {
            FSRef newFileRefl;
            err = FSCopyObjectSync(&file1Ref, &file2Ref,(__bridge CFStringRef) newfileName, &newFileRefl, 0);
            if(err == noErr)
            {
                retVal = true;
            }
        }
    }

    return retVal;
}

A few notes on this:

  • Overall, the "FSCopy*" APIs are NOT a good alternative to copyfile (or NSFileManager). Those other APIs are significantly faster for more "general" copying and there may be other edge cases where they don't do the "right" thing. For example, I don't know how well they handle APFS->APFS copies that involve complex cloned file collections.

  • Under normal circumstances I would never recommend using a long deprecated API like this, however, I understand that the performance benefits here can be large enough to justify unusual API usage. However, if you're going to use this API, you need to ensure you're specifically detecting when server side copying would apply and only using it there.

  • Server side copying is not universally supported. You can check if a particularl file system supports it by looking for "VOL_CAP_INT_COPYFILE" from "getattrlist".

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

The copyfile() method of the macOS API does not seem to use SMB server-side copying, while copying in Finder does appear to use this.

First off, please file a bug on this and then post the bug number back here.

The only API option I've found that will do this are the long deprecated Carbon File Manager copy APIs. The Carbon File Manager was originally implemented using the same support library the Finder uses and, while the Finder's overall implementation has changed considerably, this particular case is still going through exactly the same code*.

*For the curious, "server side copying" was a LONG standing feature (pre-Mac OS X) of AFP (Apple Filing Protocol). VFS Support for server side copying was originally implemented for AFP and was otherwise unused. MANY years later, SMB then implemented server side copying using the same VFS hook AFP implemented... at which point the Finder (and Carbon File Manager) suddenly support server side copying on SMB.

The Carbon File Manager API can look pretty strange to modern eyes, so here is what that copy looks like in code:

#import <CoreServices/CoreServices.h>
#import <Foundation/Foundation.h>

bool CopyFileCarbon(NSURL* source, NSURL* destFolder, NSString* newfileName)
{
    bool retVal = false;
    OSStatus err = noErr;
    FSRef file1Ref;
    BOOL isDir;
    err = FSPathMakeRef(source.path.fileSystemRepresentation, &file1Ref, &isDir);
    if(err == noErr)
    {
        FSRef file2Ref;
        err = FSPathMakeRef(destFolder.path.fileSystemRepresentation, &file2Ref, &isDir);
        if(err == noErr)
        {
            FSRef newFileRefl;
            err = FSCopyObjectSync(&file1Ref, &file2Ref,(__bridge CFStringRef) newfileName, &newFileRefl, 0);
            if(err == noErr)
            {
                retVal = true;
            }
        }
    }

    return retVal;
}

A few notes on this:

  • Overall, the "FSCopy*" APIs are NOT a good alternative to copyfile (or NSFileManager). Those other APIs are significantly faster for more "general" copying and there may be other edge cases where they don't do the "right" thing. For example, I don't know how well they handle APFS->APFS copies that involve complex cloned file collections.

  • Under normal circumstances I would never recommend using a long deprecated API like this, however, I understand that the performance benefits here can be large enough to justify unusual API usage. However, if you're going to use this API, you need to ensure you're specifically detecting when server side copying would apply and only using it there.

  • Server side copying is not universally supported. You can check if a particularl file system supports it by looking for "VOL_CAP_INT_COPYFILE" from "getattrlist".

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks for the write up! The bug report for this issue is FB13783472 .

How can I copy files inside SMB volumes as fast as the Finder?
 
 
Q