Our use case is that we need to run a CLI script (on Mac) to set up backend configuration before each test case (UI test on real iOS device). The setup logic is test specific, so using pre-build scripts is not an option.
Is this support in any way or what are best practices around such a setup?
Ideally, we would still be able to execute the tests from within Xcode.
Testing
RSS for tagDetect issues like logic failures, UI problems, and performance regressions by running tests on your app.
Posts under Testing tag
106 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
Our use case is that we need to run a CLI script (on Mac) to set up some backend configuration for a UITest that then runs on iOS (simulator or actual device). The setup logic is test specific so using pre-build scripts is not a an option. Is that support in any way or what would be alternatives to go about this.
I’m currently trying to localize one of my Mac apps for Right-to-Left languages.
In Xcode, to test this, I set the language to ‘Arabic’ in the scheme, which then displays the language and mirrors the layout accordingly.
I have two questions about this:
macOS titlebar buttons: In the title bar at the top, shouldn’t the Close and Minimize buttons also be on the right side (mirrored)? How can I test this?
Numerals: My UI still displays Latin numerals (123). I know this is correct in some Arabic-speaking regions, but I’d also like to test cases where Arabic-Indic numerals (١٢٣) are used. Is there a way to configure the scheme to test this?
Hello,
I am currently developing an iOS application that leverages FinanceKit to access transaction history. I have already enabled FinanceKit in my developer account, but I am encountering challenges when trying to test the transaction history feature.
Could anyone guide me on the best practices or specific steps needed to effectively test FinanceKit’s transaction history in a development environment? Any insights on mock data handling, API responses, or potential limitations during testing would be greatly appreciated.
Thank you in advance for your help!
How I can I have both versions of the app installed on the same phone? 1 for testing and 1 that’s live to see current version users are using?
I had this setup before but now TestFlight installs replace the live version instead of the test version.
Requirement :
We wanted to run UI Automation using xcuitest in devices with MDM profiles.
Steps :
We have created a test target for our app and signed with MDM profile (in-house)
Generated app and testbundle-Runner along with xctestrun files using “xcodebuild build-for-testing -scheme “App-TestRelease" -configuration “Release” -derivedDataPath "${CURDIR}/derivedData" -workspace "${PROJECT_NAME}.xcworkspace" -sdk iphoneos”
Tried running “xcodebuild test-without-building -destination platform=iOS,id=000842883-**** -xctestrun App-TestRelease_iphoneos17.5-arm64.xctestrun -derivedDataPath /derived/“
Facing below issues
xcodebuild[35481:3588352] IDELaunchReport: e2934a094f300d40:e2934a094f300d40: Finished with error: Unable to launch com.org.mdm.XCTest.xctrunner
Domain: com.apple.platform.iphoneos
Code: -12
User Info: {
IDERunOperationFailingWorker = IDELaunchiPhoneLauncher;
}
--
Request to launch com.org.mdm.XCTest.xctrunner failed.
Domain: com.apple.dt.deviceprocesscontrolservice
Code: 2
Failure Reason: The request to open "com.org.mdm.XCTest.xctrunner" failed. : Failed to launch process with bundle identifier 'com.org.mdm.XCTest.xctrunner'.
--
The request to open "com.org.mdm.XCTest.xctrunner" failed.
Domain: FBSOpenApplicationServiceErrorDomain
Code: 1
Failure Reason: The request was denied by service delegate (SBMainWorkspace) for reason: Security ("Unable to launch com.org.mdm.XCTest.xctrunner because it has an invalid code signature, inadequate entitlements or its profile has not been explicitly trusted by the user").
User Info: {
BSErrorCodeDescription = RequestDenied;
FBSOpenApplicationRequestID = 0x9f9;
}
--
The operation couldn’t be completed. Unable to launch com.org.mdm.XCTest.xctrunner because it has an invalid code signature, inadequate entitlements or its profile has not been explicitly trusted by the user.
Domain: FBSOpenApplicationErrorDomain
Code: 3
Failure Reason: Unable to launch com.org.mdm.XCTest.xctrunner because it has an invalid code signature, inadequate entitlements or its profile has not been explicitly trusted by the user.
User Info: {
BSErrorCodeDescription = Security;
}
I need to read results of a test in Xcode post-action of Test.
So the script that runs after tests contains
SUMMARY=$(xcrun xcresulttool get test-results summary --path $RESULT_FILE 2>> "$LOGS")
If I run my script from terminal, everything is ok but if I launch tests and the post-action is executed, I got
Warning: unknown environment variable SWIFT_DEBUG_INFORMATION_FORMAT
Warning: unknown environment variable SWIFT_DEBUG_INFORMATION_VERSION
Warning: unknown environment variable SWIFT_DEBUG_INFORMATION_FORMAT
Warning: unknown environment variable SWIFT_DEBUG_INFORMATION_VERSION
Error: Failed to create a new result bundle reader, underlying error: failed to read metadata with underlying error (type: FileSystemError: 3 - [:] - The operation couldn’t be completed. (MinimalTSCBasic.FileSystemError error 3.)
Usage: xcresulttool <subcommand>
See 'xcresulttool --help' for more information.
It might be that the environment set by Xcode is missing something but I cannot figure out what, any idea?
Hello everyone,
I'm working on an iOS application using Objective-C and UITabBarController. My app has more than 5 tabs, so the additional tabs are placed under the "More" tab. However, I've encountered an issue specific to iOS 18 where the first item in the "More" tab does not show up properly. This issue does not occur in iOS 17 or earlier versions.
Here's my setup method:
(void)mainTabbarSetUp {
NSMutableArray *tabItemArray = [NSMutableArray array];
UIViewController *viewController1, *viewController2, *viewController3, *viewController4, *viewController5, *viewController6, *viewController7;
UINavigationController *navviewController1, *navviewController2, *navviewController3, *navviewController4, *navviewController5, *navviewController6, *navviewController7;
viewController1 = [[UIViewController alloc] init];
navviewController1 = [[UINavigationController alloc] initWithRootViewController:viewController1];
navviewController1.tabBarItem.title = @"Watch List";
navviewController1.tabBarItem.image = [UIImage imageNamed:@"tab_icn_watchlist"];
[tabItemArray addObject:navviewController1];
// Similarly adding other view controllers...
viewController6 = [[UIViewController alloc] init];
navviewController6 = [[UINavigationController alloc] initWithRootViewController:viewController6];
navviewController6.tabBarItem.title = @"Cancelled";
navviewController6.tabBarItem.image = [UIImage imageNamed:@"tab_icn_cancelled"];
[tabItemArray addObject:navviewController6];
self.mainTabBarController.viewControllers = tabItemArray;
}
What I've Tried:
Verified that each view controller is correctly initialized and assigned to a UINavigationController before being added to the tab array.
Logged the contents of the moreNavigationController to confirm that it contains the correct view controllers.
Tested by reducing the number of view controllers to less than 5, and the issue does not occur.
Ensured that all UINavigationControllers are configured consistently (e.g., translucency, bar style, etc.).
No coverage is generated for private xxxx.framework, but tests are running under this framework.
This was perfectly working before and started giving this error after updating to Xcode 16.
Failed to generate coverage for target xxxx.framework at paths (
"/Users/xxxxxx/Desktop/xcode/Debug-iphonesimulator/Common/xxxxxx.framework/xxxx"
):
No object file for requested architecture
The operation couldn’t be completed. (XCTHCoverageReportGenerationErrorDomain error 2.).
(Underlying Error: No object file for requested architecture)
For my app I was trying to write some tests to ensure the functionality of all features. As I am using Xcode 16.0 I thought I might use Swift testing which was newly introduced and replaces XCTest.
I created a new test target with Swift Testing and tried to run the first test, which was created automatically by the system.
struct FinancialTests {
@Test func testExample() async throws {
#expect(true)
}
}
Xcode is also showing the test diamond next to the function so I clicked on it to execute it. The app started to build and the build ended successfully. The the next step was testing. And after waiting for 10 minutes or so, no test was executed. First I thought maybe the test was not found, but in the test case overview all tests were shown:
The run only shows this:
Can someone help me to get this running.
Many thanks!
According to Technical Note TN2083 the Window Server advertises itself in the global bootstrap namespace, which is why you can launch GUI applications from SSH sessions, even if sshd/sshd-keygen-wrapper is launched as a launch daemon (in a non-GUI per-session bootstrap namespace).
As I understand it this is also why SessionGetInfo() reports NO for sessionHasGraphicAccess, as the SSH session is not an Aqua session type, while CGSessionCopyCurrentDictionary() does return a valid dict, because in practice you have access to the window server.
However, the tech note advices against running GUI programs from SSH sessions, as other GUI services may not be exposed to the global or non-GUI per-session bootstrap namespace. It uses com.apple.dock.server as an example of such a service, showing how Activity Monitor has different behavior when launched via SSH than via the UI.
Based on the advice of the tech note, articles like https://aahlenst.dev/blog/accessing-the-macos-gui-in-automation-contexts/ recommends running CI UI tests via a Launch Agent instead of SSH.
Now, I've tried to reproduce the the Activity Monitor case on macOS 12 and macOS 15, and I can not reproduce the missing Dock features. The Testing with Xcode documentation also says that:
By default, when you use ssh to login to an macOS system that has no active user session running, a command-line session is created. To ensure that an Aqua session is created for an ssh login, you must have a user logged in on the remote macOS host system. The existence of a user running on the remote system forces Aqua session for the ssh login. Once there is a user running on the host system, running xcodebuild from an ssh login works for all types of tests.
Which begs the question: Does modern macOS versions expose GUI services to the global or non-GUI per-session bootstrap namespace, or otherwise enable UI testing from SSH sessions, so that UI tests can safely be run from SSH sessions (as long as the user is logged in to the remote system's UI). Has things changed in this regard?
I am trying to browse an SSH Service from UI Test Target using NWBrowser
let descriptor = NWBrowser.Descriptor.bonjour(type: "_superapp._tcp", domain: "local.")
let browser = NWBrowser(for: descriptor, using: .tcp)
browser.stateUpdateHandler = { newState in
print("browser.stateUpdateHandler \(newState)")
}
I get failed(-65555: NoAuth) error
nw_browser_fail_on_dns_error_locked [B1] DNSServiceBrowse failed: NoAuth(-65555)
I have added _superapp._tcp in Bonjour Services for UI Test Target Info.plist.
I have also added Local Network Permission in UI Test Target Info.plist.
Everything works fine, when I call this Bonjour service from App Target.
Doesn't work when I call this from UI Test Target.
In the latest version of Xcode, breakpoints are not displaying correctly, making it difficult to track code behavior. Previously, Xcode allowed you to step through each function, showing clear execution steps and allowing interaction to debug effectively. However, now it only shows memory addresses without providing meaningful insights, limiting the ability to interact with or test the code as it's being developed. Is anyone else experiencing this issue, or does anyone know of a workaround?
I test my app, by building it and hoping it will go on my iOS. This is the only workflow I have to test my testing app.
I use AVAudioEngine and AVAudioSession. It sounds painful but, yes. I literally comment out parts and build the app everytime and see if it would crash or not.
Sometimes I'd get Crash Logs in the Analytics part, but I haven't managed to get them there anymore. So I had no more Crash Logs.
I was wondering if there's a function or something. Maybe something that I can do in AppDelegate to make it create a Crash Log somewhere?
Uncaught Exception overwrite, didn't solve the issue for me. It really has to be an actual crash log, something that catches these.
If someone knows, let me know! Thanks
I am new. I have never used Swift nor XCode or the CLI. But I have found and managed to test my App that I created for testing on my iPhone.
I have absolutely no intentions to put it on the App Store. I even made it Open-Source on GitHub.
I learned a lot of things and ways to prevent crashes, but unfortunately I am slowly starting to lose the ability to put my App on my iOS.
I am new here as a developer. But I have troubles to create a Developer Account, so I am not sure what I should do.
Here's what I think I am having issues with:
I feel like you can only create a Developer Account if you have a business, is this true?
I really find it cool to test my own App on my own phone. But I want to do it in an official way. But I don't know how.
Others have told me that you'd need a License for the App Store. But I am not trying to put it on the App Store, will this make it impossible for me to test my own app?
Why make an app if not putting it on the App Store?
You can learn. If you make good Apps or Libraries, you can provide them to other Developers! Maybe even an opportunity to collab with someone and even indeed put it on the App Store with a valid Developer Account.
Seriously, if I wouldn't have been able to test my iOS app through other ways on my iPhone, I'd have never been able to make a short breakdown about AVAudioEngine and AVAudioSession. I've seen people that have run into these crashes. I tried to look up for a fix, but found none.
I saw apps that allowed you to use a Microphone on a Laptop/PC and I decided to make my own one to test, and succeded, while I yet need to figure out a few crashes, I managed to do it, and I really want to continue and actually use my own App...
So I was really happy that I could make a post to provide a guide in hopes that it would help someone. And I would do more, but I really need help with figuring out how I can test my own app on my own phone.
I hope that Developer Relations can help with this.
Hi,
I work on a iOS application which includes several local swift packages containing modularised code. Each of those local swift packages has a test target defined in their respective Package.swift, and when opening a local package folder standalone in Xcode, then the tests run without issues.
However I would like in the main app's test plan to add the local package test targets and have them all run together when the main app's default test plan is run. Xcode allows me to select local package test targets within the main app's test plan (also within Test portion of the main app's scheme), however attempting to build for testing throws a "Module '…' was not compiled for testing".
Any ideas on how to achieve this goal (run local package tests in conjunction with main app's tests) and avoid that error?
Thanks
Peter
Firefox and Chrome provide various options and services to set up browser characteristics while testing. These can be used with Selenium while setting up the driver.
An example with Firefox is setting a custom user agent and a download directory different than the default. Using Python:
from selenium.webdriver.firefox.options import \
Options as FirefoxOptions
from selenium.webdriver.firefox.service import \
Service as FirefoxService
firefox_options = FirefoxOptions()
# set up user agent
user_agent = UserAgent().get_user_agent()
firefox_options.set_preference("general.useragent.override", user_agent)
# set alternate download location
firefox_options.set_preference('browser.download.folderList', 2)
firefox_options.set_preference('browser.download.dir', download_dir_path)
Many of these browser-specific options and services are listed at Selenium's website under Supported Browsers.
However, Safari's information is very limited. On Selenium's site, for example, they list how to turn on Safari's limited logging: service = webdriver.SafariService(service_args=["--diagnose"]). Then, they point us to About WebDriver for Safari - which shows automaticInspection and automaticProfiling. The page primarily describes security features for testing with Safari.
In a regular user session, one can configure an alternate download directory, set a custom profile with specific settings for that profile, and so on. And, in the Developer Menu I can set some of these items manually for a given session (user agent, for example).
How can I access these features within Safari for use in automated web testing with Selenium?
Edit: Alternatively, does Apple have, or recommend, a different automated testing package for Safari?
I am using two iPhone11 devices, having iOS v17.6.1 for automation testing using Seleniumv4.X-Appium v2.x.
Today I updated the mac to Sequoia15 version which updated the XCode to v16.
Just after this update, I could not able to run the previously running automation script. The console message is "org.openqa.selenium.SessionNotCreatedException:
Could not start a new session. Response code 500. Message: Unable to launch WebDriverAgent. Original error: xcodebuild failed with code 70. This usually indicates an issue with the local Xcode setup or WebDriverAgent project configuration or the driver-to-platform version mismatch."
I tried to use XCTest to test my own project that uses EndpointSecurity, but when I created the esClient I got an error:ES_NEW_CLIENT_RESULT_ERR_NOT_PRIVILEGED, indicating that it was not root.
This makes it impossible for me to do coverage tests for the ESClient application. Is there any way I can implement this ESClient test? If so, how should I use it? The project is a macOS program, if I use gcov, but I find I can't get coverage. Using __gcov_flush will indicate that there is no symbol
#if !TARGET_IPHONE_SIMULATOR
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
setenv("GCOV_PREFIX", [documentsDirectory cStringUsingEncoding:NSUTF8StringEncoding], 1);
setenv("GCOV_PREFIX_STRIP", "13", 1);
#endif
extern void __gcov_flush(void);
__gcov_flush();
#endif
I am creating a iOS library project using Xcode 15 in objective C lang which has dependency XCTest framework and Objective C run time and having UI test cases which can be use used in various projects.
I am able to build and run unit test cases using Xcode fine but when try to run test using command line getting below issue
Command
env NSUnbufferedIO=YES xcodebuild clean test -workspace EnptyLibrary.xcworkspace -scheme EnptyLibrary -configuration Debug -destination 'platform=iOS Simulator,OS=latest,name=iPhone 15 Pro' -destination 'platform=iOS Simulator,OS=latest,name=iPad Pro (12.9-inch) (6th generation)' -testPlan EnptyLibrary -derivedDataPath build -disable-concurrent-destination-testing -retry-tests-on-failure=NO
Error log
** TEST SUCCEEDED **
2024-09-12 16:36:34.822 xcodebuild[8345:45388802] Writing error result bundle to /var/folders/12/rcxtcvjs08169cc_fzhyc8x40000gq/T/ResultBundle_2024-12-09_16-36-0034.xcresult
2024-09-12 16:36:34.832 xcodebuild[8345:45388802] [MT] DVTAssertions: ASSERTION FAILURE in IDEFrameworks/IDEFoundation/ProjectModel/ActionRecords/IDESchemeActionsInvocationRecord.m:458
Details: Assertion failed: !self.hasSaved
Object: <IDESchemeActionsInvocationRecord: 0x112c21ce0>
Method: -saveWithCompletionBlock:
Thread: <_NSMainThread: 0x600000e441c0>{number = 1, name = main}
Hints:
Backtrace:
0 -[DVTAssertionHandler handleFailureInMethod:object:fileName:lineNumber:assertionSignature:messageFormat:arguments:] (in DVTFoundation)
1 _DVTAssertionHandler (in DVTFoundation)
2 _DVTAssertionFailureHandler (in DVTFoundation)
3 _DVTAssertionWarningHandler (in DVTFoundation)
4 -[IDESchemeActionsInvocationRecord saveWithCompletionBlock:] (in IDEFoundation)
5 -[IDESchemeActionsInvocationRecord saveSynchronouslyWithError:] (in IDEFoundation)
6 -[Xcode3CommandLineBuildTool saveResultBundleForError:triageWithSchemeTask:error:] (in Xcode3Core)
7 -[Xcode3CommandLineBuildTool _printError:schemeTask:andFailWithCode:] (in Xcode3Core)
8 -[Xcode3CommandLineBuildTool _printErrorWithSchemeTaskUserInfo:] (in Xcode3Core)
9 -[Xcode3CommandLineBuildTool _buildWithTimingSection:] (in Xcode3Core)
10 -[Xcode3CommandLineBuildTool run] (in Xcode3Core)
11 XcodeBuildMain (in libxcodebuildLoader.dylib)
12 -[XcodebuildPreIDEHandler loadXcode3ProjectSupportAndRunXcode3CommandLineBuildToolWithArguments:] (in xcodebuild)
13 -[XcodebuildPreIDEHandler runWithArguments:] (in xcodebuild)
14 main (in xcodebuild)
15 start (in dyld)
zsh: abort env NSUnbufferedIO=YES xcodebuild clean test -workspace -scheme EnptyLibrary