{"wwdc2024-10179":{"language":"eng","transcript":[[7,"Hi! I’m Stuart Montgomery, "],[8,"and I’m thrilled to introduce Swift Testing to you. "],[11,"Quality and reliability are crucial for delivering a great user experience. "],[15,"Automated testing is a proven way "],[17,"to achieve and maintain software quality over time. "],[21,"That’s why this year we’re introducing a brand new set of tools "],[24,"which make testing Swift code easier and more powerful than ever. "],[27,"Meet Swift Testing, a new open source package for testing your code using Swift. "],[33,"It includes powerful capabilities for describing and organizing your tests; "],[37,"it gives actionable details when a failure occurs; "],[39,"and it scales to large codebases elegantly. "],[43,"Swift Testing was designed for Swift, "],[45,"embracing modern features like concurrency and macros. "],[48,"It supports all major platforms, including Linux and Windows. "],[52,"And it has an open development process, "],[54,"providing opportunities for you and the rest of the community "],[57,"to shape its evolution. "],[59,"In this session we’ll start by talking about "],[61,"the building blocks of Swift Testing, the core concepts you need to know. "],[65,"Then, we’ll discuss several common workflows, "],[67,"including ways to customize tests or repeat them with different arguments. "],[72,"We’ll cover how Swift Testing and XCTest relate to each other. "],[75,"And we’ll finish by talking about "],[77,"this new project’s role in the open source community. "],[80,"Let’s get started by taking a tour of the building blocks of Swift Testing. "],[84,"If you’ve never written tests for your app before, "],[86,"the first step is to add a test bundle target to your project. "],[89,"Choose File \u003e New \u003e Target.\n\n"],[96,"Then search for Unit Testing Bundle in the Test section.\n\n"],[103,"Swift Testing is now the default choice of testing system "],[105,"for this template in Xcode 16. "],[107,"Just choose a name for your new target and click Finish. "],[110,"This app already has a test target though, so let’s write our first test there. "],[114,"We'll start by importing the Testing module.\n\n"],[119,"Then, we’ll write a global function.\n\n"],[123,"And we’ll add the @Test attribute to it.\n\n"],[128,"The @Test attribute is the first building block. "],[131,"It indicates that a function is a test. "],[133,"Once we add that, Xcode recognizes it and shows a Run button alongside it. "],[138,"Test functions are just ordinary Swift functions "],[140,"that have the @Test attribute. "],[142,"They can be global functions or methods in a type. "],[145,"They can be marked async or throws, "],[147,"or be isolated to a global actor if needed. "],[150,"Next, let's make our test actually validate something "],[153,"by filling out the body of the function. "],[155,"We’ll make this test check that "],[156,"the metadata for a video file are what we expect. "],[159,"We’ll start by initializing the video we want to check "],[162,"and its expected metadata. "],[164,"Now, we’re getting an error because these types are declared this app's module "],[169,"so we need to import that first.\n\n"],[174,"Note that we use the lowercase @testable attribute on this import.\n\n"],[179,"This is a general language feature, not part of Swift Testing, "],[182,"but it allows us to reference types like these "],[184,"whose access level is internal. "],[187,"Next, we'll use the #expect macro to check that the video metadata are correct.\n\n"],[198,"The #expect macro performs an expectation, "],[201,"and this is the second building block of Swift Testing. "],[204,"You can use an expectation like the #expect macro "],[206,"to validate that an expected condition is true. "],[209,"It accepts ordinary expressions and language operators. "],[212,"And it captures the source code "],[213,"and the values of subexpressions if it fails. "],[216,"Let's run our test for the first time and see how it goes.\n\n"],[226,"It looks like it failed, indicated by the red X icon. "],[230,"We can click the test failure message "],[232,"and choose Show to see more about what went wrong on this line.\n\n"],[236,"This results view shows details "],[237,"about the expression that was passed to the #expect macro, "],[240,"including its sub-values. "],[242,"If we expand the metadata, we can compare their properties.\n\n"],[247,"It looks like both the duration and resolution fields are non-equal. "],[251,"Looking at this gives me an idea: "],[253,"I think the Video type "],[254,"might not be loading the metadata after it's initialized. "],[258,"We can fix this by going to the Video initializer in a split editor, "],[261,"and ensure that property is assigned.\n\n"],[273,"Now, let's re-run the test.\n\n"],[281,"And it succeeded! Great. "],[284,"The #expect macro is really flexible. "],[286,"You can pass any expression, including operators or method calls, "],[289,"and it will show you detailed results if it fails. "],[292,"Here are a few examples. "],[294,"You can use the == operator, "],[296,"and the left and right hand sides will be captured and shown if there’s a failure. "],[299,"You can access properties like .isEmpty. "],[302,"And you can even call methods like .contains on an array. "],[305,"Notice how the error shows you "],[307,"the contents of the numbers array automatically. "],[310,"You don’t need to learn specialized APIs to do any of this — "],[313,"just use the #expect macro. "],[316,"Sometimes, you may want to end a test early if an expectation fails. "],[320,"For this, you can use the #require macro. "],[322,"Required expectations are similar to regular expectations. "],[325,"But they have the try keyword and throw an error if the expression is false, "],[329,"causing the test to fail and not proceed any further. "],[333,"Another way you can use the #require macro "],[335,"is to try unwrapping an optional value safely, "],[337,"and stop the test if it’s nil. "],[340,"This example shows using the #require macro "],[342,"to access the .first property of a collection, "],[345,"and afterward it checks a property on the element. "],[348,"The “first” property is optional, "],[350,"but the second line of our test relies on that value, "],[353,"so this test stops early "],[354,"because it doesn’t make sense to continue if the unwrapped value is nil. "],[358,"Required expectations are a great tool for this pattern. "],[362,"Before we commit this test to the project, let’s make its purpose more clear. "],[366,"We can do that by passing a custom display name in the @Test attribute. "],[370,"That name will then be shown in the Test Navigator "],[372,"and other places in Xcode.\n\n"],[375,"A display name is an example of a trait, which is the third building block. "],[380,"Traits can do several things: "],[381,"they can add descriptive information about a test; "],[384,"they can customize when or whether a test runs; "],[387,"or they can modify how a test behaves.\n\n"],[390,"Here are some examples. "],[391,"In addition to adding information with the display name, "],[394,"you can also reference related bugs or add custom tags. "],[398,"When you only want to run a test in certain conditions, "],[400,"you can use traits to control that. "],[402,"And some traits influence how a test actually behaves, "],[405,"such as imposing a time limit or executing one at a time. "],[409,"We've finished writing our first test, "],[411,"and now let’s add a second one "],[412,"to validate another aspect of the Video type. "],[415,"This time, let’s use the built-in test snippet in Xcode 16 "],[419,"to quickly add an empty test function.\n\n"],[422,"Let’s name this test rating.\n\n"],[426,"And in the body, we'll create a video just like before, "],[428,"and #expect that its contentRating is the default value. "],[432,"It would be nice to group these two tests together "],[435,"so we can find them more easily in the project. "],[437,"We can do that by wrapping them into a struct, "],[439,"which we’ll call VideoTests.\n\n"],[445,"As soon as we do that, the hierarchy is reflected in the Test Navigator, "],[449,"and we can even click to run them as a group. "],[451,"A type like this which contains tests is called a test suite, "],[455,"and that's the fourth and final building block. "],[458,"Suites are used to group related test functions or other suites. "],[462,"They can be annotated explicitly using the @Suite attribute, "],[465,"although any type which contains @Test functions or @Suites "],[467,"is considered a @Suite itself, implicitly. "],[470,"Suites can have stored instance properties. "],[473,"They can use init or deinit to perform logic before or after each test. "],[477,"And a separate @Suite instance "],[479,"is created for every instance @Test function it contains "],[481,"to avoid unintentional state sharing. "],[484,"These two tests start the same way: "],[487,"their first lines of code for creating a video are identical. "],[491,"Now that these tests are in a suite, we can reduce repetition "],[494,"by factoring out that line into a stored property like this.\n\n"],[500,"And now we can delete that line from the second test.\n\n"],[504,"Since each test function is called on "],[506,"a new instance of its containing suite type, "],[509,"each one will get its own video instance "],[511,"and they can never share state accidentally. "],[514,"So let’s review the building blocks. "],[517,"We talked about test functions, expectations, traits, and suites. "],[521,"They were designed to feel right at home in Swift, in several ways: "],[526,"Test functions integrate seamlessly with Swift concurrency "],[529,"by supporting async/await and actor isolation. "],[532,"Expectations can use async/await too, "],[535,"and they accept all the built-in language operators. "],[538,"Both expectations and traits leverage Swift macros, "],[542,"allowing you to see detailed failure results "],[544,"and specify per-test information directly in code. "],[548,"And suites embrace value semantics, "],[550,"encouraging the use of structs to isolate state.\n\n"],[554,"Let’s now apply those building blocks to some common problems in testing "],[557,"and discuss workflows for addressing them. "],[560,"We’ll discuss controlling when tests run; "],[563,"associating tests which have things in common; "],[565,"and repeating tests more than once with different arguments each time. "],[569,"First, tests with conditions. "],[572,"Some tests should only be run in certain circumstances — "],[575,"such as on specific devices or environments. "],[578,"For those, you can apply a condition trait such as .enabled(if: ...). "],[582,"You pass it a condition to be evaluated before the test runs, "],[585,"and if the condition is false, the test will be marked as skipped. "],[589,"Other times, you might want a test to never run. "],[592,"For this, you can use the .disabled(...) trait. "],[595,"Disabling a test is preferable over other techniques, "],[598,"like commenting out the test function, "],[599,"since it verifies the code inside the test still compiled. "],[603,"The .disabled(...) trait accepts a comment, "],[605,"which you can use to explain the reason why the test is disabled. "],[609,"And comments always appear in the structured results, "],[611,"so they can be shown in your CI system for visibility. "],[615,"Oftentimes, the reason a test is disabled "],[617,"is because of an issue which is tracked in a bug-tracking system. "],[621,"In addition to a comment, "],[622,"you can include a .bug(...) trait along with any other trait "],[625,"to reference related issues with a URL. "],[628,"Then, you can see that bug trait in the Test Report in Xcode 16 "],[631,"and click to open its URL.\n\n"],[634,"When the entire body of a test can only run on certain OS versions, "],[637,"you can place the @available(...) attribute on that test "],[639,"to control which versions it will run on. "],[642,"Use the @available(...) attribute rather than checking at runtime "],[645,"using #available. "],[646,"The @available(...) attribute allows the testing library "],[649,"to know that a test has an OS version condition, "],[651,"so this can be reflected in the results more accurately.\n\n"],[656,"Next, let’s talk about how you can associate tests "],[658,"which have characteristics in common, "],[660,"even if they’re in different suites or files. "],[663,"Swift Testing supports assigning custom tags to tests. "],[666,"I've already begun using tags in this project. "],[669,"The Test Navigator shows all the tags at the bottom. "],[672,"To view the tests which each of these tags have been applied to, "],[675,"we can switch to the new Group By: Tag mode.\n\n"],[680,"Let’s apply a tag to one of the tests we wrote earlier. "],[683,"To do this, we’ll add a tags trait to the test via the @Test attribute. "],[688,"This test is validating some data formatting logic. "],[691,"There’s already another test in this project "],[693,"which relates to formatting, "],[696,"so let’s add the formatting tag to this test too.\n\n"],[700,"Once we do that, it shows in the Test Navigator under that tag.\n\n"],[704,"I wrote another test which also validates data formatting, "],[707,"which I’ll add here.\n\n"],[712,"Since these two tests are about the formatting of Video information, "],[715,"let’s group them into a sub-suite.\n\n"],[723,"Now, we can move the formatting tag up to the @Suite, "],[725,"so it will be inherited by all the tests it contains. "],[729,"Finally, we can delete the tags from each individual @Test function, "],[732,"since they’re now inherited.\n\n"],[735,"You can associate tags with tests which have things in common. "],[738,"As an example, you might apply a common tag to all the tests "],[741,"which validate a particular feature or subsystem. "],[744,"This lets you run all the tests with a particular tag. "],[747,"It also lets you filter them in the Test Report, "],[750,"and even see insights there "],[751,"such as when multiple tests with the same tag begin failing. "],[756,"Tags themselves can be applied to tests in different files, suites, or targets. "],[760,"They can even be shared among multiple projects.\n\n"],[764,"When using Swift Testing, prefer tags over specific names of tests "],[768,"when including or excluding them from a test plan. "],[772,"For best results, "],[773,"always use the most appropriate type of trait for each circumstance. "],[777,"Not every scenario should use tags. "],[779,"For example, if you’re trying to express a runtime condition, "],[782,"use .enabled(if ...) as we discussed earlier.\n\n"],[786,"To learn more about using test tags in Xcode, "],[788,"see \"Go further with Swift Testing\".\n\n"],[792,"The last workflow I’d like to show is my favorite. "],[795,"Repeating tests with different arguments each time. "],[798,"Here's an example of why that can be useful. "],[800,"In this project there are several tests which check "],[802,"the number of continents that various videos mention. "],[805,"Each of them follows a similar pattern: "],[807,"it creates a fresh videoLibrary, "],[812,"looks up a video by name, "],[814,"and then uses the #expect macro to check how many continents it mentions.\n\n"],[819,"These tests work, but they're very repetitive "],[822,"and the more videos we add a test for, "],[824,"the harder it will be to maintain them due to all the duplicated code. "],[828,"Also, when using this pattern "],[830,"we’re forced to give each test a unique function name, "],[833,"but these names are hard to read "],[835,"and they might get out-of-sync with the name of the video they're testing. "],[839,"Instead, we can write all of these as a single test "],[841,"using a feature called parameterized testing. "],[844,"Let’s transform this first test into a parameterized one. "],[848,"The first step is to add a parameter to its signature.\n\n"],[854,"As soon as we do this, an error appears "],[856,"telling us that we must specify the arguments to pass to this test, "],[859,"so let’s fix that.\n\n"],[863,"For now, let’s include the names of just three videos.\n\n"],[869,"I like to split arguments over multiple lines "],[871,"so they're easier to read, "],[872,"but you can format them however you like. "],[875,"The last step is to replace the name of the video being looked up "],[878,"with the passed-in argument.\n\n"],[883,"Since this test now covers multiple videos, "],[886,"let’s generalize its name.\n\n"],[891,"The full name of this test now includes its parameter label.\n\n"],[896,"But we can still give it a display name or other traits if we want, "],[899,"by passing them before the arguments.\n\n"],[904,"Now let's run the test and see how it goes.\n\n"],[913,"Great! It succeeded, and the Test Navigator "],[915,"shows each individual video below it as if it were a separate test. "],[919,"This structure makes it really easy to add more arguments and expand test coverage. "],[923,"Let’s add all the remaining videos to this list — "],[926,"and even a couple new ones.\n\n"],[932,"At this point, we can delete the old @Test functions "],[934,"since they're no longer necessary.\n\n"],[940,"Let's run the test one more time and make sure it's still passing.\n\n"],[946,"Hm, it looks like one of the new videos we added near the end "],[949,"is causing a failure now. "],[951,"By clicking the argument, we can see details about it, "],[953,"and the expectation which failed. "],[956,"To investigate this problem, "],[958,"it would help to re-run it with the debugger, "],[960,"but I'd prefer to only re-run the argument that failed to save some time. "],[964,"In Xcode 16 we can now run an individual argument "],[966,"by clicking its run button in the Test Navigator. "],[969,"But before we do this, let’s add a breakpoint to the beginning of the test.\n\n"],[974,"And now let’s re-run it.\n\n"],[979,"The videoName shown in the debugger is \"Scotland Coast”, "],[982,"so we know we’re running this test with exactly the argument we're interested in. "],[986,"From here, we could continue debugging further "],[988,"and identify the reason for the failure. "],[990,"Conceptually, a parameterized test is similar to a single test "],[994,"that is repeated multiple times using a for…in loop. "],[997,"Here’s an example: it has an array of videoNames "],[1000,"that it loops over to perform the test. "],[1002,"However, using a for...in loop like this has some downsides.\n\n"],[1008,"Parameterized testing allows you see "],[1010,"the details of each individual argument clearly in the results. "],[1014,"The arguments can be re-run independently for fine-grained debugging. "],[1018,"And they can be executed more efficiently by running each argument in parallel, "],[1022,"so you can get results more quickly.\n\n"],[1025,"Parameterized tests "],[1026,"can be used in even more advanced ways than we saw here, "],[1029,"such as by testing all combinations of two sets of inputs. "],[1033,"Check out \"Go further with Swift Testing\" to learn more.\n\n"],[1037,"Whenever you see a test using this pattern, "],[1039,"it’s best to transform it into a parameterized test function. "],[1043,"Just add a parameter to the function, "],[1047,"get rid of the for...in loop, "],[1051,"move the arguments up to the @Test attribute, "],[1053,"and you’re done! "],[1056,"Next, let’s talk about how Swift Testing and XCTest relate to one another. "],[1060,"If you’ve already written some XCTests, "],[1062,"you might be wondering how this new testing system compares, "],[1064,"or how to migrate your tests. "],[1067,"Swift Testing has some similarities to XCTest, "],[1069,"but it also some important differences to be aware of. "],[1072,"Let’s compare three of the building blocks from earlier, "],[1074,"test functions, expectations, and suites.\n\n"],[1078,"Tests in XCTest are any method whose name begins with “test”. "],[1082,"But Swift Testing uses the @Test attribute to denote them explicitly, "],[1086,"so there’s no ambiguity.\n\n"],[1088,"Swift Testing supports more kinds of functions, "],[1091,"so you can still use instance methods in a type, "],[1093,"but also static or global functions if you prefer. "],[1096,"Unlike XCTest, Swift Testing supports traits "],[1099,"for specifying information either per-test or per-suite. "],[1103,"And Swift Testing takes a different approach to parallelization: "],[1106,"it runs in-process using Swift Concurrency, "],[1109,"and supports physical devices like iPhone and Apple Watch.\n\n"],[1114,"Expectations are very different between these two systems. "],[1117,"XCTest refers to this concept as assertions, "],[1120,"and it uses many functions beginning with XCTAssert to denote them. "],[1124,"Swift Testing takes a different approach: "],[1127,"it has just two basic macros — #expect and #require. "],[1131,"Instead of needing many specialized functions, "],[1133,"you can pass in ordinary expressions "],[1135,"and language operators to #expect or #require. "],[1138,"For example, you can use double-equals to check equality, "],[1141,"or the greater-than operator to compare two values.\n\n"],[1145,"And you can easily use "],[1146,"the opposite operator to negate any expectation.\n\n"],[1151,"Halting a test after a test failure occurs is also handled differently. "],[1155,"In XCTest, "],[1156,"you assign the continueAfterFailure property to false, "],[1159,"and then any subsequent assertion which fails will cause the test to halt. "],[1163,"In Swift Testing, any expectation can be made into a required one "],[1167,"by using #require instead of #expect, and it will throw an error upon failure. "],[1172,"This lets you choose which expectations should halt the test, "],[1175,"and even alternate as the test progresses.\n\n"],[1178,"When it comes to suite types, "],[1180,"XCTest only supports classes and they must inherit from XCTestCase. "],[1185,"In Swift Testing, you can use a struct, actor, or class, "],[1188,"and a struct is encouraged since it uses value semantics "],[1192,"and avoids bugs due to unintentional state sharing.\n\n"],[1195,"Suites may be denoted explicitly by the @Suite attribute, "],[1199,"although it’s implicit for any type "],[1201,"which contains test functions or nested suites. "],[1204,"It is only required when specifying a display name or other trait.\n\n"],[1209,"To perform logic before each test runs, XCTest offers several setUp methods, "],[1214,"but Swift Testing uses the type’s initializer for this purpose, "],[1216,"and it can be async or throws.\n\n"],[1220,"If you need to perform logic after each test, "],[1222,"you can include a de-initializer. "],[1224,"Deinitializers can only be used when the suite type is an actor or class, "],[1228,"and that’s the most common reason to use a reference type "],[1230,"instead of a struct for a suite.\n\n"],[1233,"Finally, in Swift Testing, "],[1234,"you can group tests into subgroups via nested types.\n\n"],[1239,"XCTest and Swift Testing tests can co-exist in a single target, "],[1242,"so if you choose to migrate, you can do so incrementally "],[1245,"and you don’t need to create a new target first. "],[1248,"When migrating multiple XCTest methods which have a similar structure, "],[1251,"you can consolidate them into one parameterized test "],[1253,"as we discussed earlier. "],[1256,"For any XCTest classes with only one test method, "],[1258,"consider migrating them to a global @Test function. "],[1261,"And when naming tests, the word “test” is no longer necessary at the beginning.\n\n"],[1267,"Please continue using XCTest for any tests which use "],[1270,"UI automation APIs like XCUIApplication "],[1273,"or use performance testing APIs like XCTMetric "],[1277,"as these are not supported in Swift Testing. "],[1280,"You must also use XCTest for any tests which can only be written in Objective-C. "],[1285,"You can use Swift Testing to write tests in Swift "],[1288,"that validate code written in another language, however. "],[1291,"Finally, "],[1292,"avoid calling XCTest assertion functions from Swift Testing tests, "],[1296,"or the opposite — the #expect macro from XCTests.\n\n"],[1302,"Check out “Migrating a test from XCTest” in our documentation. "],[1305,"It has lots of details about how to translate assertions, "],[1308,"handle asynchronous waiting conditions, and more.\n\n"],[1312,"We’ve gone over the features of Swift Testing "],[1314,"and shown several ways to use it. "],[1316,"This is just the beginning for this new package, "],[1318,"and I’m so excited that it will continue to evolve in the community. "],[1322,"Swift Testing is open source and hosted on GitHub. "],[1325,"Soon it will transition "],[1326,"to the recently announced swiftlang organization. "],[1329,"It works on all Apple operating systems which support Swift Concurrency, "],[1333,"as well as on Linux and Windows. "],[1335,"And importantly, "],[1336,"it has a common codebase across all these platforms! "],[1339,"This is a significant improvement "],[1341,"over XCTest which had multiple implementations. "],[1343,"It means your tests behave much more consistently "],[1346,"when moving between platforms, "],[1348,"and there will be better functional parity between them.\n\n"],[1352,"Swift Testing is integrated into major tools and IDEs "],[1354,"in the Swift ecosystem, "],[1356,"including Swift Package Manager on the command-line, "],[1359,"as well as Xcode 16 "],[1360,"and VS Code with recent versions of the Swift extension. "],[1364,"Let’s take a look at Swift Testing’s command-line experience. "],[1368,"Here’s a simple package I created "],[1370,"using the New Package template included in Xcode 16. "],[1373,"We can run the tests for this package from the Terminal by typing swift test.\n\n"],[1382,"This causes both XCTest and Swift Testing tests to run. "],[1385,"The console shows pass and fail results using colorful output, "],[1389,"and includes detailed failure messages similar to the ones shown in Xcode. "],[1393,"Swift Testing has an open feature proposal process "],[1396,"and we discuss its evolution on the Swift Forums. "],[1399,"We invite you to get involved by writing or participating in feature proposals, "],[1403,"improving the documentation, or even filing bug reports. "],[1406,"All contributions are welcome! "],[1409,"So that’s Swift Testing. "],[1411,"Use its powerful features like expectations "],[1413,"and parameterized testing to improve the quality of your code; "],[1417,"customize your tests using traits; "],[1419,"and join us on GitHub and the Forums to shape this package’s future. "],[1423,"Don’t forget to check out \"Go further with Swift Testing\" "],[1425,"to learn even more ways you can improve your tests. "],[1428,"Thank you so much for watching!"]]}}