Open source and Community
Panel discussion on: Swift migrations, proposals and feedback for the core team.
Moderator: Jesse Squires
Jesse: Okay, so here's our panel on Open source and Community. Just to start off, I'll just have everyone introduce themselves again and say where they're using Swift currently- whether it's at work, or only side projects, or both.
Ayaka: I'm Ayaka. I am an iOS engineer at Workflow and we're using Swift in our codebase, and it's a very mixed codebase between Objective-C and Objective-C++ and ... Yeah, fun times.
Javier: Hi again, my name is Javi. Right now, most of the Swift that I write is in my free time when I mess around with things like the Rubik’s cube library and stuff like that. At work, I've been working on the Fabric app for a year but I recently started in a new team; the Twitter for iOS app, and what I'm running right now is not Swift but there's quite a bit of Swift already in that app, so I'll still be writing some there.
Benjamin: Good one. Hey again, my name is Benji. I work at Plangrid. I've been writing all of our new code in Swift almost entirely. At this point we're probably 50/50 between Swift and Objective-C. And Swift is also my favorite free time language, so once in awhile I write it in my free time too.
Jesse: Cool. So if anyone watched the talks from last year's Swift Summit or if you attended last year, you know that a big topic on the panel was migrating to Swift 2.0 and so here we are again. It's a little deja-vu with Swift 3.0 on the horizon, especially with Xcode 8.2 beta dropping Swift 2 support. So how are you guys dealing with the migration? Have you started? Are you finished?
Ayaka: I can start. We did finish, but we only had to migrate about 80 to 90 files and even that took a solid three days to finish. And sometimes it didn't feel like there was a light at the end of the tunnel because you would fix one thing and then five more bugs would show up. I wasn't sure if I was going in the right direction, but we did manage to finish.
Javier: The project where I went through a Swift 3 migration was the Fabric App. It was around the beginning of September whenever the GM landed, and I spent a good week and a half, maybe not completely full-time but quite a bit of time on it, and then I went on vacation and left the rest to the intern, Manuel, who did a great job. But the thing is that I really didn't expect him to finish at all. The thing is that I had no idea if I had done two percent or 98 percent because you only see five errors, you fix those five errors and then there's ten more errors. So there's no progress bar but he managed to finish it in another week.
Benjamin: So we're currently working on it. My team is actually wrapping up right now, I think. For us it was probably three and a half to four days, two people full time to get it to compile again. Our big advantage was that we had a separate module, so we kind of split it half and half and that allows you ... I wish we had more modules but the linker stops us from doing that. But if you have more modules, then you can fix them one by one and then you can fix the dependency of that module. And that made it a little bit less painful.
Overall, the migration went pretty fast. I think there's a lot of aesthetics stuff that we need to fix, because we don't treat argument labels the right way as we would in Swift 3. There's a lot of polish we can do. But overall, it was not as horrible as I expected after reading other people's experience. The biggest problems were things like mutating parameters. The schematics changed a lot so we have a little bit of that code to migrate but overall it was fairly straight forward. And because Swift is such an a static language, I felt pretty confident that even with all these changes, most of them are just going to be compile time verified things, so.
Making the migration process more manageable
Ayaka: I have kind of like a follow up question to that. So when you were going through the migration and stuff, the one thing that I tried to do in the first part of the migration was actually ... so you know how there's fileprivate and private now and all that? So initially I was gonna try to make everything private as I'm migrating it, but I realized that's a horrible idea because you don't know what all the errors are going to be and who knows, maybe an extension in the same file’s using that variable that you made private but it needs it to be fileprivate. So how did you kind of go about making the migration process more manageable?
Benjamin: In our case, our goal was really to get it to compile as fast as possible without doing any cosmetic changes so we didn't try to adopt Swift 3 best practices. So we were mostly sitting there hitting the, I think it’s; command-option-shift-F, to apply all fixes. We had basically a notepad where we were sharing common fixes that the compiler couldn't apply. So for dispatch cues, we had some things we had to migrate. So we were just sitting there really going for super fast and getting it to compile. Then as a second step we’re gonna audit all the changes at some point. But unfortunately, I think, yeah we’re probably sticking with fileprivate and incorrect argument names and such kind of things for quite a while and then we are gonna clean it up as we go.
Proposals with big impact in Swift 3
Jesse: So that brings up a good point. With fileprivate, that change was introduced via the Swift evolution process. That was a proposal from the community and we had quite a few proposals in Swift 3. Way more than in Swift 2.2. So are there any proposals that you all think had the biggest impact on your code? Perhaps a change that allowed you to write code in a better way or faster way or something like that?
Javier: I can start. To me, I think the one that makes the most dramatic difference to what your code is going to look like, whether it's the one you just migrated or the new one you're gonna start next year, is the implementation of some of the foundation types as value types, using structs. Especially looking back to the first version of Swift and when we all started writing some UIKit code in Swift 1.0, we were like wow, Swift is cool, but as soon as you're using the Objective-C APIs, this is not Swift and you know, it’s starting with implicitly unwrapped optionals. And now, it's kind of unbelievable that three years later, two years later?
Jesse: About two, yeah.
Javier: You're writing all this Swift code, interacting with APIs that are still Objective-C. Apple hasn't written any framework. But suddenly it starts feeling like, it's almost like they are Swift APIs, they feel a lot like you would do them in Swift. Especially with some of them being structs and value types.
Benjamin: That probably would've been my first answer to you because it's definitely the biggest impact just in terms of volume of code. And there were a few others that were pretty important to us because we have Objective-C and Swift side by side, so the ID as Any one was really important because we interop a lot from Objective-C and a lot of places we would get to use AnyObject. That was really difficult to actually implement Swift's best practices and stick with structs and enums when we had AnyObjects floating around. So treating them as Any was one of the small changes that's gonna have a pretty big impact on our codebase. So the interoperability stuff is really great. Another one was abolishing the implicitly unwrapped optional. So that all Objective-C types are now actually imported as optionals and not forcefully unwrapped.
Ayaka: I'm definitely also a fan of all the things that made Obj-C interact a lot easier because us too, we have so much Objective-C code and like Javi was saying, it's nice when you can use something that's already Objective-C but it feels like you're using a Swift API and you don't feel the need to rewrite everything and spend all that time doing that. So I think all that is gonna help us move forward and get more people and more teams using Swift.
Updating educational content online with new migrations.
Jesse: Yeah, great answers. One thing I think that people don't often consider or think about when we're talking about these migrations with Swift, from 1.0 to 2.0 and 2.0 to 3.0, we have a lot of people in the community with open source tools or projects but also all these blog posts explaining how to do certain things in Swift. Many of these things are becoming outdated now. Things that you could google on how do I do this in Swift, maybe you look it up and it's still on some article that was written in the Swift 1.0 days or the Swift 2.0 days. I know all of you, I think all of you write blogs, how do you deal with this kind of non-technical or kind of technical debt?
Ayaka: I think for me it depends on which blog post it was. So I have some blog posts that are just like, this is how to use guard and this is how to use if let and stuff like that. The thing for introductory things like that, I want to make sure I update them so I don't confuse someone who's just starting. But I think, for other things that are more complicated like I don't know, a random script to do xyz, I think if you're at that point you can probably tell there's a difference between Swift 2 and Swift 3 and maybe you can just make the changes yourself and fix them. But I definitely feel the pain of ... now when I have to Google for problems that I'm having in Swift, I keep getting Swift 2 search results but how do I do this in Swift 3? It seems like people haven't indexed them yet or it hasn't gone up the search rankings yet. So it's a little but painful in that sense.
Benjamin: For me, I mostly just touch the stuff that actually gets some form of attention. Just because time is limited, so in one case I had someone comment on a blog post, in another case I had someone open an issue on an old open source project and then I'll go and migrate it. Otherwise, usually not because it means probably nobody is using that stuff. It's kind of a good natural selection. And it's painful on the one side but on the other hand, I look at people on our backend team that work with Python and they're still working on Python 2 while up there is a version of Python 3 that no one uses. Sometimes I think this hard cut is useful in some ways so we have one language that's the only accepted current form.
Jesse: Sure, yeah. So while we're on the topic of debt, I have one last kind of question or thing to talk about. A few weeks ago, Brian Gesiak, who's been contributing to Swift on Android, wrote this blog post about technical and knowledge debt about how he would be working within the Swift compiler codebase and he'd see all these things. He wasn't really sure what they did but he would copy and paste them and continue working and mimic things that he saw, very much like what Ayaka was talking about in her talk earlier. So this is really just for Ayaka I guess. How are you dealing with this kind of technical knowledge that you talked a lot about how the things that you built, you were just kind of looking at other examples? Have you gone back to dig in more into exactly what those things were doing?
Ayaka: Yeah, so I'm trying to think of an example of something that I thought was weird but I just let go when I was working on that issue. Let's see ... this one is kind of small and stupid. I was implementing the compatibility alias, importing to typealias, but I realized that all those things in the importer were under the IDE folder, I was like why is it under IDE? To me that doesn't make sense but maybe it makes sense for someone else. So I thought that was weird but I'm trying to poke around more and understand more but obviously I still don't understand most of it. I'm just trying to read blog posts and encourage other people to also share what they know. There really isn't that much out there and I'm just kind of hoping that everything that I'm seeing is like best practices.
Feedback for the core Swift team
Jesse: So along those lines, maybe it'd be nice for the core team to step in a little more and help with these types of things. So I think this is a pretty good forum to kind of give the core team some feedback. This past year, we saw the first full release cycle of a Swift major release. So 3.0 was the first release that was completely open source. 2.2 was just kind of ... it was a point release and it was just kind of half of a cycle. So this was the first time that we got start to finish for a release. What kind of feedback do you guys have for the core team as far as they engage with the community as well as make good progress on Swift's promises?
Javier: I think it's tricky because we can obviously ask more of them in terms of making their README's more complete and helping me write code in the Swift compiler. And I think that's gonna happen naturally. I'm sure we can come up with some ideas for how to start with that. But I think we're at this point in time when I think it's natural that most of the Swift compiler code is written within Apple and I think that's okay. So it's tricky for them because they already have the knowledge, they don't need to document 100 percent of the tools for people on the outside and if they spent time documenting that, it's less time they're spending writing actual code in the compiler and they also have tight deadlines. To me, that's the tricky balance.
But I think there are some things that are being done and the example that I can come up with is that thing that you do with started tasks. I think that's a great idea to encourage people to do little things that are maybe easier and that way they can figure out how to get into it. And I think with talks like Ayaka's, we're gonna start to become more familiar with the scripts that we have to use and how to get in there and do things.
Benjamin: I actually have a funny anecdote around that because at one point when I was working on something in the Swift compiler and it kind of became obsolete while I was working on it. But the biggest thing that held me back was just knowledge of the tools. Anyone can read through the source code, there's a lot of documentation, there's a lot of comments. But just getting tests to run efficiently, that was one of those things where I just got blocked. I remember talking to someone on the Swift team, at the WWDC, about the setup and how I could do it better, and the answer I got was; it's a standard LLVM setup. I understand that it's the perspective, but then we talked about it a little further and he's like: “oh you want a tutorial of how to submit a patch”. I was like: “yes, that would be awesome”. So I think if you get to a point where, just a cycle of how do I run individual tests, how do I efficiently use the tooling, once that's set up I think we can help ourselves a bit better.
Ayaka: I do wonder, if we wanted to make the README better, I don't want to just open a PR out of nowhere. Do I go through the Swift Evolution process or is there somewhere else better that I can post about, that we can all participate in and figure out maybe this is the best way to come up with this decision. Because I feel like Swift Evolution is more for like the language itself but maybe it's also okay for the process around it? I don't know.
Jesse: Can you update the README with that information?
Ayaka: Yeah, I do want to do that but it's like, where do I insert it because there's already so much there. I feel like my update is just like a subset of what's already there. Just remove everything else. But like easy mode, README easy mode, I don't know.
Jesse: Cool. So we are a little over half time. I want to give time for the audience to ask any questions. We still have a few minutes left.
How do you divide and conquer a Swift 3 migration with a large team?
Jesse: So the question was how to divide and conquer a Swift 3 migration with a large team.
Benjamin: If you have the chance to set up modules beforehand, as I said it helped a lot for us, just because you have the dependencies you can start with the one module that everything depends on and you can get that to compile and then step by step through the other stuff. So for us it was, two main modules and then two test targets. It made it a little bit easier to split up the work- for example, on tests in the module. If you're in the same codebase, it's difficult, you can probably split up based on files and just try to not get in the array. In the beginning when we worked on the codebase together, we mostly would actually just pair and just sit side by side and that would keep up the velocity. If one person got stuck, someone else could open a Playground, play around with something and provide a fix. So if you don't have modules you can pair. And if you have modules, then you can probably do some stuff in parallel.
Ayaka: Yeah, I don't have an answer because I just did it myself. I didn't divide and I conquered.
Javier: It's very hard.
In Swift 4, what are some of the new features you're looking forward to?
Jesse: The question is, in Swift 4, what are some of the new features that you're looking forward to or new proposals that were deferred until Swift 4?
Benjamin: So one of the things that we've been most excited about are the changes around generics. I think it's called generalized existentials, that’s the formal name. Also the conditional comformances. These are the two main things that really affect our codebase. The generalized existentials is a complicated name for basically getting rid of all the type erasure that you need to do today. So you can work with a collection of some element, you can actually declare a variable that stores that, and you can use the protocol types instead of a concrete type to do that. So that would have a lot of impact on our code. We have a lot of workarounds around that, so having the generic manifesto at least implemented to some degree would be cool.
Ayaka: For me, all I really want is compile time performance and just making that faster because I think most of our compile time problems just came from ... so we do all of our auto layout in code, so every time you want to activate some constraints, we call NSLayoutConstraints, activateConstraints and just pass in an array of in line declared NSLayoutConstraints. But we realized that's making our compile time super slow, so what I have to do is I have to pull that out from the activateConstraints call and create a mutable var constraints array. I keep upending to that and then at the end, just call NSLayoutConstraints, activateConstraints. That kind of felt backwards because I like stuff because of all the immutability and safety and I noticed that when I was going through the conversion process, in some places I forgot to call to activateConstraints at the end. And why isn't anything ... why are all my views on the top left hand corner. I realize that I just messed up and I wouldn't have done that if I didn't have to make that band-aid fix for the compile time.
Javier: I'm personally excited about the memory ownership model, whatever shape that ends up taking. But I think it could be really nice, especially their goal is to make it opt-in so it's not gonna make the language more complex for everyone, but in certain parts where you're dealing with multi threading and sharing variable mutable data, I think that might be really powerful.
Are there any features in Swift 3 that you would've omitted?
Jesse: So the question is, any features in Swift 3 that you would've omitted?
Benjamin: I don't think I have anything large. The fileprivate and private thing is one that's coming up a lot and I maybe agree with but it's not a huge deal really. Another tiny thing I'm not 100 percent sure I like a lot is the protocol composition syntax that changed. Just looking at it now, it's kind of difficult to parse, we used to have the angle brackets around them but it's all aesthetic stuff. It's nothing of significance. I think most of that stuff could be reverted without breaking AVI compatibility, I think. I think all of the changes were pretty good overall.
Ayaka: Yeah, I don't think I would veto anything per say. But I think, in terms of cosmetic changes or in terms of what looks good and what doesn't look good, as I was mentioning earlier when we were migrating, we couldn't go through all the fileprivate declarations and turn them into private at the same time. So it feels kind of dirty when I see our codebase and it's littered with fileprivate everywhere. So I wish we could've migrated that separately or something like that. Like oh, do I actually not like fileprivate or is it that I just don't like that our codebase is littered with them? And it feels like it's hard to maintain.
Javier: Yeah, same with fileprivate but I think that's the one that might be the most likely to change in some way, based on what I've read on Swift Evolution.
Jesse: Yeah there's been discussion about potentially reverting that decision in Swift 4, phase 2. Chris Lattner had an email saying that the new information that we have now is actually using fileprivate in practice where before it was just a proposal, it was just kind of this theoretical idea about how to make something better and then in practice it ended up being maybe more cumbersome to deal with for developers. If there are any changes like that that you've seen, you should actually bring this up on the mailing list. This would kind of be like, the absolute last time to revert any changes like this that are this big. But if there's anything that has actually ended up being detrimental, there's a chance, this last chance to make some tweaks.
Any plans to open source Swift apps at your company like what Artsy has done?
Jesse: So the question is, any plans to open source Swift apps at your company like what Artsy has done?
Ayaka: For Workflow, probably not the app itself because I guess for Artsy, what's important to them is the actual data. But for something like Workflow, the app itself is our intellectual property and what makes Workflow special so that's probably not going to happen. But it would be cool, once we start digging into Swift a little bit more, just like open sourcing various components that we're using. So maybe not the entire app but random parts of it.
Javier: In my case, the one that would be relevant would be the Fabric app and I really can't say much. But personally, I can say that it's something I've been motivated about, I totally believe in. I guess big companies are hard, all I can say.
Benjamin: In our case, we're a startup, we have in our space some competitors. I would assume it would be a very long battle, probably much longer than I'm gonna work at this company which was originally two to four years. To convince first everyone internally and then it would have to convince investors and a lot of other stake holders. I think for a lot of companies that's unfortunately not an option. If someone at the top, I think in Artsy's case, is very motivated by this idea then they can do it. I also hope we can open source components at least.
Jesse: Okay, I think we are out of time. So let's give a round of applause to these awesome panelists.