Sharing Swift between iOS and OS X
Pros and cons with programming 3D graphics in Swift on both platforms.
Hello, thanks for having me. I've been sick all week, so you'll have to excuse me. I tried to put together the best presentation I could. I've never done a lightning talk before and also I was dreaming when I wrote this, so sue me if I go too fast. Every image will be kittens.
A big thing to know about Swift is that if you have #ifdefs in your code, the code always has to compile either way, it has to make sense either way
So, I'm gonna try to jump in. We just heard about using type aliases to bridge the gap between programming IOS and programming OS 10 without a lot of ugly #ifdefs. I'm gonna be doing some of that trick and explaining why and what are the pros and cons as well, and also just giving you a random sampling of problems I've encountered in the last year or so programming 3D graphics in Swift on both platforms. First thing I hit on was NSImage and UIImage, and what we're gonna see again and again is this NS-UI dichotomy, where it's like, they're almost the same class except you need to use these two letters differently and that changes everything. A big thing to know about Swift is that if you have #ifdefs in your code, the code always has to compile either way, it has to make sense either way. And so, you can't actually do the full thing where you can go and see, like in a method header if I'm in Swift, or use UIImage in my method definition otherwise, use NSImage. That actually won't compile in Swift, not in the header, but in the actual object file.
01:41: And I just lost my speaker notes, so that's gonna be fun. Oh, so NSImage we just see that I created a platform image, pretty straightforward. Another big advantage is that you get to actually return platform colour in your API, instead of having to have two different methods or an #ifdef. Colours are interesting because UIColor and NSColor don't share a lot of methods. NSColor is much richer but you still would want to have a type alias bringing them together, because in Apple's API it's pretty much wherever they use NSColor on one platform, they'll use UIColor on the other. And so, it's gonna make your interfacing to their APIs much cleaner to just always use a platform colour. And it does work to pass in, if you say typedef UIColor platform color, or NSColor platform color, it does work to pass those in, in place of UIColor or NSColor on the appropriate platform. So, you could set up these type aliases yourself but why bother? SpriteKit actually already did this, it's called SKColor. So, just use that if you wanna use it 'cause that way people already know what it is.
...it's gonna make your interfacing to their APIs much cleaner to just always use a platform colour.
One nice thing about having these type aliases is that you get to extend on both platforms at once. So, here I made a convenience method on SKColor for iOS only so that it responds to one of the same methods that NSColor does that I use a lot. But why do that when you can actually make it prettier and actually have a method that you add to both platforms, that makes it actually return a... Quick quiz, what's this thing called? A red, green, blue, it's called a? Right, tuple. I was testing you, I'm not just really sick.
03:31: One of the things I played with when I was looking at colors is, why not just use CGColor because especially with the new, new, new Swift 2, CGColor, which used to be called CGColorRef is, you basically can treat it like a full blown object, which is really, really cool and it's one of my favourite things about Swift is that all these CG and CF objects, objects which were really ugly to use for CFRetain, CFRelease, bridge between CF and ARC, all this just crap you had to do, and now they just act like full-fledged objects. You can extend them, you can pass them wherever you would pass an object in Swift, it's really great. They're retained and released just automatically under ARC, which doesn't happen in Objective-C in ARC. You can see that Steve wants to be garbage collected. These are actually all my cats by the way too, I wanna mention that.
Let's talk about “crass platform”, it really is “crass platform”.
And you can add cool stuff like Equatable which I think is really great. I just love it. I haven't gone completely bat shit with it yet, but you can imagine just extending CG so that the whole thing has a beautiful, Swift-like interface where you can create a CGPath and add stuff to it by passing in methods, instead of using the stupid CGPathAddRect with blah, blah, blah. Let's talk about “crass platform”, it really is “crass platform”. GL, I'm gonna finish this by explaining why you should never use GL and it's the worst thing ever. But first I have to tell you how bad it is. Here's a little summary of what we'll be talking about and why it's horrible and you can see that when you have cats on platforms, it's always bad.
05:09: So, here are some problems; on OS 10 we have Open GL 1, 3 and 4-ish. Those are sort of the flavours you might encounter. You're not really gonna use 4 because a lot of machines that are current and running El Capitan, do not run GL 4.1 or GL 4 at all. Everything that runs El Capitan does run 3 but the reason you're on one is because stupid CA, core animation, C-layer uses Open GL 1 and cannot run under GL 2, 3 or 4. So, by default, core image and SceneKit both create a GL 1 context. It is stunning, we're talking about something from 2002. On iOS it's a little better. I believe they default to GL ES 2. But what's maddening is, there's a sync-up where Open GL 3 syncs up with GL ES 3.
If you're trying to do cross platform programming, it's really important to have, you know, the same thing supported on both platforms.
If you're trying to do cross platform programming, it's really important to have, you know, the same thing supported on both platforms. You also have the problem, that on iOS, if you have floating pixels, they have to be halves. You have a problem that on iOS, they have an EAGLContext, which is an object, and on OS 10, you get both an object, open GL Context, and that is just a wrap-around obstruct, CGL Context, and you have a problem in that the GLSL, the shading language that you use to write on the actual chip is a different version, but there is a version that syncs up. It's exactly compatible, so those are the versions that you wanna use.
06:41: Here's quickly, and I'm not gonna go over the code in this slide, but here's quickly my platform GL Context that I created for both platforms. The advantage to doing this trick is that I both have some code in here to do locking on both platforms correctly, and also you can pass this directly into SceneKit or wherever to the exact same code with no #ifdefs, once you've done this type alias. Here's an example of me using the type alias, and then I just say I actually created another function which takes a @noescape, so you can just pass in the block, and safely execute any GL code, at any time, on any thread, without crashing all the damn time. One thing to know is that SCNRenderer does not lock the GL Context it uses. You're in charge of doing that. SCNView does, SCNRenderer does not, so this is really important to have to do this locking, *cough*- unless you’re using Metal.
Let's talk about enumerations and how horrifying they are.
Let's talk about enumerations and how horrifying they are. Actually, let's not yet, because this slide's mislabelled. So, we talked about what's good about this. We don't need to use all these #ifs, it lets you put in the consistent locks and unlocks. Let's talk about enumerations because they're horrible, so the GL team screwed up, or was lazy, or there is no GL team, pick one. This is the code you have to write if you wanna call GM, GL, see all the GLenum, GLint, GLsizei, GLsizei. You have to do all of that because all those are the wrong type when they're imported into Swift. So even thofugh they're never used as any other type, they're all imported as ints, and you have to cast them. That's just because the GL team really just didn't bother, which is understandable. Now that we have Metal, we understand why they did it.
08:20: Another just "gotcha" that you have to watch out for is that there are some calls that are just different in ES, if you're calling in C. For some reason, ES, they're like, "We need to pass a float when we accept the depth-fill colour, but a double if on OS 10, but a float on iOS, because floats are so much faster and you're always calling set the depth-fill colour." Thanks, guys. Those calls, literally, they're not on each other's platforms, so it's like one of those things you have to watch for that if your programming GL on both, which you shouldn't do. And then, the GLSL, that's the shading language that you program right on the chip. It goes along with programming GL, which is a C API. There's a huge problem in that in open GL, not GL ES, in open GL the version numbers did not actually track the version numbers of GL, so you can see GLSL version 150 corresponds to GL version 3.2. Now, they synced it up with version 3.3 and that happens to be the good version that we wanna use anyway, and it also happens to be the one that's compatible with the iOS GLES-3, so there's the sync point. That bar, that's what you wanna do under there, if you did it.
How are we doing? So, you can write the same shade, or GLSL, the exact same code for shade GLSL, but it's tricky because on both platforms, you have to actually tell SceneKit, or if you're creating it yourself, create it yourself, to use the right version of GL or GLES, so here's the version- you need to use 3 on both. There's a trick because both those versions require you to have a version tag at the top of your file, but the version number is different for GLES and GL, so you wanna actually just write your shader code, without a version tag, and then add the version tag in code, depending on which platform you are. That's the example code that does the correct version number.
Let's look at the same differences in Metal that we saw in GL.
10:11: Let's talk about Metal. Basically, all we have to do to talk about Metal, this is Tara. She's my snuggle bunny. Let's look at the same differences in Metal that we saw in GL. Well, the flavour of Metal is Metal, and Metal, so that's good. The size of Floating point pixels is 16 or 32 on both. The context that you're gonna use on Metal is always a Metal device, and the shading language version is always Metal 1.1. Well, let's go home, thanks a lot. No.
So the APIs are basically the same. There were presentations this year when Metal came to OS 10 about what's different. Very little is different, they're just like, "Here's a flag to tell it to cache on this instead of that." And the amazing thing is, I didn't even watch those presentations, yeah I know, but I like wrote the sample code, copied it off of iOS, ran it on OS 10, and they literally launched and said, "Oh hey, you need this flag when you create this texture because it needs to not be on the chip. It needs to be in memory," and I'm like, "Thank you" and I'm done. It was really great. They just told me what to do in the code, so it's really hard to get it wrong, unlike GL, where it's not even possible to get it right. Threading is pretty much just handled by Metal. There is a whole threading architecture that they've thought about and actually done, but mostly, when Metal calls you back to do something, they've thought about the threads already. You don't have to, you just have to know you can be called from different threads, so nice. I don't have any of that locking code, that ugly locking code I showed you before, all gone. Shading language is common, nothing to talk about that. Here's something cool that I saw with Accelerate. Yes, I have 17 cats.
They just told me what to do in the code, so it's really hard to get it wrong, unlike GL, where it's not even possible to get it right.
11:57: This is a hack, and it's no longer needed, but if you run into a problem like this, this tells you to get around it. For a while in the Accelerate framework in the DSP Header, they just forgot to do vDSP_vadd on iOS but not on OS 10. So we created this little work around shim where we had a special Objective-C file with a special name that you could actually just use to do it. This is how you do work around if there are any more of those out there. Increasingly, we're not gonna see these. So I'm hoping this won't be at all relative... Relevant, or relative, I don't care. This is how you'd use it. Again, I'm skipping through the code 'cause it's all gonna be on the web so you don't really need to read it now.
12:34: Cross-platform SceneKit, my last topic. There are a couple of big gotchas in SceneKit... Basically, SceneKit is awesome, by the way. SceneKit is super fast, it's super beautiful, it is a high-level abstraction that lets you dive down at any time to the absolute lowest level of the iron. There is no such thing as a performance penalty for SceneKit, 'cause if you have one, you're doing it wrong. You can always dive down, so it's your fault.
They did something really phenomenally stupid, and I've talked to them extensively about this, which is when they went to iOS they said, "Well, doubles are expensive, so let's just hard-code all of our floating values to type Float." Now on 10.11, you can see, this is from their headers. It's CGFloat, and we already know that CGFloat on 32-bit platforms is 32bits and x64, x64. But on iOS 9, they have 32-bit always which really, really complicates things because if you're dealing with CG, which of course you are, you've got these things which could be 32 or 64. But then, even if they are 32, even if you're building it, it's not compatible with Float because CGFloat is a different... I don't know what you call it... A different thing. It's not a struct. It's not a class. It's a different type than Float. It's not a type alias. So even on, if you're building iOS for 32 bit, you can't just use Float. You have to cast it every single time you use it, which is really stupid and annoying.
...cross-platform is just really hard with Swift in general because(...)they don't have implicit casting to richer types.
14:02: So on OS 10, you can basically pass all these things to CG call, on iOS, you never can without casting, that's really annoying. As an aside, SceneKit is just, or cross-platform is just really hard with Swift in general because they don't do any... They don't understand that when you are casting something to... They don't have implicit casting to richer types, I guess it is the simple way. I actually learned from Andy the correct way to say that, but it's like “monomorphic, monocoque architecture with a zoomified node thing”. I don't remember. We didn't even remember five seconds after he said that. We tried Googling it and I was like, "Yeah, that, whatever it is." Anyway, what we really want is if I say, "Hey, look, I'm gonna call a function that takes a double and I'm passing you a Float," Swift should just go, "Yeah, I know what you want because doubles are always richer than floats, you're never gonna lose value. You might as well just promote it to double, I mean silently. There's never any harm in that, obviously." But, they didn't do that 'cause they don't have the “double monocoque overhead cam”.
There's some mitigation, and SceneKit has gone from being actually the worst Swift client a couple months ago -I love those guys so I'm allowed to insult them - To actually one of the absolute best in terms of their extensions. They have a bunch of extensions that are only in Swift SceneKit. So they have all these initializers to make it a little easier on us. The very fact that CG graphics has all these initializers too shows you that there really is a problem with Swift, 'cause this is from the CG header. And obviously, they have all these hacks because they don't have automatic promotion. What you wanna do to solve this is create your known SCN float. SceneKit was supposed to do that. I think it didn't make 10.11 but it probably will be coming. And that's it. Thank you.