Pushing Swift to the Server

Chris Bailey at Swift Summit San Francisco, 2016

swiftsummit


Transcript

Chris: Okay, while we get that set up. Yes, I'm Chris Bailey. I'm part of the Swift @ IBM engineering team. We've been working on Swift since its announcement as open source in December last year, looking at how to help Swift come up and running on Linux and other platforms, as well as how to build server side frameworks for Swift. So that's largely what I'm going to talk about today. I'm going to talk about how we've been pushing Swift onto the server, and how in fact, everybody here should be able to build and deploy server side applications in Swift, because you already know Swift from the client and it's very, very similar when you deploy it on the server.

So, why Swift on the server? Now, when we talk about moving Swift to the server, what we often talk about is some of the real basics around things like performance. These are taken from a simple set of benchmarks running on 4 CPU Linux box, by an independent group. We don't run these, we just contribute to them and we see the results. Now, this is a simple computational benchmark, it uses 100% CPU on all four CPUs on the machine. And we can see that Swift completes the task in four seconds. Now, that's actually quite interesting because Java completes it in 4.3 seconds. So, Swift, out of the box actually has very similar performance characteristics to Java. This is actually quite a bit faster than Node.js and quite a lot faster than something like Ruby. So we see that Swift is actually quite a really fast out of the box language to be programming in.

We can also look at its memory footprint. This is the amount of physical RAM that's used when running the same tests. We see that Swift runs the test in 15 megabytes worth of RAM. We see that Java uses almost twice that, Node.js uses about 50% more, and we see that Ruby uses quite a bit more. This allows us to make the statement that Swift is ideal for cloud, because it's both fast and it's relatively small, and most clouds charge you for memory when you're deploying applications, rather than for CPU or physical machines.

Now, whilst that's an interesting story, I guess, unless you actually are paying for the hosting of this yourself or you have a very large hosting bill, right? If you're paying $20 million a year for your hosting, you might be interested in the fact that using Swift on the server will lower your bill. But as developers, what we really care about is, how easy is it for me to build that application? How safe is it going to be and how bug-free is it going to be? So maybe there's some more interesting ways we can actually look at whether Swift is valuable on the server.

That brings me to this: Mars. Now, unfortunately, not Mars 2024, so not SpaceX and the manned mission to Mars. What we're actually going to look at is something that happened 25 years before that date. So, in 1998 there was the launch of the Mars Climate Orbiter, and this was part of a series of missions to Mars that happened just before the millennium. Now, it took off in 1998 and it was part of something like a $280 million project. Now, around about a year or so after the Mars Climate Orbiter left, leaving from Cape Canaveral on December 11th, and with this aim of getting the Mars Orbiter in orbit around Mars to both check its climate and actually work as a guidance mechanism for getting future probes to actually land on the surface of the moon; so this was the entire aim.

Now, about 280 or so days after it left Cape Canaveral, unfortunately what happened was this: in 1999, it's 200 days later, on September 23rd, they lost all radio contact from the Mars Climate Orbiter, and the reason was this; it went a little bit too close to Mars, resulting in unfortunately, complete mission loss. The reason this happened was, we had the Mars Climate Orbiter, and it was on a planned trajectory to intercept Mars, and that trajectory was actually designed to hit Mars, but about a day and a half out from the planet it undertakes trajectory correction maneuver 4, so this is the fourth of the trajectory corrections that it has in its flight path. And the aim of that is to change its path so that it sets out 226 kilometers over the surface of Mars. This was the mission plan.

Now, what actually happened was, it came in on a completely different trajectory. It still went through the trajectory correction that it was supposed to, and having followed that, it came into orbit only 57 kilometers above Mars, and this led to it being far closer than it was supposed to, and leading to our mission failure. This is what happens when you get too close to a planet. How did this happen? Well, it's all about the fact that a lot of the work was done by two locations, one was Lockheed Martin, who did the ground software, and they were based in Bethesda. The second team were the NASA Jet Propulsion Lab, based in Pasadena, California. So these two teams were working on collaborative software, one that's ground software, the other one that's in the Mars Climate Orbiter, and what they were doing was creating software that works together.

So those two teams, what they did was they created what was called a Software Interface Specification. Now, you can think of that as an API of the way the two parts talk to each other. And inside that software specification, one of the bits of information that they passed was called total impulse, and this determined how long to fire the rockets for when they were doing correction maneuvers. Now, the source of the problem is, Lockheed Martin were using pounds-seconds, which is part of the United States Customary Unit. NASA, on the other hand, used Newton-seconds, part of the standard units. The reason this is so critical is, one pound second is the equivalent of 4.45 Newton seconds. So this meant we were actually burning the thrusters far longer than they were supposed to. So we ended up with a catastrophic failure, costing a $280 million project, because whilst we had a very well defined API between two bits of software, those two bits of software were actually working in completely different ways, because they were completely different implementations.

So, what this tells us is that software interop is actually really hard, at least according to rocket scientists who were capable of doing the equivalent of hitting ... What you do when you land a rocket is the equivalent of hitting a hole in one with a golf ball from something like 5,000 miles away. They can do that, they can get it there, but they can't do software interop. The reason of it is the whole design process. So they had two teams and they, between them, agreed on this API spec, and then they go off into their own silos, and they do their own implementations of the two sides of that API specification and they deploy them into their own pieces, and there's the API specification between the two. But they're completely siloed pieces of development.

So, we can do this better if we use collaborative development, where the two sides still have to agree on an API, but they do this by collaborating on one project. That one project then gets deployed with the pieces going to the ground software and the Climate Orbiter, and then you can actually generate that API that sits in the middle. So when you've got your end to end project, the API is just the way the two parts talk to each other. Now, in Swift terms this is collaborating on a Swift project, and in terms of client server, where our client is going to be iOS devices and our server is going to be something like the Kitura server in Swift running with a database in the cloud. Then that API specification becomes something called Swagger, which is a formal definition language for REST APIs.

So, taking this model, we can actually work in the same language on the same piece of code, and make sure that we're not using pound-seconds and Newton-seconds, because our getters and our setters for our data model are identical on both sides, and we're just transmitting that data from one side to the other, using exactly the same code, and we avoid this entire problem space. So, how do we get to the point that we can collaboratively build applications in Swift? Well, it all started off with Swift everywhere, and that's a side effect of the open sourcing of Swift on December the 3rd last year. So, the fact that Swift became open source, and it became open source under an Apache 2.0 license, meant that there was a lot of freedom for hardware vendors and independent open source collaborators to get involved and start bringing Swift to other platforms.

Now, on the day it was open sourced, Swift was actually available on two sets of platforms. They were available on the Mac OS and iOS platforms and on Linux Intel. Now, since then it's actually been brought to a number of other places. Now, some of them are still well in development, rather than actually released. But we've now got to the point that Swift is available on LinuxONE, which is Linus on main frames. It's available on Raspberry Pi, due to Linux on ARM. There's some work being tried out to get Linux running on Android, and there's actually some work as well to have Swift running on Cygwin inside Windows.

So now we've got this ability to run Swift in many, many different places, and that makes it possible for us to have client server and interoperability of our software with Swift on both sides. Once you have that and then you start to have a Swift server as well, then we have the ability to start to do this. On our client side, on our iOS side, we have Swift, we have the Standard Library, we have Foundation, we have Grand Central Dispatch. On the server we now have the same components, we have the Standard Library, we have Foundation, we have Dispatch. Which means, any application or any package that uses those standard components can run in both places.

Now, in order to run a server we do actually need some extra components. We need server sockets, so for Kitura we added a networking library. We need to do HTTPS, so we need SSL capabilities, so we added a security library. And we need to do HTTP parsing as well, so we added one of those. Now, with those added, we can then have our Kitura framework and we can have various frameworks on the client side and we can have our application libraries on the client, shared with the server. Because, all of that standard code that you wanted to use is available.

This is actually one of the reasons why in Kitura, for our server framework we were so keen on using Foundation and we were so keen on using Dispatch. Because, other third party packages use those, and if you're writing concurrency code, you don't want to have conflicts of concurrency model. If you're parsing a HTTP request that we give you, we want to give you data objects that you can pass on to other code. So we are using standard types quite deliberately to make it easier for you to share your application code on both sides of the connection.

Now, the one problem with this picture is these three gray boxes on the right, where we, for our server framework, had to build some proprietary code. What we've been doing is we've been working with Apple and the community to try and extend Swift across those components and add some server APIs as a standard part of the language. That's an announcement that you probably saw two or three weeks ago if you've been following the Swift.org blogs or you've been following any of the mailing lists. So we're now trying to remove those non-standard components and make them a standard part of the language, to make interop even easier.

Okay, so that means that between Swift 3.0, which brought support to Linux, and Kitura 1.0, both of which went out in Linux, you now have this ability to run Swift on the server, and you can actually run real applications to have Swift on the server side as well as having Swift on the client side. So, how do you actually build one? Let's take a very, very quick tour of building your first application. To create an application, the first thing you need to do is create a directory in which to put it, then you can use Swift Package Manager. So you go into that directory, you use swift package init, ask it for an executable as well, and that builds the scaffolding for your project. Then there's really only two bits of things you have to do. You go into your Package.swift file and you say you'd like to have a dependency on Kitura, which is our web framework, major version 1, because we now have 1.0 available. So that's one piece, that's saying you need your dependency.

The next piece is then to actually add the meat of the code for your server application, and here it is, it's less than 10 lines to code to get a server up and running. The first thing you do is you import Kitura, then you say you need a router. A router is just how we take incoming HTTP requests on a given URL and send it to code, so it routes the incoming URL request to some code. Now, in this case, router.get, this is a get request on (“/“). What we do inside is, here we're just sending a response back of "Hello, World". So, any request that comes in on (“/“), we send back "Hello, World." Then finally we have to say, we want to add a HTTP server, we give it the port and we tell it- that’s the router for it. So any request coming in on port 8090 goes to that router, prints, "Hello, World." And then we just tell it to run, which starts a run loop in the background. That's it. It's something like eight lines of code and it will stand up and run a web server in Swift.

Again, that's interesting. You know, it's interesting that you can do that, that you can then run it, go to your local host, and you get your splash screen, you go to (“/“) and you get "Hello, World". But, "Hello, World" isn't really an application. We do provide a lot of tutorials on how to build full REST APIs, how to set out a whole load of routes with GET and PUT and CREATE and so on, in order to build a full REST API. But what we're trying to do is make it easy for people to build applications. That's why "Hello, World" is less than 10 lines of code. So what we also want to do is make it easy for you to build a REST API. That's what we've been doing over the last couple of months, which I'm now going to try and go through in a demo, assuming this lets me switch properly, okay.

So, what we released on Friday via GitHub, so it's open source, is a utility called Swift Server Generator. So all I've done is, I've already got a demo directory and inside that I can run my tool swiftservergenerator, and because I'm in a demo directory it will choose to use that. Let's see if I can actually make this a little bit bigger. Yeah. Okay, so it will create the application, call it Demo, because I'm in a demo directory. I can give it another name if I want to. Okay. Then it creates a bunch of files for me and it actually then runs swift build. So it's, at this point actually built a full Kitura application for me. Now I just have to tell it what is the data that I want to be able to send to and from the server, what is the rest API that I want. I do this using swiftservergenerator --model, to create the data model that it's going to be working with. And this gives me a builder for that data model, so I have to give it a name.

Let's say we're going to build a to do list, and that to do list is just going to have a very simple set of data types in it, which have a name of the task and whether or not I've completed it. So, my model name can be called todo, multiple to dos are called a todos. Okay, so property names, let's say I'm going to have title. Title is a string and I'm going to say that yes, that is required, it must have a title, I'm not going to set a default. Okay, next property, let's have completed, and completed is a boolean, I'm not going to require it, but I am going to set a default, and I'm going to say by default, tasks which are created are not completed. Okay, and yep, let's update that package.Swift file, and that's it; it's now built a full REST API for that model.

Now, there's a couple of things it does at the same time. It creates a config file. This config file gives it a name, it tells it which port to run on, and it also tells it a “store”. Now, in this case, any models that I create are stored in “memory”. You can currently switch that to be a Cloudant or CouchDB database, but we're also building out the set of data stores that you can use over time. For this demo, I'm just going to use the in memory one, and that's the sort of thing that you might want to use for testing. So, let's say you actually have an existing back end that your server team have created, but you want to be able to do localized testing against a back end without actually running the full back end server. You could use this to create a back end, and use the in memory data store so you can populate it with data and then retrieve it again afterwards. So, that's one asset, which is the configuration.

The other is, we also provide you with something called a demo.yaml file, and that's that Swagger definition that I mentioned. This is a formal definition of the REST API. So if we look in that file, this actually defines a REST API, and at the end of it, it defines our model. So, that we said, we created a model called “todo”, with a type of string for the title, a type of boolean for completed, and you'll notice it already adds a number and id filled in for you. Now, if we go to the top, there's just one more thing I'm going to do. Now, REST APIs don't actually really define where they're hosted. You're supposed to add where that REST API is. For convenience for the tool we're about to use, I'm just going to put a host in so it automatically picks up the host for me. So, if I say it's localhost 8090, and save that. Right, so now all I have to do is run my server. Okay, and that's starting my demo server.

Now, the next thing I'm going to do is I'm going to start a utility called “Postman”. Now, what Postman does is, Postman's a utility for making REST requests. Because I have this yaml file, this Swagger definition, what that allows me to do is import that Swagger file. Once I've imported it, because it's a formal definition of my REST API, on the left-hand side it actually knows about the post requests. So it allows me to make those requests very easily because they've got a formal definition of the specification of the application. This means I can actually run a GET to find out what data that I've got. And it returns an empty array, because I haven't actually put any data into my server yet. Now, I can do that by using a post. So, I want to do a post to create a new entry, and I can do that by sending a JSON object and that JSON object needs to have a title, which is going to be say, “Create Demo”.

Audience member: You forgot the L.

Chris: No, there's an L in title.

Audience: [inaudible - audience informs Chris that he forgot to include the “L” in “title” earlier.]

Oh, okay. When I send that it's created my entry of a “tite” of “create demo” and it completed a false. Now, because it has an id value of 1, if I want to update it, I can go to PUT, set the id of 1, and tell it that for this item I want to set my completed value to true, and when I send that, if there's no extra typos, it tells me I've now said it’s true. If I go and get all of my data models, it's now updated it. So, this has actually built a fully running Swift server, and I didn't actually write a single line of code at any point.

Now, if I wanted to then go and do something more significant, I could go back and do that. I built my application. If I wanted to do anything more than just a basic API, I can do that. We provide modules that do authentication against things like Facebook, in Google, in GitHub. We've got services you can use, so, MQTT for messaging, IBM Watson for cognitive analytics. There's stores for various databases like MySQL, Cassandra, PostgreSQL, CouchDB, Redis, and so on. Most of the things you might want to plug into a server are already available. We've also got some utilities for deploying to the cloud. So, if you deploy to Docker there's a Docker image to allow you to do that. If you use Cloud Foundry you can deploy to Cloud Foundry in IBM Bluemix. And we provide some UI tools to actually enable you to do that more easily than having to understand all the command lines required.

There's also a whole bunch of tutorials, so there is the builder, but in addition to that, if you for example, want to know how you add authentication in manually, there's a tutorial that helps you do that. On top of that, there's a couple of full, complete, end to end demos. So, we've got one called Bluepic and we've got another one, which is called Blitter, which is an example of a social network with a similar name. So, that's largely what we've been doing at Swift @ IBM. We've got a Catalog, we've got a Sandbox, we've got Kitura, we've got the Cloud Tools.

So, together, you've got the ability to actually really start to build these collaborative applications where you have Swift on the client, Swift on the server. You can have the same engineers working on both parts, or at least, you can have the both parts being looked at and viewed by two separate teams who can comment on each other's code and you can share data models. All of this should mean that Swift is now there to build safer missions to Mars. So, thank you.