Cloud Platform Webinar Series

Made for Each Other: Microservices + PaaS


Share

Companies need to build better software faster to compete. But existing monolithic applications, legacy platforms and lengthy operational deployment cycles are holding innovation back. Microservices are becoming the cloud architecture of choice because they offer the ability to loosely couple applications into discrete services that can be surgically changed without requiring disruptive overhauls. This approach enables the responsiveness and rapid change needed by the business.

Enterprise PaaS is a critical foundation to simplify the operations, governance and health management of these new architectures. Together with a DevOps culture, microservices and PaaS are the engine that drives innovation at speed.

In this webinar you will learn:

  • How Continuous Delivery drives business innovation
  • What you need to know about microservices to get started
  • How to use enterprise PaaS to facilitate operational agility

Join us on November 18th for Pivotal and RedMonk’s web seminar: Made for Each Other: Microservices + PaaS, where you will learn how to architect your applications for rapid innovation.

Speakers

Donnie Berkholz
Research Director, 451 Research

Donnie Berkholz is the Research Director for the Development, DevOps, & IT Ops channel at 451 Research, which covers the technologies employed for software development and software lifecycle management to drive business growth. Donnie's own coverage encompasses development, testing and release engineering. This includes areas such as programming languages & tooling, source-code management, application lifecycle management, and continuous integration & delivery.



Alan Ho
Head of Developer Programs, Apigee

Alan Ho is the Head of Apigee's Developer Programs covering developer relations and technology partner marketing. Previously, he lead product management of Apigee Insights, API BaaS, and mobile APM. Prior to Apigee, Alan was the CEO and co-founder of InstaOps (acquired by Apigee).



Matt Stine
Senior Product Manager, Spring Cloud Services for Pivotal Cloud Foundry, Pivotal

Matt is a technical product manager at Pivotal. He is a 15 year veteran of the enterprise IT industry, with experience spanning multiple business verticals. His current specialty is helping enterprise IT drive faster and more innovative value streams through cloud-native application architectures, leveraging the techniques of Continuous Delivery, DevOps and Microservices on Pivotal Cloud Foundry.


Transcript

Peter: Good morning. Thanks everyone for joining us on the pivotal webinar program. We’re here this morning to talk about DevOps, microservices and the new composable enterprise. Today’s webinar are microservices and PaaS made for each other.

We’re going be having two speakers today. First is Donnie Berkholz from RedMonk who is senior analyst at RedMonk and he’s also going to be joined by Matt Stein from Pivotal, one of our senior engineers. Before we get started just a couple of quick housekeeping notes.

If you have a question during the webinar please use the Q&A panel to enter your questions. We’ll do our best to answer your questions during the webinar, but generally we’ll probably save most of them for the end of the webinar where we’ll take your questions and feed them out to Donnie and Matt who will then answer them. Definitely stay on till the end. If we can answer your question earlier we will.

All of your microphones have been placed on mute, so the Q&A is the only way to enter your questions. If you’re having any trouble with getting connected via Voice over IP or listening over the computer you can go to the event menu and go to the information tab and there’s a dial in number for you there to use. If you’re having any trouble with the audio you can always, the first thing to do, you can always try switching and dialing in. Sometimes exiting Event Center and reentering helps as well.

This webinar is being recorded. It will be posted to Pivotal.io site when we’re all finished. The slides and a transcript will also be posted to Pivotal.io. We’ll be sending out a thanks for attending mail to all that registered for the webinar and it will contain that information in it. Don’t worry if you don’t have that information before you leave today at the end of the webinar. We’ll be, like I said, recording it and making sure that that’s posted so you can share it with your colleagues.

Great. If you’re having technical issues or you have a question unrelated to the content of the program today and more about WebEx or something like that please feel free to send your question via the chat panel to the host which shows up as Pivotal Webinars and we’ll help you out. Otherwise if you have questions about the content please enter them in the Q&A panel.

Great. With that I’d like to turn today’s presentation, microservices and PaaS made for each other over to our first speaker Donnie Berkholz, senior analyst at RedMonk. Donnie, take it away. Thank you.

Donnie: Thank you very much. I appreciate the kind introduction. RedMonk, if you haven’t heard of it before, we’re an industry analyst firm and we’re focused all about developers, developers, DevOps, sys admin, people actually doing the work in the trenches. We spend a lot of time focusing on what they’re doing, what they’re talking about, what technologies are getting traction.

I think the technology at the moment, the concept of the moment, the architecture of the moment is microservices. We’re here to talk about microservices today primarily. One of the reasons that DevOps is part of this is that the two are tightly interlinked, they’re critical. Microservices is what you deliver and DevOps is how you deliver it.

I’m going to spend a little bit of time today talking about what DevOps is and how you can transform your team, your work style, your organization through a DevOps approach so that you can deliver microservices and in the end increase your agility so you can ship faster, ship more often and ship more reliably.

What in the world is this DevOps thing? The easiest way to think about it, if you just take one thing out of this webinar today or at least one thing about DevOps is that it’s extending agile development all the way from the developers out to production.

You take that same agile approach and instead of writing code and shipping it to Q&A you write code, send it to test, send it to production and do all of that every couple of weeks. Or in the case of continuous delivery maybe even every couple of minutes.

John Allspaw from Etsy blew everybody’s minds about five years back when he said they’re deploying, what was it, 80 times a day, blew everybody’s minds. Everybody was wondering how is this possible? It’s possible because he took a DevOps approach and that was what backed up their microservices philosophy of the way they build their products.

The really key part of this is it’s not just iterating from developers to ops. It’s iterating from the people on the other side of the developer to the people on the other side of the ops. In other words, iterating between the business and the customer. If you can better understand what your users actually need and iterate on that you’re not going to spend 18 months doing waterfall style development and then end up shipping the wrong thing at the wrong time.

Instead you’re going to be able to ship sooner and you’re going to be able to ship the things that the market is actually looking for instead of something that has no relation and you have no opportunity to correct.The really key part about DevOps is that you’re connecting your company to the customer and everybody in between including developers, test and ops.

Now I want to talk about the three pillars behind DevOps. The first one is an organizational aspect. The other two are technical. The first one is really about collaboration. You can’t have DevOps if you don’t have empathy and if you don’t get rid of the silos.

There’s no way to be taking the agile or even a continuous delivery approach to how you’re shipping microservices if your teams are on opposite sides of the country, on opposite sides of the world and if you have no way to work together. The really important part of this is getting all of your people in the same room.

Whether that’s a physical room or a chat room isn’t necessarily important, but the important point is to get developers, business analysts, test, ops and everybody else involved all in the same place at the same time.

One example that I heard of at a DevOps status conference earlier this spring they even had marketing in the room too. So you can take the customer feedback, the user feedback and bring that all the way back to the marketers who’d understood how do you build the right product and then develop as you go and actually build the right product. Figure out how to do it and how to ship it.

The first piece of this is really about collaboration. There’s all kinds of collaboration tools out there. Chat is a particularly effective one. There’s in fact a concept called Chat Ops that’s really worth checking out where they integrate a lot of their monitoring, real user monitoring, into their chat room.

You can see exactly what’s happening, what kinds of alerts are popping up, what else is happening in the chat room, discuss it, collaborate around it and iterate on it. Especially if you’re taking a continuous delivery approach to this you have the opportunity to iterate very quickly based on what’s actually happening to your users right now.

The second pillar of DevOps is about automation. You can’t possibly ship 80 or 100 or 150 times a day or really even 10 times a day if you can’t automate everything all the way down. This is something that’s becoming much more common at the code level, at the app level, but the underlying infrastructure is something that’s been a lot slower, it’s late behind.

In fact 21 years ago is when a guy name Mark Bridges released the first version of CFEngine. It’s shocking how old this configuration management idea is when the whole DevOps concept only came up a few years back.

This whole infrastructure is code ideas automating not just the way you deliver your applications, but the way you deliver the infrastructure all the way down so that you can reproduce it, so that you can scale it and so that it’s more reliable and robust.

One great example of the stuff I want to talk about is the story about a company called Knight Capital. A couple of you may have heard of it. A couple of years back what happened to this company named Knight Capital is they lost 460 million dollars in the span of 45 minutes.

They lost it because they weren’t automating their infrastructure. They deployed code to seven of their eight servers by hand and the eighth one was running the old code, running the old code in fact from ten years ago where they reuse the same feature flag for a different purpose.

While seven of these eight servers were doing the right thing the eight one was fighting against them doing things that were exactly wrong. In the 45 minutes it took them to track down that problem they lost nearly half a billion dollars. This should really hit it on for you how large an impact automation all the way down can have.

The third pillar is monitoring. You have to have data if you’re going to improve on what’s happening. I think it was a famous quote from Peter Drucker a while back, “You can’t prove what you can’t measure.” It’s very much true in software. Whether it’s APM or whether it’s live monitoring you have to have the data.

One of the problems however has been that monitoring has been extremely difficult, the experience has been terrible and there’s been a lot of complaints which started this whole monitoring [inaudible 00:09:22] a couple of years back. But finally it got to a point where people decided it’s not that great to just sit around and complain. It seems like it might be a better idea if we actually did something about it.

There was this resurgence of the importance of user experience on the developer side, but also on the operator side because in this DevOps world the two are blending. A lot of developers are doing operational tasks. They’re doing the configuration management to deploy the clouds. They’re doing a lot of the operator work. In some cases they’re even on pager duty when their apps are in production and they have to go fix the bugs and go get woken up at 3:00 in the morning. That’s only in the most modern cases.

We’ve seen this resurgence of user experiences [inaudible 00:10:07] interest and monitoring in part because of the sheer scale of what’s happening now where instead of having a couple of servers you’ve got hundreds of servers or even thousands of servers.

You can’t track all of those unless you have a rock solid monitoring experience because looking at that many graphs just doesn’t scale. You have a much smarter approach, you have to have much smarter searches, aggregation, alerting and even machine learning to have an understanding of what’s going on.

Those are really the three pillars of DevOps. The organizational, one about collaboration, one about automation and one about monitoring so that you can improve upon what you’re doing.

I want to show you some really interesting data from a survey by a guy named Gene Kim among other people, some academics, puppet labs and [inaudible 00:10:53]. He found after surveying more than 9,000 people who were interested in DevOps that high performers were incredibly likely to have not just more technical success, but more business success as well, which is pretty incredible when you think about, “Oh, we’re doing a better job of shipping our technology, so our business is more profitable?” It makes sense in retrospect, but it was not obvious thinking about that.

Or for example the last point about higher market cap over three years. That’s crazy. The technology can become such a differentiator. That’s what’s changing. There’s a famous [inaudible 00:11:26] from a couple of years back called Software Leading the World about how software is becoming critical to the success of every business now. Not just software companies, not just vendors, not just web companies, but tire manufacturers, factory floors, car manufacturers. Software is becoming critical in every one of these places.

Who’s taking this approach? Who’s doing DevOps? Who’s doing microservices because a lot of times the two go hand in hand? It’s no longer just web companies, no longer just the software vendors. It’s finance, it’s retail, it’s manufacturing, it’s government, it’s academia. Everybody is starting to take this approach. There are leaders in every industry doing it.

Here’s some really interesting data. I’m going to walk you through it very quickly. The basic idea here is that I saw this data, Gene Kim walked me through. I went back to him and said, “I’m really curious. Is it possible for big companies to do DevOps at scale or are those examples just outliers?”

We went back, we spliced through these 9,000 respondents, looked at them by how many employees they had and that’s running up and down the left side. The green indicates how many of those respondents were high performing DevOps organizations.

You can see down at the bottom, those are the little tiny startups, one to four people. You’ve got nearly 40% of them that are high performing DevOps organizations. What I found really intriguing is that even as you get up to the highest numbers, the largest companies, that number goes down by about half. It does get down. It’s absolutely harder, but it’s far, far from zero. It’s not 1%, it’s not 5%, it’s not 10%, it’s almost 20% of these companies, people who want to do DevOps, who have successfully made it happen.

It’s absolutely a challenge, but as you have just seen it has a huge impact not just on the way you deliver [inaudible 00:13:23] the way your company works. It’s not often that sitting back in IT, sitting back in development you can think “I’m having a real impact on the company. I’m having a financial impact on the company. We’re shipping better products and we’re being more successful.”

IT is starting to become less of a cost center and more of a profit center. This is a big shift. Instead of people saying, “Oh, let’s outsource everything, let’s fire as many developers as we can” the game is starting to change. The say, “Oh, we’ve got to hire more developers. We have to build more software because it’s so critical to our businesses.” This is a really dramatic shift.

One example of this shift is the way the rules are changing. At the top of this diagram it shows you the traditional way software is built and maintained where dev test, you look at developers, they’re maintaining their owns stacks all the way down. They’ve build out their laptops, they’ve built out their dev systems.

From installing Linux or installing Windows all the way up to the application even though that whole bottom, everything underneath the app, that’s not a differentiator. That’s just a huge waste of time and only happens because you have to customize, because you have to have the latest versions so that by the time you ship the version is current.

Then production or ops goes back and does the same thing all over again, but the problem they run into is, “We forgot to specify the dependencies, they’re missing some. Diversions mismatch, we’re deploying on a different JVM in production than we use in development. We deploy on a different database in production than we did in development” it creates all kinds of crazy issues and it’s really hard to get it right.

This is starting to change. It’s starting to shift such that developers are now beginning to maintain their applications all the way through to production where they are responsible for fixing it out there and they are responsible for getting paged and woken up.

They go on call rotations and frankly it makes a lot of sense, because if you wrote the code you’re the person who’s best suited to fix the code whereas if you call some sys admin at 3:00 in the morning and they say, “Oh, there’s a memory leak in the JVM” you know what they do? They just restart the JVM.

Eventually they just start up a cron job to restart the JVM because by and large they’re not professional software engineers. They’re not experts in the language that the code is implemented in and they have to resort to a hack because what happens is the developers are forced to move on to the next step. They don’t have time to devote anymore.

This is starting to change because it has to change. But on the flipside, operations, it’s not just about more work for developers. It has to go both ways because there aren’t infinite amounts of effort available. Operations is going back the other way. They’re starting to build VMs and container stacks of the infrastructure so that developers have a stable base to build upon without losing control over their systems.

That’s where we come across ideas like this. If you combine the idea of continuous integration and automation you can suddenly apply testing to your infrastructure. You can build your infrastructure just like software, apply real software engineering techniques and take a real approach using version control, you’ve got sys admins learning how to code, not just copy and pasting, but doing legitimate software engineering. Reusing code, using libraries, all these things which weren’t possible just a small number of years ago. The game has changed.

This starts to create the possibility for a composable business, built on microservices. A great example of this is a memo written by a guy named Steve Yegge. Steve Yegge works at Google now. He used to work at Amazon. Here, what was intended to be an internal, employees-only memo on Google Plus or Google employees he accidentally posted it to the public one instead.

What happened was very revealing. We got to learn how amazon worked internally. What he wrote was that Jeff Bezos who runs Amazon sent down a dictate that said any data sharing that has to happen across any organization across the entire company must happen through an API, it must happen through an API.

You can’t email around spreadsheets, you can’t call people up on the phone, it all has to happen through an API. This is the advent of Amazon of microservices where suddenly if you’re a finance department you’ve got to have a microservice available for people to get accounting data, to get a procurement data.

It changed the game because it took away all the leg time. It made everybody more agile because instead of having to go wait for somebody or maybe they say, “Oh sorry, I’m too busy for that” all you had to do was go hit the API, which makes businesses more agile.

The other point I was talking about was infrastructure. If you look at containerization, if you look at virtualization, suddenly this has become part of the whole automation story as well. A tool like Vagrant will help automate the creation of VMs. Tools like Docker and Docker files in particular will help you automate the creation of containers.

This is really changing the game because now you can have the exact same environment in development and in production because it’s created using the same tools. The container sits in some other setting. It might sit on a developer system running Linux. It might sit on a production environment running a different version of Linux, a different distribution, different kernels. It doesn’t Matter because inside you’ve got the same exact environment and that’s the key. It eliminates a whole class of errors.

You can see that in the past few years, if you look at Stack Overflow which is a pretty popular discussion forum, container traction has gone nuts. Look at that green line for Docker. It’s one of the fastest growing technologies we’ve ever seen.

I want to dig into microservices a little bit more to define more clearly what they are. What they’re really about is separating business concerns into different services that you can access with a REST API. The challenge with microservices is really that you don’t split down everything to the tiniest possible bit. That would be absurd.

Generally you’re looking for moderation. You’re not going completely monolithic, you’re not going completely microservice based, as small as you could possible go. You don’t have a service that has one function call that serves one little piece of information.

You have a service that’s logical. This is the finance service. It’s got an API with a bunch of different functions for getting at purchases, getting at orders, getting at past history, all these different sorts of things. You think about it at a business level or you might have a search microservice that takes care of search data for you. It hosts all the indexes. You say, “Hey, I want a search,” it gives you back the results.

That’s the kind of thing that a microservice looks like. This allows you to take a resilient approach to the way you’re building websites where one example of this is Etsy. They do have a search microservice and they can search their websites on the fly on the client side. If the search microservice isn’t there the whole website doesn’t go down.

This is how you’re changing, game changing the user experience. If the search isn’t there you don’t get a search bar. The rest of the website works just fine. This is the way that microservices really change the way you deliver software.

Another example is Netflix. These are easy ones, these are unicorns, but there are other companies doing exactly the same thing. If you imagine Netflix, if you look at the user experience right now you see row after row after row of recommendations.

Sometimes one of those rows isn’t there. No big deal. Your movies based on John Wayne, your movies based on Bruce Lee, they’re not there. It’s a little frustrating, but everything else works. That’s the way microservices really change the game.

One thing and one of the reasons we’re here today is to talk about how PaaS plays into microservices. The idea of platform as a service has been around for a while. It seems, from my point of view at least, I’m shocked that they haven’t already taken over just because it creates a very easy experience if you think about the advantages the middleware gave you.

PaaS is essentially venue middleware as one of my colleagues has written and it gives you the same kind of advantages. It gives you all the framework you need to do some of these things that are architecturally difficult, like microservices, like DevOps. If somebody just hands you that pipeline and says, “Hey, you’re ready to go” it makes a lot of sense to me.

We’ll see if its time has arrived. You certainly look at examples like Cloud Foundry, one of the reasons we’re here today. It’s gotten a lot of traction recently. It’s really interesting to me to track this and to see, all right, is this really enabling what it’s supposed to enable and where is it going to go from here.

To wrap it up here’s the view from 40,000 feet. Everybody always says a view from 20,000. Here’s 40,000, it’s really high level. Just to put this in one simple picture for you, how do you go from agile to CI to CD to DevOps and what steps does that cover? I’ll just put that up for a second, move on.

It’s sort of a wrap up slide here. These are the kinds of things you can do to minimize the risk and maximize the agility every step of the way. You take a microservice based approach, you’re using CI, you’re using CD. From a deployment perspective you can do rolling updates, you can do continuous deployment, take a resilient engineering approach.

We don’t have time to dig into all of these, but if you take all of them you’re able to deploy smaller bits of code to smaller numbers of people and roll back small pieces easily which is going to make me more agile and which is going to top any failures to be minimized.

You’re going to minimize the number of failures, you’re going to minimize their impact. That’s the advantage you can gain from taking a modern approach to IT including microservice, including continuous delivery, including DevOps. With that I’d like to thank you for your time and I’m going to attempt to pass off the torch to my colleague Matt.

Matt: All right, fantastic. I’ll just keep charging right on ahead with where Donnie left us all. Part of my focus is going to be just to talk about how we think it pivotal that Cloud Foundry and then also some new efforts that have been going on within the Spring Engineering team contribute to enable enterprises to take a step into DevOps and also microservices.

I usually like to define a lot of what I’m talking about upfront, but Donnie took care of that for me. I want to go right to this picture here which is the crux of where I see the story unfolding. We talked about continuous delivery. We talked about DevOps.

We talked a little bit about just the need to innovate, the need to go faster and essentially this idea that we need to be able to get feedback from customers and pour that back into our flow and constantly be iterating as quickly and as agilely as we can.

In addition to that do a few things differently such as the move away from a focus on scaling vertically, which was how we tended to do things with traditional middleware to scaling horizontally and the transition that we’ve taken from physical hardware to virtual hardware and then now to containers.

Then also this issue around mobile devices. The clients that are interacting with the services that we’re building are no longer people sitting at a desk using a computer that’s tethered to that desk, but they’re people walking around with supercomputers in their pockets and interacting with services basically anytime and anywhere. You just have a much larger number of people that are hitting this set of services that we’re building and the underlying infrastructure that’s intended to support that.

I take all of those ideas and I pour them into this funnel. I look at the results from the infrastructure side and from the application side and even though mine’s flipped top to bottom from what Donnie’s view looked like, but you end up with this two phased approach again.

You have an infrastructure platform whose job is to support delivering this type of innovation and you’ve got a platform team that’s responsible for operating that. To us that means Pivotal, Cloud Foundry. We’ll take at great length about what that brings to the table.

But then also … this started for me, people were asking me, “Okay, I want to develop applications for a platform like that because I understand the benefits that I get from it, but what does an application that’s intended to run on a platform as a service product like Cloud Foundry, what does that look like?”

It’s probably not this, as you see here at the bottom, this monolithic application that I’m used to building where I might have some modules, but eventually I bundle all that up into a single deployable unit. By bundling that up I also tend to bundle my ability to change things together.

If you have multiple different lines of business that want to do different types of things at different times you make it more difficult by the fact that we have to get everybody to line up and meet quality checks before we can actually go forward.

Moving that into more of a microservices approach where we have decentralized coordination, we have autonomous themes building autonomous services. What that looks like tends to jive very well with Cloud Foundry. I think we’ll see a little bit of why that is as we go forward.

One of the things that I want to stress is that platform as a service brings a lot to the table in terms of features and benefits, but they’re not free. It’s not just that they’re not free in terms of you go out and you pay your software, like big cost, to get products up and running and have support, but also you pay for those features in terms of constraints that you’re going to start to willingly embrace in order to be able to leverage those benefits that you get on a platform.

You take a look at Cloud Foundry and platforms like it you can also compare this with say Hiroku who wrote this manifesto around this idea of a 12 factor application. The 12 factors basically boil down the idea in small chunks of what does it mean to build an application that’s intended to run on a cloud platform.

Many of these features are about just being very explicit about how you want to realize your dependencies rather than implicitly realizing them from some infrastructure that was set up by somebody at some point in time. Then as you move throughout environments maybe things change, maybe things have drifted around and we start to run into quality issues.

Not expecting dependent libraries to be somewhere in environment. Not expecting config files to show up inside some application server by doing things very basic like injecting environment variables in a controlled way through a platform and building these stateless processes so that if one instance of a process dies rather than trying to go and fix things we can just recreate that from scratch from an immutable image and we realize a successful running process again.

Having absolute parity between all of our different environments, not building different ways for different environments, also not deploying different ways to different environments. We don’t have time to go into all of these, but that’s just a taste of what’s there. If you go out to 12Factor.net you’ll be able to read this in probably half an hour, it’s very quick.

You take that and you start to think about microservices. One of the things you realize is that microservices tend to be 12 factor apps when you look at how they’re built, except you’re layering on top of the fact that it’s a 12 factor app that it has a specific purpose.

That it’s supporting a particular business ability within a particular bounded context of the business domain and you have a very dedicated team that is taking that from business requirements all the way to running that microservice in production as we talked about as you’re doing DevOps.

You take all these ideas and you realize this is very different from how we tended to do things. As I start to think about how do microservice and PaaS come together it really reminded me a lot of how it looked like when we were trying to adopt agile several years ago.

You go back about 15 years and extreme programming is starting to become the thing that everybody’s talking about. Extreme programming was presented at the 13 Practices. We were told, “You don’t just come and pick and choose what you want and go do that because you’re likely going to fail.”

“We actually picked these 13 practices for a reason and it’s because the network of these practices and how they interact actually made the whole stronger than any one of these parts. If I were to reach in and take something like simple design or pair programming and yank it out of this network all of the sudden the system would be much weaker.”

That analogy really worked out for me when I started to think about Cloud Foundry and microservices. Cloud Foundry with its optimization around 12 factor apps it forced me to adopt some constraints. Microservices fit very tightly into that.

Microservices bring a whole set of other benefits to the table, but they also bring quite a lot of cost and that’s what we’re getting ready to get into. As it turns out one of the things that platform as a service and Cloud Foundry in particular is designed for is addressing exactly the weaknesses that microservices have so that we can start to realize their strength.

In a sense you have again, this synergy, this symbiotic relationship between the two where one doesn’t necessarily directly imply the other, but when you bring them together you get this network effect again where the whole is much greater than what the individual parts turn out to be.

What are the costs? There’s been a lot of writing done in this space. One article that really stuck out to me back in April was written by Ben Wootton who’s a DevOps and continuous delivery consultant in the UK. He wrote this article that gathered a lot of attention called Microservices – Not a Free Lunch.

Back at our Cloud Foundry summit earlier this year I actually built a whole talk just around addressing the points in this article. Basically what he did was he boiled down, here’s all of the stuff that you’re going to have start doing now as you adopt microservices.

He points out that operations becomes even more critical in this space. As you start to really adopt the philosophies of doing things like microservices having their own data stores and adopting polyglot persistence where you use different tools for data stores and different places because of shapes of data or query patterns that you want to adopt.

You still have to be able to go operate all of those technologies and you still need to be able to stamp out consistent application environments. It’s not just stamping out a few, but as you go to microservices it’s tens and hundreds now of environments that you need to keep consistent.

Being able to develop that DevOps skillset across the entire business to keep all of that operating is something that you very much have to be able to do. But then you also run into all sorts of additional integration concerns. It’s no longer a compiler that’s going to help you integrate with other services.

You now have these APIs that you have to start to think about versioning and deprecation and how you make sure that contracts are met. How do you deal with duplication of effort because sharing libraries across teams can become more interesting.

Not the least of the problem is the fact that you’re now starting to build much more distributed systems and all of the complexity that comes into that and asynchronicity that starts to become one of your primary interaction patterns. Getting that right is also in many ways harder than doing synchronous interaction.

Just the overall testability challenges that you run into as you’re integrating these larger distributed systems. You take that and that sounds a little bit hard to swallow. I thought it was very interesting that not too long ago Martin Fallor came out with his own list that actually lowered the bar somewhat, but that also made that bar an absolute minimum requirement to go forward.

It was this idea that if you even want to get on this ride you have to be this tall. If you can’t rapidly provision consistent environments to deploy applications that automation all the way down throughout dev and production you’re not going to be able to do microservices at all.

If you can’t actually do monitoring very well, all of the topics that Donnie touched on in the monitoring space, if that’s not a core competency of your dev and organization right now it’s going to be very difficult to get this done.

It’s not just monitoring an individual app. You think about monitoring your monolithic application that may be you deployed before. I can hook a monitoring tool up to that and get some data fairly easily, but now I’ve got hundreds of microservices. How do you monitor hundreds of things?

It’s not just hundreds of things, but you’ve now got this distributed system that’s made up of all of these different pieces. How do you monitor that system when you can’t really point at that system and say, “Hey, this is the piece that’s broken” when really that piece that’s broken is just a logical piece that’s composed of several different services and how they interact with one another. You have to be able to address challenges like that to succeed.

Then being able to rapidly deploy apps. When I make a change how fast can I get a one line of code change into production? If I can't do that very quickly it’s going to be very difficult to get a lot of value out of microservices because and for the most part they’re all about rapidly changing. If you can't change rapidly anyway you’re not going to get a lot of benefit. You’re going to get a lot of hurt out of this. Again, that underlines DevOps culture running through all of this.

If you start to think about the benefits of microservices, how they enable us to do continuous delivery by decoupling change and being able to have small product teams start to produce value without a whole lot of overhead of centralized governance and coordination.

There’s a couple of different things that you’re going to have to be able to deal with at the high level. You’ve got organizational concerns that you’re going to need to be able to solve. In many cases the way you solve those are going to be very context dependent.

But then you also have a lot of technology concerns. That you’re going to have to be able to solve. The cool thing about the technology concerns is they turn out to have a lot of patterns associated with them. In many cases those patterns can be implemented by using a set of boiler plate implementations.

That’s why you start to see a lot of tools that worked very similarly, but have slight differences from one another, populating the space and solving these various automation concerns.

The way that we like to think about this from a Cloud Foundry and Spring perspective is that what you really want is a platform that encapsulates and provides as a service to product teams that are building microservices the substrate that they need to create all of these boiler plate implementations and allow those teams to focus on delivering the value associated with that particular business capability.

You have some infrastructure level concerns that you need to be able to realize and we tend to realize those with features that are built into the Pivotal Cloud Foundry platform. But you’ve also got some application level patterns that you’re going to need to be able to employ. We’re starting to realize those with the Spring Cloud project that was announced at Spring One a couple of months ago and is now on its second milestone of its 1.0 release.

Platform features, what do we need to be able to do? We said we needed to be able to stamp out application environments quickly and consistently. If you think about Cloud Foundry we have the concept of pushing an application. You give me for example an application WAR file, you push that to Cloud Foundry. Cloud Foundry takes that, analyzes it and says, “Hey, this is a WAR file, I need to deploy a Java environment for that.”

It stamps out an environment with a consistent JVM, a consistent in most cases Tomcat container, although you have flexibility there. If there were any monitoring services that were specified it will automatically provision the monitoring agents for those services and then take your application bit, put them in that environment and spin up again, we talked about containers, a Linux container for that service to run in.

It will do that the same way for every Java application you deploy. But if you want to deploy a PHP application, same deal, except specific to PHPs concerns, or Python or Ruby or Node JS. We have that same flow that stamps out those consistent environments.

Then as you get those services deployed you can start to say, “All right, I need more instances of those.” You run a CF scale operation and specify the number of instances and the platform takes care of that for you. Then Pivotal Cloud Foundry has an auto scaling service that will start to allow you to make some decisions around automatically scaling applications up or down.

What happens if one of your applications fails? We have a service that falls over. Cloud Foundry has a health management component that is watching those services. As one of your services start to get unhealthy Cloud Foundry will automatically destroy that application instance and restart another application instance in its place.

We talked about data services. You want to be able to use Redis and Cassandra and Postgress and Rabbit MQ and MemCached and other services in the environment with all of your applications that you’re building. We’re baking in the operational knowledge to be able to run those services into the platform itself so that when you come to the platform and when you say, “I need Cassandra” it’s as simple as CF Create service Cassandra and you get a Cassandra database for your microservice that you’re building.

Then the monitoring concerns, being able to get all of the information that we need such as CPU, memory, disk, basically, but also above that things like request latency, request volume, number of success HTTP response codes, number of error HTTP response codes. Doing that and then over various points in time looking at percentiles.

All of that monitoring being built into the platform and immediately made visible to you through two different means. One, this thing that we call the Fire Hose where all of the applications on the platform emit all of that data. You can plug into that Fire Hose and get visibility and to absolute everything that’s happening on the platform right now or being able to go into a web console as a developer for my microservice that I’m running and get that data visualized for me in place.

That’s the platform side. What about the application side? You start to build distributed systems. You start to run into things you need like how do I distribute and version configuration consistently across these microservices? How do I know what microservices are out there?

Can I have a feature that will allow microservices to registers themselves with a central depot so that another microservice can come out and say, “Hey, I need to talk to the inventory service, can you help me do that?” and that service registry helps me do that.

Being able to make intelligent routing and load balancing decisions to other applications either running on the platform or outside of the platform at the application side so that those decisions are made with the most current knowledge of what’s going on.

Then one other point I want to touch on is this idea of fault tolerance. You have a distributed system, you have more things floating around, you have more things that can fail. How do I prevent those failures from cascading across the system?

We’re baking a lot of the solutions to those problems into what we call Spring Cloud. What Spring Cloud is at its essence today is a wrapper around a lot of the battle-tested implementations of these patterns that were written by Netflix and run in production by Netflix and that were open-sourced.

What we’re doing is taking those technologies an modifying them a little bit to make them work very well on a Spring programming environment and then wrapping a consistent programming module around there so that Spring developers can actually leverage these in a very natural way.

Then make sure that also these things tightly integrate with Cloud Foundry so that where they need operational support that support is provided very cleanly by the platform.

We’re not going to demo this in this setting, but if you go out to this YouTube link what you’ll be able to see is a live demo from another webinar of a set of microservices. In this case it’s a very critical system that will tell all of my customers where they can go get coffee.

These microservices are wired together using all of these different boiler plate patterns from Netflix, wrapped in Spring Cloud. You see service discovery and Registration from Eureka, that’s a Netflix technology, a configuration server that was written by the Spring team.

A Hystrix dashboard to understand, “I have these circuit breakers now protecting all of my different service interactions from cascading failures. Give me visibility and monitoring into those interactions between those components that I’ve talked about, but all of that running on this platform substrate of Cloud Foundry so that these applications have a place to live and they have a very clean way to interact with all of these services.

Just to wrap this up on my side very quickly, I think it’s a very interesting observation that another colleague of mine from ThoughtWorks, Neil Ford made. He said that microservices were the first architectural style that was developed post the continuous delivery and post this DevOps conversation.

We’ve had these conversations going on for several years now. You take teams that have continuous delivery and DevOps as an ideal and you tell them, “Okay, let’s go build software. What types of architectures are popping out?” Consistently and repeatedly it’s been this microservices style that we’ve been talking about.

I think one of the primary reasons for that I get from another quote from Neil where he says that architecture’s abstract until it’s operationalized. Why is this different from the service oriented architectures of the past? You start to think about the way traditional SOA is typically realized.

There’s this big enterprise service bus at the middle of the architecture and we have everything plugging into that bus. It’s governing policy and interaction and routing and all of the various concerns. If I need to move something in my system invariably I’m going to have to touch that bus.

It looks very nice on a whiteboard and conceptually it’s understandable, but then you take that and you try to operationalize it, you try to do continuous delivery with it you realize that it’s just as inflexible as the monolith that we had before because we traded a monolithic web application if you will from monolithic enterprise service files.

One of the things that’s at the heart of microservices is this idea that operationalization is the first place concern. That takes me to my statement that architectures that aren’t operationalized, they really truly only exist effectively on whiteboards.

You try to find and effective continuous delivery based architecture that’s not built around these ideas. It’s going to be very difficult or impossible for you to find.

Ultimately, what we’re really trying to do here at Pivotal from a cloud perspective is build out the pieces that you need to create one of these operationalized architectures by giving you the tools to run an infrastructure platform to deal with the boiler plate patterns that you run into with distributed systems with an end towards building effective small business capability focused teams around these microservices so that you can ultimately get to the point where you see the innovation circle behind all of this. Being able to move faster, but at the same time being able to move very effectively and faithfully at the same time. That’s all I brought today. I’ll just go ahead and hand it back over to Peter.

Peter: Hey Matt, can you hear me?

Matt: I can hear you great sir.

Peter: Okay, that’s all right. I guess I was on mute. Great, thanks for that. A couple of questions from our audience. I think this one is for … you can leave up the action slide, that’ fine. I think first question was from Kenny Sarbresy. He asks, “I don’t understand why microservices enable that sort of Etsy example of a search.” Donnie, I think this one is for you because it came in during your section.

He says he doesn’t understand why microservices enable that Etsy example of search. Can’t the UI simply be coded to do that without a microservice backend? Donnie, I think he was talking about your example where some parts of the UI fail because they lead to microservice and the rest of the app keeps on going or maybe that wasn’t your Etsy example. I’m not sure.

Donnie: I’ll talk through that a little bit and try and throw a little bit more color on that. If you think about, and this isn’t the frontend problem primarily. This is the backend problem, which I think is the part that we didn’t really get into. If you think about the backend side, if you don’t have a microservice architecture you’ve got a big monolith sitting there providing a web service that you’re then hitting the API because you construct your frontend.

If you’ve had a bug that you deploy it in your search component of that monolith the whole monolith falls over. You get back nothing when you hit the API. What’s the client side JavaScript going to do when it gets back nothing? You’re just going to have to throw up a 404 or something.

Whereas if you’ve got a microservice based architecture on the backend and part of it is gone you can have the client side be a little bit smarter about it and you can still provide 90% to 95% of the overall experience just without search.

Peter: Got it. Cool. Kenny, if that clears it up for you great. Feel free to clarify in the Q&A window if you like. Let’s see, next question. Here’s our favorite guys. This one comes from DKang. He says, “Is there a way from an architecture point of view to refactor a legacy system or like a traditional monolithic application into microservices?”

Matt: You want me to take that one Peter?

Peter: Sure, yeah, sounds good.

Matt: I’ve told this so many times that I think I’ve got it baked down into an answer that’s short enough for where we are. I really think about this as a very basic sort of three step process. The work is not going to be basic, but the high level approach that I take to this is basic.

I realize this from seeing this from seeing this actually happening in the wild. Invariably you’re almost always going to start with a monolith. You either have a monolith now that you’ve already built, or even if you’re starting new you’re probably going to start with a monolith anyway because it’s easier to get started that way. Eventually you’re going to get to a point where it’s really cost-effective to move towards microservices.

The first thing that I would tell a team to do is, “Okay, let’s draw a line in the sand” and say, “From now on any net new functionality that we’re building we’re not going to attach it to the monolith anymore. We’re going to start to build out microservices around those capabilities. Then we will build an interaction level between those new microservices and the monolith.”

It can be think of as integration layer, I’ve heard it called an anti-corruption layer. There’s many different ways you can label it, but it’s a way for the monolith to communicate with functionality that it expects to be there and for the microservices to communicate back and get things that maybe they need because we haven’t fully decomposed the solution yet. You get good at building new functionality this way first so that you know what it’s like to build microservices.

The next thing you do after you feel like you’re effective at building out new microservices is you start to go back to the monolith and say, “Okay, what needs to change the most? Where is most of the innovation needing to happen? Where are the change requests coming from the business? Why there?”

If you think about it, the main reason that we’re really wanting to go towards microservices is that we’re wanting to enable us to innovate and change more quickly. The parts of the monolith that don’t need to change, we’re not going to get much benefit out of refactoring those out. The parts that need to change all the time do.

Now that we’ve gotten good at building microservices by building new things let’s go find the functionality that is most volatile and start to factor that out into new services. Then again, build that anti-corruption layer, integration layer between the monolith and those microservices.

Then once you’ve done that first piece that’s most volatile then go find the next most volatile piece and move that out and slowly work your way through that methodically. As you’re doing that you’ll start to see, “Hey, these things need to move around a little bit. Maybe this should move from this microservice to this microservice.”

You’ll run into situations like that. There’s no easy solutions there, but that’s the path overall. Then you’re going to have one of two possible outcomes in my mind. One possible outcome is you’re done. You’ve factored everything out. Everything’s a microservice now and that’s actually probably the least likely concern.

But you get there, then you can delete all these integration layers and have things start to draw directly to one another or you’re going to reach a point where the cost of extracting anything else out of the monolith outweighs the benefit.

This is probably where most of us are going to end up. There’s going to be some code out there that the technical challenges and risk associated with separating that from its original home just isn’t worth it. There might be some piece of monolith that is always left behind and runs in sort of a legacy way.

Now, if you get to where it’s too expensive to run that then obviously the cost benefit of moving it out is still there. But it’s likely that, I get a quote from my colleague Andrew Schaeffer. He said, “It’s probably true that certain legacy services are going to run until the heat death of the universe.”

That’s probably somewhat true for all of us. We’re probably going to end up somewhere in the space between everything being refactored and nothing being refactored. That’s going to differ from context to context. That’s my answer to that question.

Donnie: I’ve got a little bit of color to add from that. There’s two points that I want to make. One is that I totally agree with Matt’s point about areas that are changing quickly, breaking those up. The other area worth breaking up is ones that are high risk components, either because of security requirements or because they’re very bug-prone components that tend to take down the whole monolith on a regular basis.

Those are two examples of where it makes a lot of sense. It’s a lot cheaper for example if you’ve got the part of your code base that handles credit card information, that handles social security numbers or whatever. Break that out in a microservice is a lot less [inaudible 00:56:20] to do. You’re able to reduce the surface area of the attack.

Then the other side of it is if you’ve got this buggy code base or at least a buggy component, maybe it’s the oldest piece and you’re able to break that out and turn that into a microservice. Then you can make the product that you’re delivering to your users a lot more reliable overall. Even if that piece continues to crash at the same rate that it did before it’s not taking everything down.

The second thing I wanted to add is there’s actually a fantastic book by a guy named Michael Feathers that’s called Working Effectively with Legacy Code. What it’s about is putting legacy code bases under test, but it’s got a lot of application to how you break things into microservices about how do you insert artificial interfaces, how do you refactor most easily in such a way that you can put things in our test. Really, so many parallels to pulling things out on a microservice. I would definitely recommend checking that out if you’re curious about more technical details on how to do that.

Matt: Yeah, those are really superstitions. Thanks for adding that in there.

Donnie: All right, do we have any more questions?

Peter: Yeah, sorry, I was on mute. Keith Hoops had a similar question, which I think we just answered. He asked “What’s the first step towards a gradual implementation of DevOps?” or in other words how do I become tall enough to ride the microservices rollercoaster?

I think we just did pretty good justice to that. Is there anything you’d want to add in terms of the more DevOps-ed side of going from, “Hey, I have a traditional monolith and a more traditional non-agile development process.” Any first steps on that front that you want to talk about or do you feel like we just covered it?

Donnie: I think it’s worth addressing a couple of things quickly on the DevOps side. How do you get started if you’re an organization that’s waterfall? The first shift is agile, no question about it. My approach in agile is very much not process driven. You don’t want to become a slave of the process. You want to build the product.

However you can make agile or scrum or kanban or whatever you’re doing work for you take those parts and do them. Maybe pair programming doesn’t work for you. I don’t have a little bit of a disagreement with Matt on the pieces that you can pull out. That I would just take what works effectively in your organization because everyone’s going to be a little bit different and good.

Then the next step is it depends on where you’re coming from. If you’re coming from the development side you have a lot more control over the development pipeline then you do over the operational pipeline. It’s a lot easier chances are for you to start doing things like CI. Start playing with Vagrant in terms of doing automated test deployment than it is for you to get into the production side.

Whereas if you’re coming from the operational world it might be easier for you to start taking an infrastructure as code approach and improving your monitoring than building a CI and CD pipeline. It depends on where you’re coming from, but you can start piecemeal by adding piece by piece and moving toward that continuous delivery approach.

The hardest part frankly is the culture aspect because this is where you have to get top-down buy in for the most part to do things like reorganize teams, move people around and so on because that’s going to require either new job racks or shifting teams around. People are going to be losing effort apparently until you’ve made the investment and seen it work.

Peter: Got it. All right. Let’s see, The Veck asks, “Could you please comment on how some companies that are using microservices have implemented business analytics in a polyglot database environment?” Can you hear me?

Matt: Yeah, whom are you directing that one at?

Peter: Good question. I think that’s one at Donnie for now and then Matt if you want to add onto it feel free.

Matt: Certainly.

Donnie: I guess I’d start up by saying we’re definitely seeing a lot of companies moving toward “right tool for the job” approaches to building things, polyglot everywhere. It used to be polyglot languages, polyglot toolkits. Now we’re seeing a lot more of the polyglot database trend as well.

People are saying, “My Sequel or Postgress Oracle isn’t enough for me. I need a key value store. Maybe it’s Cassandra. I need a faster memory database. Maybe it’s Redis. I need a document store. Maybe it’s Mongodb, maybe it’s TouchBase. Who knows?”

We’re seeing a lot more of that polyglot database approach. How does business analytics fit into it? It fits in very much in the way I was talking about monitoring fitting in. Because how do you make the DevOps shift Matter to the company? Easy, monitor things that Matter to the company.

You don’t just monitor uptime, CPU use. You monitor things like successful transaction. How many people manage to take items in their shopping carts and check out effectively? That’ the thing you have to be monitoring. That’s I think where business analytics comes into it.

In terms of where that ties into polyglot databases specifically I guess my perspective would be more that you’re putting the right kinds of data in the right kinds of data stores and then you have to choose analytics solutions that are able to effectively work with all of them, particularly when it comes to structured versus unstructured. You have to have a tool that will handle both. Matt, do you have anything to add to that?

Matt: I’ll throw on top of that just something that I have played with myself in practice, but I haven’t necessarily seen an example of this in the wild at a customer yet. But this idea of starting to merge the ideas a little bit of microservices and polyglot databases with this idea of the lambda architecture.

Being able to do some advanced patterns around as data starts to change in all of my bounded context of my microservices, not just persisting that to the local data store that the microservice is managing, but also publishing events to some bus.

Whether that’s RabbitMQ or something else that also has those data domain changes that are taking place and then having a central store that’s grabbing all of those events and writing them out append only style so that I can then start to construct cross-microservice views of the data and use some of the lambda architecture concepts of building up batch use that answer different questions that cross microservices boundaries. All that’s being done out of band from the microservices themselves.

It kind of flies in the face a little bit of the Yagge memo idea that everything’s got to go through an API because now we're building this offline back channel that’s able to in a read only way see what’s happened inside of these other services. But I think it’s an interesting thought experiment of what it would be like to take these two ideas and bind them together.

Peter: Excellent, okay. Keith Hoops asks “Many corporations, all that I’ve worked for, have third party applications running on different platforms, specifically .Net and Java. How can Cloud Foundry be used in such an environment to help with DevOps?” I think this is a softball for Matt.

Matt: How can specifically third party applications run on Cloud Foundry or how to integrate with them?

Peter: I’m not quite sure.

Matt: I might take it that direction and if I take it the wrong direction you just shoot a follow up out there. I think of this a couple of different ways. The first way that I think about this is, okay, I’m probably not for the most part going to try to take off-the-shelf software and run it on top of Cloud Foundry.

I’m going to run that where it was expected to be run by the vendor that sold me that piece of software, but by golly I’m going to want to be able to integrate with it with the applications that I’m building. Some of those applications might be Java. Some of them might be .Net. Some of them might be whatever other language.

We have a couple of different ways that we can integrate with third party application services through our service binding capability. One is what we call user provided services which is just basically taking a key-value pair set of data such as where the services lived, what are the credentials I used to connect to it and then binding that into my application environment in a direct wire sort of way. The application doesn’t know I’m talking to hard-wired credentials, but the platform’s mediating that for me.

Then I can go a little bit further and use this managed services concept where we have an API called the Service Broker API that encapsulates five different sets of operations. One is what’s a catalogue of services that this broker makes me aware of, but then being able to create and delete instances of that service and then being able to create and delete credential sets for that service.

I can take that API and implement it in whatever language I like and use that to make calls into that third party service and do whatever makes sense from a create me an account or create me some space in that service that I can interact with and then provide me access to it, again, through those environments binding credentials that we find on the application side.

That’s one way to look at it. Another slice, and I’ll spend a lot less time talking about, is that we’re starting to see an awful lot of traction with ISDs around actually shipping their products to run on top of Cloud Foundry in sort of an OEM fashion.

What you’ll start to see I think is more and more vendors not just shipping you third party products, but shipping you third party products that are running on top of Cloud Foundry themselves and then they’re being very natural ways to integrate between the Cloud Foundry environment where you’re running your apps versus the Cloud Foundry environments where these other applications that you say purchased off the shelf are running.

That’s something that’s starting to evolve and I think we’ll see more and more of as we start to go into the next couple of years. That’s my take on it. Hopefully I answered your question, but if not feel free to chime in again.

Donnie: I’ll just add one quick point to that too. What Matt said is basically the same thing we end up recommending regularly, which is doing forklifts existing enterprise up to the cloud because what ends up happening in short is that you get all of the downsides and none of the benefits.

Peter: I’m not going to say his name correctly. Dieter, Dieter. He asked, “How fast is rapid development? Are we talking hours, days, weeks?” Matt, what do you have, what do you say there?

Matt: I can throw out the typical consultant answer which is it depends.

Peter: Yeah, I know.

Matt: It’s probably not going to be very satisfying. I think what it really boils down to as being able to go as fast as you need to be able to go. You start to think about the Flickrs and the Etsys and Google and Amazon. You start to ask them, “How many times you deploy something to production every day?”

It really is on the scale of hours or even minutes. That may not be where you are. What it definitely is not is us releasing software every six months or us releasing software every year or year and a half.

We want to be able to get to where as we see changes in the market or changes in our customer’s behavior that make us want to change the code that we’ve written, being able to make that change as fast as we can realize A, what change do we need to make and B, how do we write the code and make that change and C, actually writing the code and deploying it.

We want to remove all of the barriers to getting that. That’s probably going to get you down to weeks at most, in some cases probably days depending upon just how big of a change that is.

And I’ll throw one more thing on top of that which dovetails nicely with the analytics question, what we’d really like to be able to get to. This is where you start to merge in what is … Pivotal talks a lot about cloud, but we also talk a lot about data, how do those two things start to play together.

It would be great as we start to see our customers doing things in our application we can start to in real time analyze that behavior and then pour that back into the apps and have the apps start to change their behavior at runtime based on the analytics that we’re seeing.

A good example of that, I’ve seen a couple of different case studies recently around just about changing supply chain behavior at runtime based on changing behavior of customers or suppliers in the field and not actually having to rewrite those algorithms and redeploy them, but actually be able to intelligently make decisions about that based on the data as it’s analyzed in real time.

That’s ultimately I think where … Then we’re not talking about deploying code. We’re actually talking about, “Hey, the code’s redeploying new behavior autonomously based on stuff that it’s realized about what’s going on in the world.”

There you’re starting to think about, “Hey, we can start to respond in seconds, not just minutes or days or hours.” I think where that makes sense, that’s really the goal that I’d like to see us be able to get to. Wrong answer.

Peter: No, it’s fine. Gosh, we have a ton more questions. Unfortunately we’re not going to be able to get to all of them. However, if we’ve not been able to get to your question we will follow up with you directly at the email you used to register for this webinar. We’ll make sure to save that question and answer and we will get to you answers.

Geez, a lot of questions, guys. This is clearly a hot topic. I guess with that, just to wrap up a little bit. I wanted to just mention again that the event is being recorded. We will post a replay to the Pivotal.io website as well as the slides, they’ll also get a slide share.

The links to both of those resources will be in your thanks for attending email. If you’re looking to share these with your colleagues or you missed a part and you want to replay it or what have you, you’ll be able to do that fairly soon.

Let’s see. Also, wanted to ask you to join us on our next webinar. We’re going to be talking about a mobile first world. It’s the next webinar on the Pivotal webinar series with Pivotal and VMware. That is on December 3rd at 10 am. Please feel free to join us for that. Registration information should be on the Pivotal website and we look forward to seeing you at that topic. I should mention that the webinar replay and the slides will be in the resources section of the Pivotal website.

Last thing is if you’d like to take some of this, try playing with some of the stuff that Matt and Donnie were talking about today, a great place to do that is at Pivotal web services. That is our hosted public Cloud Foundry. You get a 60 day free trial before you get asked for a credit card. A great playground to try some of this stuff out.

Also, of course, if you’re looking for trying and playing some of the early milestones of Spring Cloud you can certainly do that localhost as well, but Pivotal web services is a great place to also to try this stuff out in the public cloud. With that I just want to ask Donnie and Matt, did you have anything else you wanted to say before we call it a day?

Donnie: The last thing I’d add is as you can see up on the slide at the moment, if you’re interested in checking out more of our research it’s all open source, it’s all free. We’ve been doing a lot of work around DevOps, around data science and around development, so go check it out. Thanks.

Peter: Awesome. Feel free to keep in touch with us. Follow us on Twitter or check us out on YouTube. There’s great resources for developers. Certainly, there’s also occasionally other information in there as well, but for the most part they’re pretty developer focused. These are great ways to stay in touch.

Thanks everyone. Have a great day and see you at the next webinar on December 3rd. Take care.