Emad Benjamin

There is no doubt that much debate and opinion exists in our industry as to what really is a cloud native app platform! Regardless of what your definition is, it is certain that we simply don't have time to re-write all of the second generation monolithic apps, but yet we must modernize at the same time. With this reality in mind, I begin the discussion of how to build platforms that are both 3rd generation and 2nd generation capable while delivering feasible enterprise application platforms.

As for the cloud native movement, it is important to understand the elements of this rapidly moving phenomenon through our industry, a phenomenon of building platforms, not just business logic software but infrastructure as software. I humbly believe that the drive towards these platform solutions is due to the following fact: approximately half of new applications fail to meet their performance objectives, and almost all of these have 2.x more cloud capacity provisioned than what is actually needed. As developers we live with this fact every day, always chasing performance and feasible scalability, but never actually cementing it into a scientific equation where it is predictable, but rather it has always been trial based, and heavily prone to error. As a result we find ourselves delving with some interesting platforming patterns of this decade, and unfortunately we are lead to believe that such patterns as Microservices, 3rd platforms, cloud native, and 12factor are mainly a change in coding patterns, to the contrary – these patterns reveal a much more deep shift in the way developers view and consume infrastructure. In fact these patterns represent a major change in “deployment” approach, a change in how we deploy and structure code artifacts within applications runtimes, and how those application runtimes can leverage the underlying cloud capacity. These patterns are not code design patterns, but rather platform engineering patterns, with a drive to using APIs/Software to define application platform policies to manage scalability, availability and performance in a predictable manner. In this session I will briefly inspect platform patterns that we built over the last decade, and the ones we are building for the next decade. The main objective of the session will be to layout the definition of Platform Engineering as the software engineering science needed in order to understand how to precisely deploy application components onto application runtimes and how in-turn one should appropriately map the application runtimes onto the infrastructure that it needs. With this knowledge you should be able to more effectively decide when to scale-out (by how many instances), and when to scale-up both manually and dynamically as needed within your software defined platform. Which are key ingredients for knowing how to run 2nd and 3rd gen application platforms at the same time under one platform.

The role of the infrastructure engineers and software developers is very well understood in the industry; however, what happens in the exchanges between these two schools of thought during critical design and implementation phases of projects are often misunderstood in the industry. Now, in order to move towards the next era of Java platform designs and implementations, one should fully understand the nature of this problem by acknowledging the following fact: it is true that half of the world's new Java applications miss their SLA, and almost all of these applications have 2.x over-provisioned hardware. Such cost is deeply hurting our industry, and when CIOs are being told that in 3 to 5 years' time most of them want to be out of the “infrastructure business”, most CIOs seem to think that by getting away from the “infrastructure business” then somehow magically the 2.x hardware over-provisioning problem will also disappear. This is not true, and platform engineering is the exact remedy needed to fix this industry. Attend this session to learn about how new industry and skill-sets are being formed around platform engineering concepts, initially in the session we showcase this industry gap and how to best fill it, but then we quickly move towards showing an actual recent example of how to apply platform engineering concepts to refine a design. The era of platforms is upon-us, from the Third Platform, Cloud Native Apps, Microservices, Virtualization, and to Containers are all key platforming concepts completing the platform engineering story.

In this session we cover various aspects of these platforming technologies, and specifically deep dive into GC tuning techniques and how to best build platform engineered systems; in particular, the focus will be on tuning large scale JVM deployments. Come to this session to learn about GC tuning recipe that can give you the best configuration for latency sensitive applications. While predominantly most enterprise class Java workloads can fit into a scaled-out set of JVM instances of less than 4GB JVM heap, there are workloads in the in memory database space that require fairly large JVMs. We will look at various Java platform scales, some holding a few large JVMs of 90GB heap space, while other platforms are of thousands of JVM instances of less than 4GB heap space on each. We also inspect the extreme vertical scalability the JVM by looking at a recent 360GB JVM that we worked on.

The session will cover various GC tuning techniques, in particular focus on tuning large scale JVM deployments. Come to this session to learn about GC tuning recipe that can give you the best configuration for latency sensitive applications. While predominantly most enterprise class Java workloads can fit into a scaled-out set of JVM instances of less than 4GB JVM heap, there are workloads in the in memory database space that require fairly large JVMs.

In this session we take a deep dive into the issues and the optimal tuning configurations for tuning large JVMs in the range of 4GB to 128GB. In this session the GC tuning recipe shared is a refinement from 15 years of GC engagements and an adaptation in recent years for tuning some of the largest JVMs in the industry using plain HotSpot and CMS GC policy. You should be able to walk away with the ability to commence a decent GC tuning exercise on your own. The session does summarize the techniques and the necessary JVM options needed to accomplish this task. Naturally when tuning large scale JVM platforms, the underlying hardware tuning cannot be ignored, hence the session will take detour from the traditional GC tuning talks out there and dive into how you optimally size a platform for enhanced memory consumption. Lastly, the session will also cover vfabric reference architecture where a comprehensive performance study was done.

In this 2 part session (each part 1.5 hours), I will deep dive into the deployment architectures of large scale Java platforms. I will first set the context of the discussion around what problems exist in our industry before I proceed to lay out the concept of platform engineering and its renaissance among the developer/deployment community today. It is astonishing to see that majority of new application platforms being rolled out today miss their SLA, and 90% of these systems require 2X more hardware than what they actually need in order to run. This is an industry suffering from a double whammy, where you spend 2X on hardware and still miss your SLA; clearly something is completely broken. Now prior to delving into these new concepts, such as Microservices, Cloud Native, 3rd Platform, & 12Factor App, it is imperative to first understand the problem at hand before we apply these deployment architectural patterns. I will layout the definition of Platform Engineering as the software engineering science needed in order to understand how to precisely deploy application components onto application runtimes and how in-turn one should appropriately map the application runtimes onto the infrastructure that it needs. With this knowledge you should be able to more effectively decide when to scale-out (by how many instances), and when to scale-up.

We will conclude with covering various GC tuning techniques, and how to best build platform engineered systems; in particular, the focus will be on tuning large scale JVM deployments and various sizing techniques. While predominantly most enterprise class Java workloads can fit into a scaled-out set of JVM instances of less than 4GB JVM heap, there are workloads in the in memory database space that require fairly large JVMs. We will look at various Java platform scales, some holding a few large JVMs of 90GB heap space – servicing large multi-terabyte in memory DBs, while other platforms are of thousands of JVM instances of less than 4GB heap space on each, typical of web-app deployments. We will also take a close look at an example XYZCars.com, where a microservices approach was designed and deployed. The discussion will cover how to more correctly deploy microservices, without causing fragmentation of scale, and hence without impeding performance. In this session, we take a deep dive into the issues and the optimal tuning configurations for tuning large JVMs in the range of 4GB to 360GB, using the GC tuning recipes that were gained over the past 15 years of GC tuning engagements. You should be able to walk away with the ability to commence a decent GC tuning exercise on your own, and your way to platform engineer your application runtimes to feasibly meet your SLAs.