For us developers, there has been a lot of change in the infrastructure where our apps will run. New to the mix is running in containers in some type of cloud enabled environment. After some basic concepts around containers and running them in production we go full on ALL DEMO, ALL THE TIME! I am going to show how to accomplish your local fast iteration development but now seamlessly deploy that to a frickin’ awesome run time platform a container platform based off of Kubernetes.
Come and see how the future of development is evolving!
For most of my lifetime in the computing world, data crunching and web serving were two very separate worlds. If a web app wanted access to the analysis there was a long process of ETL, DB work, imports and exports, and bribing various network and storage people for the resources you needed. With the rise of containers, orchestration, cheap computing and networking, and over 10 years of people tackling large problems at new scales we have finally come to a convergence.
It is now possible for us to actually use the same hardware, and more importantly, clustering software to converge both types of workloads. We are going to lay out a high level vision of how this can look with Containers, Kubernetes, web servers, and Apache Spark. This can be considered a germ of what we can look to build in the future. We will demo this in action and show this is actually now achievable for mere mortals such as us. This convergence allows data analysis move from custom R or Python scripts on an analyst’s desktop to a front-line accessible web app. At the same time, the analyst can “constrain” the analysis to prevent “statistical overreach”.