Scalable, Shard-able, Containerized & Distributed Computing

Joe Alongi
6 min readJun 4, 2022

It all started with a silicon chip and an O’Reilly book and then it became a dream that was a reality. One of those was free the other, well that came at market price, both certainly worth the exercise — given over the last year as I’ve poured over notes, Docker builds, and CLI commands.

Evolving as a programmer means diving deeper into known technologies to meet modern technologies and circling back around consistently to develop a pattern of repetition to turn documentation into knowledge.

The core factors of success I have learned in life, are adventuring within constraints to explore the narrow and reach back to the macro and encompass the understanding by turning them towards each other adversarially.

Silicon Chips As A Development Standard

You often hear the mantra “if you build it, they will come” and amongst programmers, frequently “if you buy it, you will code”. This is the logic behind investing in compute capacity whether on your desk or in the cloud.

When the Apple Mac mini M1 became the talk of the crowd, I explored the idea that maybe a small machine with a fair amount of power could be the focused antithesis that I needed. Programming with a unique computer chip that focused on divided power, would be new to a lot of people and provide me an opportunity to create some useful tweets about adaptation.

Along with the exploration of the silicon chip, I would find that gaming, on a small yet powerful computer, makes visual graphics rendering quite difficult, which means instead of gaming, I more, focused on programming. Dedicating free time to personal development and knowledge had lead me down a path, that we as humans often forget and remember again and again over time.

Graph Mediums With Graph Queries

Graph databases, theories, algorithms, and well visual tools are much alike — segmenting the view of data into relatable slices and sharing the cost of the energy whether compute or cerebral across their collective being.

Specifically, graph databases became an insightful topic upon reading over the neo4j Graph Databases book that was free, online, as an eBook. Leading me down the path of modeling data instead of pushing and pulling, I found serenity in placing parts of the whole where they made intrinsic sense.

Graph queries are very related to the idea of graph databases where interwoven distinct segments applied across topics can accumulate or narrow to provide iterative variations that have distinct usability.

GraphQL solutions to this theory by applying structured queries with flexible mutations to extract requests and provide methodical approaches to relational data. Improving response times, simply, you build requests off of multiple requests, and occasionally reduce the requests, based on components, which cascades as a performance impact.

Clustering and Sharding for Applications and Databases

Modern application development often focuses on the order of operations. As M.V.C (Model View Controller) set a distributed relational pattern to structured data and code, agile frameworks apply qualitative enumeration. Components of higher-order, pass properties, data, and functions to lower-order components, where sometimes these components are present statically or evaluate dynamically.

SaaS applications, specifically user-focused products, are interaction lead mediums and continuously fluctuating in data states, with data, without data, authentication, and perception. If the depth of the user varies so frequently then why wouldn’t the concurrent activity?

Leading a performant, scalable, and portable standardization is Docker. There were some before, and will be many after, but in the instance applicable to my timeline, it is Docker that has been the efficiency model for moving an application from desktop to cloud in a matter of minutes.

Where Docker delivers its greatest impact for monolithic applications is often in Kubernetes where Docker is met with extensible and managed services that are hoisted upon the Linux backbone of virtual machines and brought into existence by the many variances across the field.

Services, Ingresses, Reverse Proxies, and Load Balancers deliver a containerized runtime split across multiple pods ready to be accessed by the global internet. If we can do this with applications, why can’t we do it with databases?

Vitess is my recent favorite of database exploration. Having been an avid SQL-based developer, my familiarity with schemas and relations has allowed me to build relations simply with powerful and extensible services. Sharding these databases across clusters allows for the efficient access of these experiences to be normalized as they ebb and flow through their maturation. With everything at scale, why not scale, to the scale, more to come on that later.

Spring Input/Output Methodologies With Gradle

Fitting enough with all of the flexible graphs, clusters, scripts, and network access is the ability to quickly develop class and interface-based software applications with multithreading. Spring takes Java and provides the backbone of a framework that takes hours to imbue and simplifies its application for a much smoother workflow.

Amplifying the ability of developers is the goal of every productivity-focused agile team. Delivering packages, structuring, wiring, repositories, and connectors with a few clicks, to a few lines of code, makes building software uniform. Spring I/O is to Java as Node.js is to JavaScript and together they are a performant stack.

Building with JavaScript and Java in two forms and connecting them over Restful & GraphQL services creates a dynamic and literal duo that inspires the functional programmer to enumerate to their imagination and beyond. The similarities between both object-oriented languages and advantages in scripting to software performance make it a hard duo to top.

Gradle introduces the efficiency that Docker allows for applications, even with additional build tools and runtimes, these two formats enable a forward-moving development cycle with simple configurations. Having studied both methodologies individually for their practice and tested paralleled solutions, the build performance of both is quite enjoyable.

Java unlocks the stability and flexibility where JavaScript meets the flow and approach in the browser. Gradle combines the elasticity of modern development with the ease of use for observable change. Docker bundles multiple languages into containerized elegance. GraphQL distributes efficient queries of data in a fluid methodology for users at multiple stages. Kubernetes steers the demand for accessibility and amplifies the reach of the application. Vitess empowers the data users submit and request into its realm of sharding and scalability in a familiar structure. All of this is made simple and approachable by time, time spent, on a small, silicon-based machine that creates an efficient and focused environment for learning and building.

Thanks for Reading, Keep Computing!

Looking for more Application Development advice? Follow along on Twitter, GitHub, and LinkedIn. Visit online for the latest updates, news, and information at heyitsjoealongi.com.

--

--