In this article, we want to share our journey of searching for optimizations in one of Allegro’s main microservices: opbox-web. You’ll read about the issues we had to deal with and how we managed to overcome them — together with a few surprises along the way and even one golden rule broken.
As part of Allegro Hacktoberfest celebrations, Andamio Task Force (the team responsible for Andamio, a set of common libraries used by most JVM projects at Allegro) posted the following message on our social platform…
In this article we’ll present methods for efficiently optimizing physical resources and fine-tuning the configuration of a Google Cloud Platform (GCP) Dataflow pipeline in order to achieve cost reductions. Optimization will be presented as a real-life scenario, which will be performed in stages.
In early 2024, I hit ten years at Allegro, which also happens to be how long I’ve been working with microservices. This timespan also roughly corresponds to how long the company as a whole has been using them, so I think it’s a good time to outline the story of project Rubicon: a very ambitious gamble which completely changed how we work and what our software is like. The idea probably seemed rather extreme at the time, yet I am certain that without this change, Allegro would not be where it is today, or perhaps would not be there at all.
At Allegro, we use Kafka as a backbone for asynchronous communication between microservices. With up to 300k messages published and 1M messages consumed every second, it is a key part of our infrastructure. A few months ago, in our main Kafka cluster, we noticed the following discrepancy: while median response times for produce requests were in single-digit milliseconds, the tail latency was much worse. Namely, the p99 latency was up to 1 second, and the p999 latency was up to 3 seconds. This was unacceptable for a new project that we were about to start, so we decided to look into this issue. In this blog post, we would like to describe our journey — how we used Kafka protocol sniffing and eBPF to identify and remove the performance bottleneck.
B-tree is a structure that helps to search through great amounts of data. It was invented over 40 years ago, yet it is still employed by the majority of modern databases. Although there are newer index structures, like LSM trees, B-tree is unbeaten when handling most of the database queries.
Software Architecture is an elusive thing which, if neglected, can lead to a hard-to-develop and maintain codebase, and in more drastic circumstances to the failure of a product. This article discusses one of the backend application architecture styles which proved to be successful in providing a good foundation for building and maintaining an application in the long run: Onion Architecture.
When you go through articles related to Hexagonal Architecture (HA) you usually search for practical examples. HA isn’t simple, that’s why most trivial examples make readers even more confused, though it is not as complex as many theoretical elucidations present it. In most posts you have to scroll through exact citations or rephrased definitions of concepts such as Ports and Adapters or their conceptual diagrams. They have already been well defined and described by popular authors i.e. Alistair Cockburn or Martin Fowler. I assume you already have a general understanding of Domain Driven Design and that you understand terms such as Ports and Adapters. I’m not a HA expert, yet I use it everyday and I find it useful. The only reason I write this post is to show you that Hexagonal Architecture makes sense, at least if your service is a little more than a JsonToDatabaseMapper.
Recently our team has been tasked to write a very fast cache service. The goal was pretty clear but possible to achieve in many ways. Finally we decided to try something new and implement the service in Go. We have described how we did it and what values come from that.
Microservices are now the mainstream approach for scalable systems architecture. There is little controversy when we are talking about designing backend services. Well-behaved backend microservice should cover one BoundedContext and communicate over the REST API. Things get complicated when we need to use microservices as building blocks for a frontend solution. How to build a consistent website or a mobile app using tens or sometimes hundreds of microservices?
