Articles tagged with
tech

16 Jul 2024

BPMN: The Key to Understanding Business Processes

If you have experience with Event Storming and have ever found yourself wishing there was a way to document the insights gathered during a session, or wanting to communicate the process to other team members, then I have a solution for you. This idea can be expressed in a famous saying: One picture is worth more than a thousand words.


01 Jul 2024

INP — what is the new Core Web Vitals metric and how do we work with it at Allegro.

Site performance is very important, first of all, from the perspective of users, who expect a good experience when visiting the site. The user should not wait too long for the page to load. We all know how annoying it can be when we want to press an element and it jumps to another place on the page or when we click on a button and then nothing happens for a very long time. The state of a site’s performance in these aspects is measured by Web Vitals performance metrics and most importantly by a set of three major Core Web Vitals metrics (LCP — Largest Contentful Paint, CLS — Cumulative Layout Shift, INP — Interaction to Next Paint). They are responsible for measuring the 3 things: loading time, visual stability and interactivity. These metrics are also important for the websites themselves because, in addition to the user experience, they are also taken into account in terms of the website’s positioning in search engines (SEO), which is crucial for most websites on the Internet, Allegro included.




16 May 2024

Unveiling bottlenecks of couchbase sub-documents operations

This story shows our journey in addressing a platform stability issue related to autoscaling, which, paradoxically, added some additional overhead instead of reducing the load. A pivotal part of this narrative is how we used Couchbase — a distributed NoSQL database. If you find yourself intrigued by another enigmatic story involving Couchbase, don’t miss my blog post on tuning expired doc settings.


12 Apr 2024

Ten Years and Counting: My Affair with Microservices

In early 2024, I hit ten years at Allegro, which also happens to be how long I’ve been working with microservices. This timespan also roughly corresponds to how long the company as a whole has been using them, so I think it’s a good time to outline the story of project Rubicon: a very ambitious gamble which completely changed how we work and what our software is like. The idea probably seemed rather extreme at the time, yet I am certain that without this change, Allegro would not be where it is today, or perhaps would not be there at all.


06 Mar 2024

Unlocking Kafka's Potential: Tackling Tail Latency with eBPF

At Allegro, we use Kafka as a backbone for asynchronous communication between microservices. With up to 300k messages published and 1M messages consumed every second, it is a key part of our infrastructure. A few months ago, in our main Kafka cluster, we noticed the following discrepancy: while median response times for produce requests were in single-digit milliseconds, the tail latency was much worse. Namely, the p99 latency was up to 1 second, and the p999 latency was up to 3 seconds. This was unacceptable for a new project that we were about to start, so we decided to look into this issue. In this blog post, we would like to describe our journey — how we used Kafka protocol sniffing and eBPF to identify and remove the performance bottleneck.


20 Feb 2024

Tired of repetitive tasks?! Go for RPA!

Have you ever thought about ways of reducing repetitive, monotonous tasks? Maybe you would like to try to automate your own tasks? I will show you what technology we use at Allegro, what processes we have automated, and how to do it on your own.


12 Feb 2024

Don’t bother: it is only a little expired

This story shows how we strive to fix issues reported by our customers regarding inconsistent listing views on our e-commerce platform. We will use a top-down manner to guide you through our story. At the beginning, we highlight the challenges faced by our customers, followed by presenting basic information on how views are personalized on our web application. We then delve deeper into our internal architecture, aiming to clarify how it supports High Availability (HA) by using two data centers. Finally, we advertise a little Couchbase, distributed NoSQL database, and explain why it is an excellent storage solution for such an architecture.


24 Jan 2024

WCAG 2.2 is here! And what about 3.0?

Ready to turn web accessibility from a headache into a breeze? Join us as we demystify WCAG, explore its latest 2.2 version, and gaze into the future of digital inclusivity. Get ready for a journey that’s as enlightening as it is entertaining!



14 Dec 2023

Clever, surprised and gray-haired

This article is a form of a public postmortem in which we would like to share our bumpy way of revealing the cause of a mysterious performance problem. Besides unveiling part of our technical stack based on open-source solutions, we also show how some false assumptions made such a bug triage process much harder. Besides all NOT TO DOs, you can find some exciting information about performance hunting and reproducing performance issues on a small scale. As a perk, we prepared a repository where you can reproduce the problem and make yourself familiar with tools that allowed us to confirm the cause. The last part (lessons learned) is the most valuable if you prefer to learn from the mistakes of others.


27 Nov 2023

How does B-tree make your queries fast?

B-tree is a structure that helps to search through great amounts of data. It was invented over 40 years ago, yet it is still employed by the majority of modern databases. Although there are newer index structures, like LSM trees, B-tree is unbeaten when handling most of the database queries.


30 Oct 2023

Beyond the Code - An Engineer’s Battle Against Knowledge Loss

The idea for this article arose during a meeting where we learned that our supervisor would be leaving the company to pursue new opportunities. In response, a colleague lamented that what we would miss most is the knowledge departing with the leader. Unfortunately, that’s how it goes. Not only do we lose a colleague, but we also lose valuable knowledge and experience. However, this isn’t a story about my supervisor; it’s a story about all those individuals who are experts in their fields, who understand the paths to success and paths that lead to catastrophic failures. When they leave, they take with them knowledge that you won’t find in any book, note, or Jira ticket. And this leads to a fundamental question: What can be done to avoid this “black hole” of knowledge? How can we ensure it doesn’t vanish along with them? That’s what this article is all about.


14 Sep 2023

Online MongoDB migration

MongoDB is the most popular database used at Allegro. We have hundreds of MongoDB databases running on our on—premise servers. In 2022 we decided that we need to migrate all our MongoDB databases from existing shared clusters to new MongoDB clusters hosted on Kubernetes pods with separated resources. To perform the migration of all databases we needed a tool for transfering all the data and keeping consistency between old and new databases. That’s how mongo-migration-stream project was born.



10 Jul 2023

From 3TB to 100GB: A Cost-Saving Journey in Database Maintenance

In the era of ubiquitous cloud services and an increasingly growing PaaS and serverless-oriented approach, performance and resources seem to be becoming less and less important. After all, we can scale horizontally and vertically at any time, without worrying about potential performance challenges that the business may introduce.


31 May 2023

Debugging hangs - piecing together why nothing happens

As a part of a broader initiative of refreshing Allegro platform, we are upgrading our internal libraries to Spring Boot 3.0 and Java 17. The task is daunting and filled with challenges, however overall progress is steady and thanks to the modular nature of our code it should end in finite time. Everyone who has performed such an upgrade knows that you need to expect the unexpected and at the end of the day prepare for lots of debugging. No amount of migration guide would prepare you for what’s coming in the field. In the words of Donald Rumsfeld there are unknown unknowns and we need to be equipped with the tools to uncover these unknowns and patch them up. In this blog post I’d like to walk you through a process that should show where the application hangs, although there seems to be nothing wrong with it. I will also show that you don’t always know what code you have – problem known as dependecy hell, place we got quite cosy in during this upgrade.


18 Apr 2023

Trust no one, not even your training data! Machine learning from noisy data

Label noise is ever-present in machine learning practice. Allegro datasets are no exception. We compared 7 methods for training classifiers robust to label noise. All of them improved the model’s performance on noisy datasets. Some of the methods decreased the model’s performance in the absence of label noise.


05 Apr 2023

Dynamic Workload Balancing in Hermes

Hermes is a distributed publish-subscribe message broker that we use at Allegro to facilitate asynchronous communication between our microservices. As our usage of Hermes has grown over time, we faced a challenge in effectively distributing the load it handles to optimize resource utilization. In this blog post, we will present the implementation of a dynamic workload balancing algorithm that we developed to address this challenge. We will describe the approach we took, the lessons we learned along the way, and the results we achieved.


21 Mar 2023

How neuroscience can help you as a software engineer - motivation

Many of us, software engineers, have experienced those days when nothing really sparks joy in coding, debugging, preparing spikes or refining tasks for the next sprints. Obviously, we would like to have as few of such days as possible and go on with our work effectively. A solution to this definitely is not tormenting our brains with guilt and forced labour. There are other ways, and I would like to invite you to explore them with me and learn a little about our nervous systems in the process. We’ll find out where the motivation comes from on a biological and psychological level. We’ll also take a look at the changes you can introduce into your day to take advantage of certain mechanisms working on a neural level and boost your motivation and productivity.


13 Feb 2023

Onion Architecture

Software Architecture is an elusive thing which, if neglected, can lead to a hard-to-develop and maintain codebase, and in more drastic circumstances to the failure of a product. This article discusses one of the backend application architecture styles which proved to be successful in providing a good foundation for building and maintaining an application in the long run: Onion Architecture.




10 Nov 2022

Vanilla JS is not dead! Microfrontends without web performance issues.

Building a complex web platform can be a real challenge, especially when parts of it are delivered by independent teams. Picking out the correct architecture is crucial, but maintaining it can be even more challenging. Frontend microservices, aka microfrontends, is an architecture that gives a lot of flexibility, but can cause performance issues in the future, if not managed well. This article presents an approach to the microfrontends architecture to keep the frontend technology stack efficient based on the complexity of user interface.


04 Oct 2022

Probabilistic Data Structures and Algorithms in NoSQL databases

What would you say if we stored 1 000 records in a database, and the database claimed that there were only 998 of them? Or, if we created a database storing sets of values and in some cases the database would claim that some element was in that set, while in fact it was not? It definitely must be a bug, right? It turns out such behavior is not necessarily an error, as long as we use a database that implements probabilistic algorithms and data structures. In this post we will learn about two probability-based techniques, perform some experiments and consider when it is worth using a database that lies to us a bit.


26 Sep 2022

Example of modularization in Allegro Pay Android application

Currently, in the Android world, the topic of modularization is very popular. Many bloggers describe their experiences with it and analyze what Google recommends. Our team started the modularization process before it was hot. I will describe our reasons, decisions, problems and give you some advice. We will see if modularization makes sense and what it brings to the table. I will also post some statistics showing what it looked like before and after the modularization process.


13 Sep 2022

How to efficiently write millions of records in the cloud and not go bankrupt — an Azure CosmosDB case study

Cloud providers like to brag about high availability and unlimited scaling of their services – and they are correct, as these features are indeed significant advantages of cloud solutions. Their computational power is so high that for most use cases, it’s almost unlimited. In this blog post, I would like to tell you about our experiences with Azure Cosmos DB and batch processing.


03 Aug 2022

MBox: server-driven UI for mobile apps

In this article, we want to share our approach to using server-driven UI in native mobile apps. In 2019 we created the first version of the in-house server-driven rendering tool called MBox and used it to render the homepage in the Allegro app on Android and iOS. We have come a long way since then, and now we use this tool to render more and more screens in the Allegro apps. After almost three years of working on MBox, we want to share how it works and the key advantages and challenges of using this approach.


19 Jul 2022

How to facilitate EventStorming workshops

With this article, I would like to introduce you to EventStorming and explain to you how to get started. I am not discovering anything new, just gathering available knowledge in one place. What I will show you is a few tips on how to conduct and facilitate EventStorming workshops.


30 Jun 2022

GC, hands off my data!

Each of us has probably experienced a time in our career when we wanted to get rid of the Garbage Collector from our application because it was running too long, too often, and perhaps even led to temporary system freezes. What if we could still benefit from the GC, but in special cases, also be able to store data beyond its control? We could still take advantage of its convenience and, at the same time, be able to easily get rid of long GC pauses. It turns out that it is possible. In this article, we will look at whether and when it is worth storing data beyond the reach of the Garbage Collector’s greedy hands.



07 Jun 2022

How to include refactoring into product development roadmap? Reducing technical debt inspired by real options identification

Nowadays, technical debt can be considered the bread and butter of most IT-powered enterprises around the world. Almost every company that survived the startup phase and managed to deliver its first products to customers will face at some point technical challenges related to past architectural decisions. Although code engineering gets better every year, we cannot argue with the obvious fact of life: the market will always force many of us to deliver tech products faster than we wish. Time To Market has always been a key success factor for many product companies and it puts a lot of pressure on Engineering to keep up with challenging deadlines.




30 Mar 2022

Visual thinking

We use written (source code) language to express our intentions in a machine-readable form. We use spoken language to communicate with other people. We pride ourselves as ones choosing a programming language optimized to the task at hand. Do we use the optimal way to express our ideas?



01 Mar 2022

The War in Ukraine

The attack on Ukraine has shocked the entire world, but in Poland it is making especially big waves. Not just due to geographical proximity - Ukrainians are one of the largest minorities in Poland and simply put it is affecting people we know personally.


22 Feb 2022

Make your tests readable by example

Have you ever worked on a project where after downloading the code from the repository you start to wonder what business requirements are hidden under the layer of unreadable tests? Or maybe you are currently wondering how to test a new feature that you have been entrusted to implement?


10 Feb 2022

Why should we rewrite enzyme.js to react-testing-library and how to do that?

Everyone repeats like a mantra that tests are an indispensable element of development work. Is there anything to it? Well, I need to admit that as a developer, I rather often want to skip the test writing stage. I assume I’m not the only one. I’m aware that it’s a mistake, even with testers on board. Effective and efficient testing of your own code can help with catching bugs in new functionalities, as well as in changes to already existing ones. It cannot be questioned. Sometimes tests also help to understand how some long-unused functionality or component works. And that’s a small bonus too. Can a large project cope without testing? Probably so. But the number of errors and their severity will probably be much higher. That’s why in Allegro Ads we pay attention to writing tests.


24 Jan 2022

How do coroutines work internally in Python?

Most of applications we create are basically loops. An average program waits for an event, then processes it following some business logic. Afterwards it begins waiting for another event to arrive. Java Servlets work this way too. Popular frameworks such as Spring allow us to only care about the business logic, while the framework takes care of the application main loop.


12 Jan 2022

Shrinking the size of a monorepo

The source code of Allegro iOS app for buyers used to be divided into separate modules hosted in multiple repositories (polyrepo). The source code was migrated to a monorepo a few years back along with the history of all repos that constituted the app. Updating source code of a module on one repository could affect another module hosted on a separate repository. Versioning modules and propagation of dependency update led to long release process of the entire application. Our main repository for the iOS application thus became our monorepo. After 9 years of development of the app the repo size has grown enormously and the git clone command became a nightmare taking too much time. We had a possibility to shrink the project size during the migration from an on-premise to an external git repo hosting provider.


20 Dec 2021

Evaluating performance of time series collections

A new version of MongoDB, 5.0, has been recently launched. The list of changes included one that I found particularly interesting: the time series collections. It is a method of effective storing and processing of time-ordered value series. In this article we will verify whether the processing of time series is really as fast as promised by the authors.


13 Dec 2021

Clean Architecture Story

The Clean Architecture concept has been around for some time and keeps surfacing in one place or another, yet it is not widely adopted. In this post I would like to introduce this topic in a less conventional way: starting with customer’s needs and going through various stages to present a solution that is clean enough to satisfy concepts from the aforementioned blog (or the book with the same name).


09 Dec 2021

Which skills to choose in order to be more valuable

At some point in your career, you realize that it’s time to try to advance through the hierarchy. You think you are doing a good job. You are constantly developing and learning something new. But at the same time, someone you know, with much less experience and knowledge than you, has long been higher up the hierarchy than you. Then you ask yourself: what is wrong with me? In my case, the answer turned out to be properly gathering the expectations concerning my skills and work.


22 Nov 2021

Moving towards Micronaut

Micronaut is one of the new application frameworks that have recently sprung up. It promises low memory usage and faster application startup. At Allegro we decided to give it a try. In this article we’ll learn what came out of it and if it’s worth considering when creating microservices-based systems.


09 Nov 2021

OAuth rate-limiting

Every e-commerce platform needs some kind of central authorization system. At Allegro we use OAuth and have our own implementation that stays true to the RFC. Allegro has millions of users. There are also a lot of requests that go through OAuth services. At some point there comes a need to have better control over how much traffic we want to allow in a certain time window, while maintaining full performance of the platform. Here is where the idea of rate-limiting comes in handy.


26 Oct 2021

How we refactored the search form UI component

This article describes a classic case of refactoring a search form UI component, a critical part of every e-commerce platform. In it I’ll explain the precursor of change, analysis process, as well as aspects to pay attention to and principles to apply while designing a new solution. If you are planning to conduct refactoring of a codebase or just curious to learn more about frontend internals at Allegro, you might learn a thing or two from this article. Sounds interesting? Hop on!


18 Oct 2021

Comparing MongoDB composite indexes

One of the key elements ensuring efficient operation of the services we work on every day at Allegro is fast responses from the database. We spend a lot of time to properly model the data so that storing and querying take as little time as possible. You can read more about why good schema design is important in one of my earlier posts. It’s also equally important to make sure that all queries are covered with indexes of the correct type whenever possible. Indexes are used to quickly search the database and under certain conditions even allow results to be returned directly from the index, without the need to access the data itself. However, indexes are not all the same and it’s important to learn more about their different types in order to make the right choices later on.


07 Oct 2021

How to ruin your Elasticsearch performance — Part II: Breaking things, one at a time

It’s easy to find resources about improving Elasticsearch performance, but what if you wanted to reduce it? In Part I of this two-part series we looked under the hood in order to learn how ES works internally. Now, in Part II, is the time to apply this knowledge in practice and ruin our ES performance. Most tips should also be applicable to Solr, raw Lucene, or, for that matter, to any other full-text search engine as well.


30 Sep 2021

How to ruin your Elasticsearch performance — Part I: Know your enemy

It’s easy to find resources about improving Elasticsearch performance, but what if you wanted to reduce it? This is Part I of a two-post series, and will present some ES internals. In Part II we’ll deduce from them a collection of select tips which can help you ruin your ES performance in no time. Most should also be applicable to Solr, raw Lucene, or, for that matter, to any other full-text search engine as well.


23 Sep 2021

Evolution of web performance culture

The main goal of boosting website performance is to improve the user experience. In theory, a satisfied customer is more likely to use a particular company’s services, which is then reflected in business results. However, from my own experience I can say that not every change can be easily converted into money. I would like to tell you how to reconcile these two worlds, how to convince the business that the benefits of better performance are a long-term investment, and how to streamline the development process during the design or code writing process.



29 Jul 2021

CSS Architecture and Performance in Micro Frontends

It’s been over 5 years since the introduction of the article describing the ongoing transformation of Allegro’s frontend architecture — an approach that was later formalized by the industry under the name of Micro Frontends. I think that after all this time we can safely say that this direction was correct and remained almost entirely unchanged in relation to the original idea. Still, some of the challenges foreseen in the publication soon became the reality. In this article I would like to focus on the CSS part of the whole adventure to tell you about how we manage consistency and frontend performance across over half a thousand components, and what it took us to get to where we stand today.


28 Jun 2021

One task — two solutions: Apache Spark or Apache Beam?

Some time ago, our team faced the issue of moving an existing Apache Spark job from an on-premise Hadoop cluster to public cloud. While working on the transition we came across another way to process data that is Apache Beam. We were curious whether this tool had more advantages in comparison to traditional Apache Spark. We wanted to find the answer relatively quickly with minimal effort. Hence, we built two projects to process the same data using these technologies. Below you can get to know the architecture of the jobs written in Apache Spark and Apache Beam.


17 Jun 2021

How to make context logging in Python less cumbersome

I’m a big fan of logging as much extra data as possible. I’m also a DRY approach believer. I feel strong anxiety when I see repetition in code. Combining all these “passions” is not always easy. It’s hard to log everything without repeating things. Even if it’s possible, it usually leads to inelegant code.


08 Jun 2021

Measuring Web Performance

Some time ago we announced that Allegro passes Core Web Vitals assessment and thanks to that we were awarded in “Core Web Vitals Hall of Fame”. It means that Allegro is in the group of the 27% fastest websites in Polish Internet.



11 May 2021

My first days at Allegro

The beginnings in a new job can be really tough, especially in such uncertain times as the pandemic. Remote onboarding, Zoom meetings, inability to talk face-to-face — it’s a big test, especially if you’re switching industries. My story is quite similar: in July 2020 I started a three-month internship at Allegro to train for product management. When the internship ended, I was offered to stay permanently as Junior Product Manager. I’d like to describe my beginnings at Allegro. But first — a few words about me.


13 Apr 2021

Kotlin - a language for everyone and for everything. Even scripts.

According to Wikipedia there are approximately 700 computer languages available. Seven hundred. This is an unbelievable number and it’s the reason why programmers face the problem of choosing a programming language to work with, which frameworks to use and which tech stack to learn. All of them have pros and cons, but when looking for an all-purpose language you should take Kotlin into consideration and ask yourself the question: “Do I really need another programming language?”.


23 Mar 2021

Digging into Allegro, or how I started my testing career

Maybe it’s another “How I’ve changed my worklife” story, but I hope it is an interesting one. It seems quite important to tell: before the beginning of my Allegro adventure, I was working as… an archaeologist. It was both physical work on excavations and research work at my PhD studies. And the only thing that connects my past work and the current one is: attention to detail.


05 Mar 2021

Finite-state machines made easy

Coordinating complex processes, both business and technical, can be a challenging issue in a distributed system. Especially when the complications associated with them, such as concurrency, idempotency, scalability and hindered testability, come into play — possibly all at once. This is definitely something that can keep many programmers awake at night.


15 Feb 2021

My first days at Allegro

So you’re new to Allegro, have just finished your tech onboarding and are stunned with information overflow? Or perhaps you are planning to join Allegro and don’t know what it looks like in here? I am about to try and describe how I felt just a few months ago and what startled or dismayed me. I hope this short article will answer all your concerns.


01 Feb 2021

Implement stateless authentication like a pro using OAuth: A 100% correct approach

Many of us spend most of their software development careers improving and extending applications protected by pre-existing security mechanisms. That’s why we rarely address problems related directly to authentication and authorization unless we build apps from scratch. Regardless of your experience I still hope you will find this article interesting. It’s not meant to be a tutorial. I would like to focus on clarifying basic concepts and highlighting common misconceptions.







01 Aug 2019

A/B testing good practice – calculating the required sample size

Developing our own A/B testing platform from scratch came with many lessons during the process. Our platform was built for people who differ vastly in terms of statistical knowledge, also for those who are not familiar with statistics at all. So the main challenge wasn’t technical implementation. The hardest thing was (and still is) developing good practices and spreading them among our users. We want to share some of our current knowledge that we think is crucial for A/B testing at scale. From this article, you will get to know about calculating sample size, what it means and what benefits it provides for you and your organization.


15 Feb 2018

How to approach testing in development process?

Application release process, or in fact software development process, as a release is the final stage of application development, is not an easy thing. Books and IT websites discuss many approaches and each has its supporters and opponents. On the one hand, you have product owners, project managers and customers who want a ready-to-use application as soon as possible. On the other hand, we developers and testers, would like to release an application of the highest quality, which may affect the delivery time. Balancing these needs is a hard nut to crack. Usually, both sides need to make some compromises to establish a common way of working. For developers and testers, it involves answering several questions concerning software development methods, skills, use of manual or automated testing, and storage of test cases and test logs. In this article I describe best practices and tips for starting a new project. I think that by following them, you will make the software development process as effective as possible and adjusted to conditions of your project.



08 Feb 2016

The great adventure of migrating Allegro infrastructure to new Data Center - DC5

Moving Allegro’s services and infrastructure from its main Data Center (DC) was considered a huge endeavour fraught with high risk from the very start. This particular Data Center, known as DC2, was situated in Poznań, Poland. It was our first genuine Data Center and back in 2007, when we moved in there, it was the state—of—the—art DC in Poland. For a couple of years our infrastructure was growing along with the company and the number of devices grew up to 2,000 filling 90 rack cabinets in 2014. DC was used by Allegro core sites such as allegro.pl, aukro.ua and aukro.cz as well as other Naspers-owned Polish operations like: payu.pl, olx.pl, ceneo.pl, otomoto.pl, otodom.pl. To make the story simpler, Allegro synonym is used to name them all.


08 Jan 2016

Microservices and macro mistakes

Microservices architecture has been given a lot of attention in recent years. You can find many articles defining what it is and explaining the difference between microservices and monolithic applications. Unfortunately, the dark side of the solution is rarely mentioned — such as its level of complexity and how difficult it is to design it correctly. Nevertheless, I would like to talk about my experience with microservices architecture. I hope that this article will help you avoid the mistakes I made and save time.