May 16 2019
Hermes 1.0 released
After 5 years of developing and maintaining Hermes, we are very excited to announce that version 1.0 has been released.
Apr 9 2019
Why Allegro Ads chose TypeScript
A modern look and fully mobile-friendly design — this is how we created the new version of ads.allegro.pl. Unfortunately, several hundred tests did not protect us from errors. The last straw was when a simple but very annoying problem appeared in production. We decided to start a revolution. We considered Flow, but ended up adopting TypeScript for Allegro Ads.
Apr 1 2019
[April 1st] From PHP to Java and back again: a retreat from microservices
This post was published on April 1st, 2019, and should not be taken too seriously.
Jan 28 2019
Making friends with legacy code
What kind of builders do you use in your tests? Do you use the old good builders where properties are set by a withProperty method? Or maybe, you use builders that make use of closures? As soon as I joined my team, I found out that there is another approach to test builders, and it embraces maps. It seemed innovative, but after a while, it became annoying legacy code. After one of the debugging sessions, I decided to get rid of it, but the monster fought me back.
Dec 18 2018
Java 11 and beyond at Allegro
Since the beginning of my career as a Java developer, keeping up to date with Java was fairly straightforward. Big releases came every few years, causing a bit of mayhem in tooling, IDEs and job interview questions. However with each release there was a lot of time to adjust and migrate to newer versions. With Java 9 things have changed. The so called new release cadence was announced by Oracle. Instead of releasing a new major version each few years, we will get one every half year. This poses a lot of challenges to companies using Java in production, developers and the community as a whole.
Nov 19 2018
It’s all about naming
We all know that “there are only two hard things in Computer Science: cache invalidation and naming things”. However, we know that writing good tests is hard, too. Does it mean that the author of the famous quote is wrong? Not necessarily! In fact, what are tests? These are descriptions of some concepts. They answer the questions: “what?”, “when?” and “how?”. When writing tests, you are answering these questions by giving names to objects, reasons, and processes. I think writing tests is hard because it is all about naming. I believe you can significantly improve your tests just by focusing on giving them correct names. Let me tell you a story of a pull request I was reviewing some time ago.
Oct 22 2018
Turnilo — let’s change the way people explore Big Data
Two years ago at Allegro we used to have a very typical Big Data technology stack. The architecture was based on a Hadoop cluster and we would query it with plain Hive queries, Spark jobs and Jupyter notebooks. Over those last two years we have transformed it into a more efficient and easy to use OLAP platform.
Oct 1 2018
From Simple to Fast - Custom Solr Document Collapsing
At Allegro we use Solr as our main search engine. Due to the traffic to our search engine being thousands of requests per second and index size on the order of a hundred million documents we need to develop custom optimizations. In this post I will describe the story of our Solr plugin development.
Aug 31 2018
Postmortem — why Allegro went down
We messed up. On July 18th, 2018, at noon, Allegro went down and was unavailable for twenty minutes. The direct cause was a special offer in which one hundred Honor 7C phones whose regular price is around PLN 850 (about € 200), were offered at a price of PLN 1 (less than € 1). This attracted more traffic than we anticipated and at the same time triggered a configuration error in the way services are scaled out. This caused the site to go down despite there being plenty of CPUs, RAM, and network capacity available in our data centers.
Jul 24 2018
Squeezing performance out of Elasticsearch
Lately my colleague Michał described how he tracked a Java memory leak. Although that problem was completely solved, new complications suddenly appeared on the horizon. As it usually happens, everything started with an alert from our monitoring system which woke me up in the middle of the night.
Jun 26 2018
From Java to C# and back again
Throughout my studies at university and work in the industry I switched my primary programming language from Java to C# and back again to Java. This article gathers some of my thoughts on using both languages. It’s not intended to be a comprehensive comparison of Java and C#. There are a lot of other resources on the Internet that cover this topic. Instead, I want to focus on what I personally liked about both languages and how it felt to transition between them.
May 28 2018
Static linking vs dyld3
The following article has two parts. The first part describes improving Allegro iOS app launch time by adopting static linking and sums it up with a speedup analysis. The second part describes how I managed to launch a custom macOS app using not-yet-fully-released dyld3 dynamic linker and also completes with an app launch speedup analysis.
May 23 2018
From Java to Kotlin and Back Again
Due to the high interest and controversy concerning this blog post, we believe that it is worth adding some context on how we work and make decisions at Allegro. Each of more than 50 development teams at Allegro has the freedom to choose technologies from those supported by our PaaS. We mainly code in Java, Kotlin, Python and Golang. The point of view presented in the article results from the author’s experience.
May 7 2018
A comedy of errors. Debugging Java memory leaks.
We all make errors, but some errors seem so ridiculous we wonder how anyone, let alone we ourselves, could have done such a thing. This is, of course, easy to notice only after the fact. Below, I describe a series of such errors which we recently made in one of our applications. What makes it interesting is that initial symptoms indicated a completely different kind of problem than the one actually present.
Apr 24 2018
Spring @WebMvcTest with Spock Framework
Spring is one of the most popular JVM-targeted frameworks. One of the reasons why it has become so popular is writing tests. Even before Spring Boot era, it was easy to run an embedded Spring application in tests. With Spring Boot, it became trivial. JUnit and Spock are two most popular frameworks for writing tests. They both provide great support and integration with Spring, but until recently it was not possible to leverage Spring’s @WebMvcTest in Spock. Why does it matter? @WebMvcTest is a type of an integration test that only starts a specified slice of Spring Application and thus its execution time is significantly lower compared to full end-to-end tests. Things have changed with Spock 1.2. Let me show you, how to leverage this new feature.
Apr 4 2018
Intuition Engineering at Allegro with Phobos
At Allegro, feature velocity is a top priority. We believe that one of our critical competitive advantages is the rate at which we introduce new features. In order to achieve a high feature velocity, one of the architectural choices that Allegro made a while back was to move to microservice architecture. So when somebody uses Allegro, a request comes in (and we just have a hypothetical example here) to service D — and we can imagine a service D being a proxy or an API layer — and whatever that service is, it is not going to have all the information it needs to serve a response. So the service D is going to reach out to service C and service F, and F in turn will reach out to A, and it will in turn reach out to B and E, and you see that this very quickly gets complicated. Allegro has somewhere around 500 microservices, so you can imagine how complicated the communication map looks.
Mar 12 2018
Psychological needs at work
This is a post for those seeking to accomplish business goals and ensure stability of the solutions developed while maintaining focus on people. The model of three basic psychological needs that I’m presenting here may be useful for leaders, agile coaches, and scrum masters. I also encourage developers to do some self-reflection. This is the knowledge I’ve gained at the World Conference of Transactional Analysis in Berlin. Transactional Analysis (TA) is a theory of interpersonal relationships developed by Eric Berne which has a practical application in various fields, including organizations.
Feb 15 2018
How to approach testing in development process?
Application release process, or in fact software development process, as a release is the final stage of application development, is not an easy thing. Books and IT websites discuss many approaches and each has its supporters and opponents. On the one hand, you have product owners, project managers and customers who want a ready-to-use application as soon as possible. On the other hand, we developers and testers, would like to release an application of the highest quality, which may affect the delivery time. Balancing these needs is a hard nut to crack. Usually, both sides need to make some compromises to establish a common way of working. For developers and testers, it involves answering several questions concerning software development methods, skills, use of manual or automated testing, and storage of test cases and test logs. In this article I describe best practices and tips for starting a new project. I think that by following them, you will make the software development process as effective as possible and adjusted to conditions of your project.
Jan 11 2018
Mesos Executor
Apache Mesos is an open-source project to manage computer clusters. In this article we present one of its components called Executor and more specifically the Custom Executor. We also tell you why you should consider writing your own executor by giving examples of features that you can benefit from by taking more control over how tasks are executed. Our executor implementation is available at github.com/allegro/mesos-executor
Nov 24 2017
This Fall digest
Some of our engineers run their own tech blogs. We encourage them to move here, but for various reasons, they prefer to publish on private blogs. We respect their decisions. What we can do is to gather all the blog posts published by allegro.tech engineers around the web in the one place.
Jul 24 2017
Golang slices gotcha
In this post I present a story of a bug that hit us recently. Everything was caused by unexpected (although documented) behavior of Go built-in function append. This bug has lived silently for nearly a year in allegro/marathon-consul. Ensure you run the latest version.
Jun 29 2017
Presto - a small step for DevOps engineer but a big step for BigData analyst
I bet you have found this article after googling some of the issues you encounter when working with a Hadoop cluster. You probably deal with Hive queries used for exploratory data analysis that are processed way too long. Moreover, you cannot adapt Spark in your organization for every use case because of the fact that writing jobs requires quite strong programming skills. Clogged Yarn queues might be your nightmare and waiting for the launch of the container when you run even a small query drives you mad. Before we deployed Presto — a Fast SQL engine provided by Facebook — our analysts struggled with these problems on a regular basis.
Jun 12 2017
The Agile Testing Days Conference - Potsdam 2016
Last year Agile Testing Days Conference was held between 6th and 8th December in Potsdam, Germany. According to statistics provided by organizers there were around 600 attendants and about 30 speakers in keynotes / workshops. During the whole event around 22 different workshops took place, 9 keynotes and around 30 other speeches / presentations divided into 7 parallel tracks. I had the opportunity to attend the first two days of the conference and in this article I’d like to share with you a short review of the most interesting sessions and my impressions about the event.
Apr 21 2017
Distributed rate limiting of delivery attempts
In our services ecosystem it’s usually the case that services can handle a limited amount of requests per second. We show how we introduced a new algorithm for our publish-subscribe queue system. The road to production deployment highlights some key distributed systems’ takeaways we’d like to discuss.
Mar 20 2017
Hitting the Wall
Running Mesosphere Marathon is like running… a marathon. When you are preparing for a long distance run, you’ll often hear about Hitting the wall. This effect is described mostly in running and cycling but affects all endurance sports. It happens when your body does not have enough glycogen to produce power and this results in a sudden “power loss” so you can’t run anymore. At Allegro we have experienced a similar thing with Mesosphere Marathon. This is our story on using Marathon in a growing microservice ecosystem, from tens of tasks and a couple applications, to thousands of tasks and over a hundred applications. If there is no mention of Marathon version, it is 1.3.10 and below; we need some time to test and deploy the latest 1.4 release. If you are interested in how our ecosystem is built, take a look at below MesosCon presentation.