The following article is an excerpt from Software Mistakes and Trade-offs book. In real-world big data applications, the amount of data that we need to store and process can be often counted in the hundreds of terabytes or petabytes. It is not feasible to store such an amount of data on one physical node. We need a way to split that data into N data nodes.
It’s been over 5 years since the introduction of the article describing the ongoing transformation of Allegro’s frontend architecture — an approach that was later formalized by the industry under the name of Micro Frontends. I think that after all this time we can safely say that this direction was correct and remained almost entirely unchanged in relation to the original idea. Still, some of the challenges foreseen in the publication soon became the reality. In this article I would like to focus on the CSS part of the whole adventure to tell you about how we manage consistency and frontend performance across over half a thousand components, and what it took us to get to where we stand today.
Almost every modern web application somehow interacts with a backend - be it loading data, doing background sync, submitting a form, or publishing the metrics. Making API requests is not an easy task - we have to consider multiple outcomes and handle them properly. Otherwise, we might end up with confused users and decreased conversion. Although the stakes are high, it is still very likely to encounter an application designed with only a happy path scenario in mind. The question is - how can we improve it?
Some time ago, our team faced the issue of moving an existing Apache Spark job from an on-premise Hadoop cluster to public cloud. While working on the transition we came across another way to process data that is Apache Beam. We were curious whether this tool had more advantages in comparison to traditional Apache Spark. We wanted to find the answer relatively quickly with minimal effort. Hence, we built two projects to process the same data using these technologies. Below you can get to know the architecture of the jobs written in Apache Spark and Apache Beam.
I’m a big fan of logging as much extra data as possible. I’m also a DRY approach believer. I feel strong anxiety when I see repetition in code. Combining all these “passions” is not always easy. It’s hard to log everything without repeating things. Even if it’s possible, it usually leads to inelegant code.