In the team that develops Allegro Recommendation Platform we weren’t happy with our integration tests. Long setup and assertions blocks resulted in a low signal—to—noise ratio and poor readability. These tests were also full of ad hoc variables like 1, ABC, OK or NOK, which caused that it was hard to find a connection between input and output data. Moreover, any change in an API caused changes in many tests.
Today, in an age of great popularity of test automation, we can observe a growing number of different frameworks which allow us to write scripts and programs that simulate human actions performed on the website under test. The most popular solution is Selenium WebDriver framework, which is compatible with the majority of common languages. Broad compatibility is a big advantage of Selenium especially in cases when we have to write some tests integrated to our application project, because no matter in which language we write the application, Selenium probably has an API for that language too. But there are also situations, like writing a standalone test suite, when we can decide which technology we’d most like to use. Taking into consideration this case, let me introduce to you a short comparison of three JDK-based technologies:
Today we would like to share our experience with Apache Spark, and how to deal with one of the most annoying aspects of the framework. This article assumes basic knowledge of Apache Spark. If you feel uncomfortable with the basics of Spark, we recommend you to participate in an excellent online course prepared by the creators of Spark.
