![\"Paging](\"/static/paging_demo-a325402259bf9864dced311c9eafa7c8.gif\")
Let's talk about harmony! If you have a look at an English dictionary, you'll see that even though almost everyone has some knowledge of the meaning of the word harmony, it's not so simple to properly define it. There are many contexts that it may apply, such as people relationships, business, law, and so on.
\nBut, at the end of the day, this concept imported from the music means that something happens in a way or frequency that makes it pleasant and less error-prone. So I want you to focus on the idea that the proper frequency is a way to achieve harmony.
\nThere are some cases where you would like to limit the frequency that something happens to it work properly. Imagine that you have a coffee machine, it's obvious that you would like it to dispense coffee at a frequency that you can replace the cups and be able to serve all your guests.
\nIt happens everywhere, even on software. At least while we have limited resources, we have to orchestrate them to work in harmony. One strategy is to use Rate Limiters, a maneuver to control the frequency in which some action is handled.
\nA common pattern to achieve the Rate Limiter strategy is to use the Token Bucket algorithm. In a nutshell, the Token Bucket mimics a real bucket where Theme Park tickets are stored. To enter to play around at the Theme Park, you need to take a ticket. But if the bucket is empty, you have to wait until some new tickets are put in there. Doing like that, you become able to control the flow of people to the Theme Park by controlling the number of tickets that are put into the bucket and the time interval in which the refill occurs.
\nIf we put 10 tickets per minute on that bucket, it'll assure that at most 10 people will enter the Theme Park in each 1-minute interval.
\nLibraries like Google Guava, Resilience4j, and Bucket4j are examples of how we can achieve rate-limiting in our Java software.
\nDespite being a beta version by the time of this writing, using the Google Guava RateLimiter library is pretty straightforward. You create a RateLimiter object defining the number of permits per second and, for each task, you specify that it should \"acquire\" the token that allows it to proceed.
\nfinal RateLimiter rateLimiter = RateLimiter.create(2.0); // rate is \"2 permits per second\"\n void submitTasks(List<Runnable> tasks, Executor executor) {\n for (Runnable task : tasks) {\n rateLimiter.acquire(); // may wait\n executor.execute(task);\n }\n}For Resilience4J you need to create some configuration before and decorate your task:
\n // Create a custom RateLimiter configuration\nRateLimiterConfig config = RateLimiterConfig.custom()\n .timeoutDuration(Duration.ofMillis(100))\n .limitRefreshPeriod(Duration.ofSeconds(1))\n .limitForPeriod(2)\n .build();\n// Create a RateLimiter\nRateLimiter rateLimiter = RateLimiter.of(**\"backendName\"**, config);\n\n// Decorate your call to BackendService.doSomething()\nSupplier<String> restrictedSupplier = RateLimiter\n .decorateSupplier(rateLimiter, backendService::doSomething);\n\n// First call is successful\nTry<String> firstTry = Try.ofSupplier(restrictedSupplier);\nassertThat(firstTry.isSuccess()).isTrue();In the same path, now using Bucket4j, you need to specify the refill interval and the limit:
\nRefill refill = Refill.intervally(2, Duration.ofSeconds(1));\nBandwidth limit = Bandwidth.classic(2, refill);\nBucket bucket = Bucket4j.builder()\n .addLimit(limit)\n .build();\n\nif (bucket.tryConsume(1)) {\n executor.execute(task);\n}But if you want that the employees from the theme park's main entrance do other tasks while the customers are in the queue and still not allowed to enter, I mean if you want that your thread is used for other tasks, at least these 3 libraries have a pattern that may be a problem to you: they block the thread.
\nNow we are in the second part of this article, and as we are talking about Threads, it's good to know some basics about them. The first step is to know that they allow us to work concurrently, two tasks running virtually at the same time. But Threads are notoriously complex and error-prone, inaugurating a new sorts of problems like Deadlock, Race Condition, and so on.
\nAs we said at the start of this article, considering that we have limited resources, we have to use Threads with wisdom and be as clever as possible.
\nIf I and my wife are in the kitchen and we have just one knife available, it is appropriate that once one of us stop using the knife we leave it available to the other over the table, so that the other could start working with this piece of cutlery.
\nWhen one forgets about this rule, the other one is kept waiting for the tool, and important work is not held, preventing the flow of a harmonious cuisine.
\nThe computer environment works like this kitchen, we have a limited number of Threads. And when something blocks one thread with a Thread.sleep() or LockSupport.parkNanos(), this Thread goes to the TIMED_WAITING state and during this time it can not be used. It is like the unused knife in the cooker's hand of an unharmonious kitchen.
\nIn practical terms, consider the following code:
\nprivate void guavaRateLimiter() {\nRateLimiter rateLimiter = RateLimiter.create(2);\nlogWithThreadTimeStamped(\"Starting\");\n\nIntStream.rangeClosed(1, 4).forEach(i -> {\n rateLimiter.acquire();\n logWithThreadTimeStamped(\"Task n. \" + i);\n});\n\nlogWithThreadTimeStamped(\"I'm blocked =(\");\nlogWithThreadTimeStamped(\"Finishing\");\n\n}It will output as follow:
\n[12:01:31.972073] Starting Thread: main\n[12:01:31.978520] Task n. 1 Thread: main\n[12:01:32.471752] Task n. 2 Thread: main\n[12:01:32.967231] Task n. 3 Thread: main\n[12:01:33.470331] Task n. 4 Thread: main\n[12:01:33.470662] I'm blocked =( Thread: main\n[12:01:33.470845] Finishing Thread: mainWhat we wanted was that our 4 tasks were completed in a frequency of at most 2 for each second, but the Rate Limiter also has blocked the other tasks.
\nYou may ask, what to do? Perhaps, we should schedule instead of blocking!
\nImagine when you need to go to the doctor. Unless there is an emergency, you will likely have to make an appointment. The schedules are spread in a timeline, considering the doctor's ability to do the entire checkup: anamnesis, some additional exams, and so on. So to say, if the doctor takes an average of 30 minutes to attend to a patient appointment, those appointments should be guaranteed in intervals of 30 minutes.
\nWith this in mind, we could refactor our previous code. If we need that our tasks are executed at a rate of 2 per second, it means that we should schedule our tasks for every 500 milliseconds, which is one second (1000 milliseconds) divided by 2.
\nNow that we know our interval, we have to find a way to keep track of our \"appointments\", to avoid scheduling more than one task to the same interval. Let's use an AtomicInteger class for that.
\nLet's scratch some code:
\nprivate void taskScheduling() {\nvar scheduleExecutor = Executors.newSingleThreadScheduledExecutor();\nvar intervalDiary = new AtomicInteger();\nvar expectedInterval = 500;\nlogWithThread(\"Starting\");\n\nIntStream.rangeClosed(1, 4).forEach(i -> {\n scheduleExecutor.schedule(\n () -> logWithThread(\"Task n. \" + i),\n intervalDiary.getAndAdd(expectedInterval), TimeUnit.MILLISECONDS\n );\n});\n\nlogWithThread(\"I'm NOT blocked =D\");\nlogWithThread(\"Finishing\");\n}\n\nprivate void logWithThread(String message) {\n System.out.println(message + \" Thread: \" + Thread.currentThread().getName());\n}It will output as follows:
\nStarting - Thread: main\nI'm NOT blocked =D Thread: main\nFinishing - Thread: main\nTask n. 1 - Thread: pool-1-thread-1\nTask n. 2 - Thread: pool-1-thread-1\nTask n. 3 - Thread: pool-1-thread-1\nTask n. 4 - Thread: pool-1-thread-1Oh, it worked, but seems like we are cheating, because we just added a new thread to the game. How can we know for sure if that new thread is not being blocked? Well, we can test it like this:
\nprivate void taskSchedulingV2() {\nvar scheduleExecutor = Executors.newSingleThreadScheduledExecutor();\nvar intervalDiary = new AtomicInteger();\nvar expectedInterval = 500;\nscheduleExecutor.execute(() -> logWithThread(\"Starting\"));\n\nIntStream.rangeClosed(1, 4).forEach(i -> {\n scheduleExecutor.schedule(\n () -> logWithThread(\"Task n. \" + i),\n intervalDiary.getAndAdd(expectedInterval), TimeUnit.MILLISECONDS\n );\n});\n\nscheduleExecutor.execute(() -> logWithThread(\"I'm NOT blocked =D\"));\nscheduleExecutor.execute(() -> logWithThread(\"Finishing\"));\n}And voilà:
\nStarting Thread: pool-1-thread-1\nTask n. 1 Thread: pool-1-thread-1\nI'm NOT blocked =D Thread: pool-1-thread-1\nFinishing Thread: pool-1-thread-1\nTask n. 2 Thread: pool-1-thread-1\nTask n. 3 Thread: pool-1-thread-1\nTask n. 4 Thread: pool-1-thread-1The downside of this approach is that the ScheduledExecutorService, that we receive from the newSingleThreadScheduledExecutor() method of the Executors class, uses an unbounded queue. This means that no task will ever be rejected (by the meaning of the unbounded word, it is like an unlimited queue), and because of that, we have the risk of consuming too much memory if we receive tasks more quickly than we can handle them.
\nOf course, we can avoid that in many ways. One approach could be to track the number of scheduled tasks and reject new tasks if we hit the specified threshold. First of all, let's refactor our code so that we use a queue of tasks as the source to be used by the scheduler.
\nprivate void taskSchedulingV3() {\nvar scheduleExecutor = Executors.newSingleThreadScheduledExecutor();\nvar myTasks = new ConcurrentLinkedQueue<Runnable>();\nvar EXPECTED_INTERVAL = 500;\nscheduleExecutor.execute(() -> logWithThread(\"Starting\"));\n\nscheduleExecutor.scheduleAtFixedRate(() ->\n ofNullable(myTasks.poll()).ifPresent(Runnable::run),\n 0, EXPECTED_INTERVAL, TimeUnit.MILLISECONDS\n);\n\nIntStream.rangeClosed(1, 4).forEach(i ->\n myTasks.add(() -> logWithThread(\"Task n.\" + i))\n);\n\nscheduleExecutor.execute(() -> logWithThread(\"I'm NOT blocked =D\"));\nscheduleExecutor.execute(() -> logWithThread(\"Finishing\"));\n}Our log:
\nStarting Thread: pool-1-thread-1\nTask n.1 Thread: pool-1-thread-1\nI'm NOT blocked =D Thread: pool-1-thread-1\nFinishing Thread: pool-1-thread-1\nTask n.2 Thread: pool-1-thread-1\nTask n.3 Thread: pool-1-thread-1\nTask n.4 Thread: pool-1-thread-1Now in this version, we are using the scheduleAtFixedRate() method, which frees us from keeping track of the already used intervals. It now checks for new tasks for every expected interval.
\nBut what would happen if we have more tasks coming than we are able to handle? The following code creates a new task for each 250ms but handles new tasks just in an interval of 500ms.
\nprivate void taskSchedulingOverload() {\nvar scheduleExecutor = Executors.newSingleThreadScheduledExecutor();\nvar myTasks = new ConcurrentLinkedQueue<Runnable>();\nvar EXPECTED_INTERVAL = 500;\nvar SMALLER_INTERVAL = 250;\nscheduleExecutor.execute(() -> logWithThread(\"Starting\"));\n\nscheduleExecutor.scheduleAtFixedRate(() ->\n ofNullable(myTasks.poll()).ifPresent(Runnable::run),\n 0, EXPECTED_INTERVAL, TimeUnit.MILLISECONDS\n);\n\n\nscheduleExecutor.scheduleAtFixedRate(() ->\n myTasks.add(() -> logWithThread(\"Fast Task! Task lasting: \" + myTasks.size())),\n 0, SMALLER_INTERVAL, TimeUnit.MILLISECONDS\n);\n\n\nscheduleExecutor.execute(() -> logWithThread(\"I'm NOT blocked =D\"));\nscheduleExecutor.execute(() -> logWithThread(\"Finishing\"));\n}Our output would be something like this:
\nStarting Thread: pool-1-thread-1\nI'm NOT blocked =D Thread: pool-1-thread-1\nFinishing Thread: pool-1-thread-1\nFast Task! Task lasting: 1 Thread: pool-1-thread-1\nFast Task! Task lasting: 2 Thread: pool-1-thread-1\nFast Task! Task lasting: 3 Thread: pool-1-thread-1\nFast Task! Task lasting: 4 Thread: pool-1-thread-1\nFast Task! Task lasting: 5 Thread: pool-1-thread-1\nFast Task! Task lasting: 6 Thread: pool-1-thread-1\nFast Task! Task lasting: 7 Thread: pool-1-thread-1\nFast Task! Task lasting: 8 Thread: pool-1-thread-1\nFast Task! Task lasting: 9 Thread: pool-1-thread-1\n\n(...)The size of the queue will not stop raising, leading us to an out-of-memory error. Oh, but don't be so afraid my friend, as we said we can limit the schedule based on that queue. Like if you call the doctor to make an appointment and they answer you: \"sorry, for the next 10 years all our schedules are already taken, please call in another day (or year).\"
\nprivate void taskSchedulingLimitingByQueueSize() {\nvar scheduleExecutor = Executors.newSingleThreadScheduledExecutor();\nvar myTasks = new ConcurrentLinkedQueue<Runnable>();\nvar EXPECTED_INTERVAL = 500;\nvar SMALLER_INTERVAL = 250;\nvar QUEUE_LIMIT = 4;\nscheduleExecutor.execute(() -> logWithThread(\"Starting\"));\n\nscheduleExecutor.scheduleAtFixedRate(() ->\n ofNullable(myTasks.poll()).ifPresent(Runnable::run),\n 0, EXPECTED_INTERVAL, TimeUnit.MILLISECONDS\n);\n\n\nscheduleExecutor.scheduleAtFixedRate(() -> {\n if (myTasks.size() < QUEUE_LIMIT)\n myTasks.add(() -> logWithThread(\"Fast Task! Task lasting: \" + myTasks.size()));\n else\n logWithThread(\"I was rejected =(\");\n },\n 0, SMALLER_INTERVAL, TimeUnit.MILLISECONDS\n);\n\nscheduleExecutor.execute(() -> logWithThread(\"I'm NOT blocked =D\"));\nscheduleExecutor.execute(() -> logWithThread(\"Finishing\"));\n}Our output:
\nStarting Thread: pool-1-thread-1\nI'm NOT blocked =D Thread: pool-1-thread-1\nFinishing Thread: pool-1-thread-1\nFast Task! Task lasting: 1 Thread: pool-1-thread-1\nFast Task! Task lasting: 2 Thread: pool-1-thread-1\nFast Task! Task lasting: 3 Thread: pool-1-thread-1\nI was rejected =( Thread: pool-1-thread-1\nFast Task! Task lasting: 3 Thread: pool-1-thread-1\nI was rejected =( Thread: pool-1-thread-1\nFast Task! Task lasting: 3 Thread: pool-1-thread-1\nI was rejected =( Thread: pool-1-thread-1\nFast Task! Task lasting: 3 Thread: pool-1-thread-1Here we have to talk about backpressure. Remember that this problem about memory only occurs if always the input of tasks is faster than the output. If the input of tasks was faster in just a point of time, this surplus of tasks will be kept on the queue and properly handled in the following time. So with this approach, our application could control the rate of events so that fast producers (sending new tasks) does not overwhelm our consumer.
\nIn a blocking approach, the producer will be forced to wait. In the scheduling approach, the new tasks will be handled in time, and only if it overloads that some of them will be queued.
\nYou could say, but how long should be my queue? Well, I may say that it depends on your use case. A doctor could have a schedule book for all the current year. And if it shows to be not enough, the doctor can buy a new schedule book for the next year. As in your application service, if you can afford more memory, your queues can be as long as you want.
\nBut in the case of a traveling circus, if it will be in the town for just 3 weeks it doesn't make sense to accept schedules for the next month. The same as your application, if its capacity of handling tasks isn't fast enough at a point that the handling of an older schedule task starts to become unmeaningful, you should shrink your queue.
\nWe can conclude that scheduling is a good approach for rate-limiting without the downside of blocking the thread. It gives us the opportunity of controlling the flow of incoming tasks, queueing peaks, and always handle the tasks in harmony with all other components.
","fields":{"slug":"/rate-limiters/","tags":["auto1","engineering","RateLimiter","Thread","Scheduling"]}},"categoryArticles":{"edges":[{"node":{"id":"cb45e6dc-ba61-576d-adc2-ba609b0e722a","frontmatter":{"category":"Engineering","title":"Go 1.23: the new Pattern property in http.Request","date":"2024-11-26","summary":"A short review of the new Pattern property in net/http.Request","thumbnail":null,"authorName":"Andrei Gusev","authorDescription":"Senior Software Engineer at AUTO1 Group","authorAvatar":{"relativePath":"pages/go-123-the-new-pattern-property-in-http-request/avatar_Andrei.jpg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAUBBgT/xAAWAQEBAQAAAAAAAAAAAAAAAAAAAQL/2gAMAwEAAhADEAAAAaeTsi2mDmfWV0wzf//EAB0QAAICAQUAAAAAAAAAAAAAAAECAAMEEBESEyH/2gAIAQEAAQUCtYiUvyE71ZkuQNuzaYvuQTP/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAEDAQE/AR//xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAcEAACAgIDAAAAAAAAAAAAAAABEQACEBIhMTL/2gAIAQEABj8C4KnprGnRhqdnHWpWKPH/xAAdEAACAgMAAwAAAAAAAAAAAAABEQAhEEFRMXGB/9oACAEBAAE/IRoFjyQHGILdksEYAYLXeSuANz4mvAhI736hHP/aAAwDAQACAAMAAAAQNBc+/8QAFhEAAwAAAAAAAAAAAAAAAAAAABAR/9oACAEDAQE/ECv/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/EB//xAAcEAEBAAMBAAMAAAAAAAAAAAABEQAhUTEQYaH/2gAIAQEAAT8Q0H5agDhjJzCu2+nxvKetJ9/fd9x0k6GIzjdTPf69KX9xcAAKlBR2wEDDmf/Z","width":50,"height":50,"src":"/static/479e481648d69b84a342bd2575a225d4/d2d31/avatar_Andrei.jpg","srcSet":"/static/479e481648d69b84a342bd2575a225d4/d2d31/avatar_Andrei.jpg 1x,\n/static/479e481648d69b84a342bd2575a225d4/0b804/avatar_Andrei.jpg 1.5x,\n/static/479e481648d69b84a342bd2575a225d4/753c3/avatar_Andrei.jpg 2x,\n/static/479e481648d69b84a342bd2575a225d4/31ca8/avatar_Andrei.jpg 3x"}}},"headerImage":null},"html":"In Go 1.23, a new Pattern property in net/http.Request was added.\nIt contains the route pattern used to handle the request. This improvement may have gone unnoticed by many, but it can\nsimplify request processing and improve performance in cases where we need to obtain the matched routing pattern\nfor the current request. Let’s see how we can use this feature.
At AUTO1, we have more than 200 microservices, some of which are written in Go and include an HTTP server.\nIt is critically important for us to collect metrics from these services in order to quickly respond to incidents,\nmake data-driven decisions, and optimize system performance.\nFor example, we collect http_requests_total metric:
requestsTotal = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"http_requests_total\",\n\t\tHelp: \"total request count\",\n\t}, []string{\"path\", \"method\", \"code\"})Obviously, for the path label, we should store the path template, not the specific value, for example, /test/{id} instead of /test/123.\nOtherwise, we risk a cardinality explosion, which can significantly increase the amount of stored data. You can read how to choose labels properly in \"Prometheus best practices\".
Previously, to extract the route pattern, we had to pass the multiplexer to a middleware function:
\nfunc metricsMiddleware(next http.Handler, mux *http.ServeMux) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t_, pattern := mux.Handler(r)\n\t\t\n\t\t...\n\t\trequestsTotal.WithLabelValues(pattern, method, statusCode).Inc()\n\t}\n}In this approach, we matched the request to the route a second time, and that negatively affected performance.
\nStarting with Go 1.23, we can get the route pattern directly from the request, without passing the multiplexer to the middleware:
\nfunc middleware(f http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tpattern := r.Pattern\n\t\t\n\t\t...\n\t\trequestsTotal.WithLabelValues(pattern, method, statusCode).Inc()\n\t})\n}This significantly simplifies the middleware, making the code cleaner and more readable. We are no longer dependent on ServeMux and no longer need to re-match the request.
It is important to remember that Request.Pattern is only available after the request has been matched to a route. For example:
func main() {\n\tmux := http.NewServeMux()\n\tmux.Handle(\"/test/{id}\", routeMiddleware(handler()))\n\n\tsrv := http.Server{\n\t\tAddr: \":8080\",\n\t\tHandler: globalMiddleware(mux),\n\t}\n\n\tif err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) {\n\t\tlog.Fatalf(\"server error: %s\", err)\n\t}\n}\n\nfunc routeMiddleware(h http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tlog.Printf(\"ROUTE pattern: %s\", r.Pattern)\n\t\th.ServeHTTP(w, r)\n\t})\n}\n\nfunc globalMiddleware(h http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tlog.Printf(\"GLOBAL pattern: %s\", r.Pattern)\n\t\th.ServeHTTP(w, r)\n\t\tlog.Printf(\"GLOBAL pattern after: %s\", r.Pattern)\n\t})\n}If we send a request GET http://localhost:8080/test/123, in logs we will see:
2024/11/10 13:20:37 GLOBAL pattern: \n2024/11/10 13:20:37 ROUTE pattern: /test/{id}\n2024/11/10 13:20:37 GLOBAL pattern after: /test/{id}This is the expected behavior. The multiplexer implements the http.Handler interface, and inside the ServeHTTP method, it searches for the handler that matches the request. Starting from Go 1.23, it fills the Pattern property of the request with the matching route pattern:
// net/http/server.go\nfunc (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {\n\t...\n\tvar h Handler\n\tif use121 {\n\t\th, _ = mux.mux121.findHandler(r)\n\t} else {\n\t\th, r.Pattern, r.pat, r.matches = mux.findHandler(r)\n\t}\n\th.ServeHTTP(w, r)\n}Thus, the r.Pattern is filled only after the ServeHTTP method of the multiplexer has been called, and the logs confirm this behavior.
In the last two Go releases, the net/http package has received significant improvements that reduced the gap with third-party packages, like gorilla/mux. Now, net/http.ServeMux supports HTTP methods and wildcards in routing patterns, bringing its functionality closer to what third-party packages previously provided.
Additionally, the introduction of the net/http.Request.Pattern property in Go 1.23 has simplified middleware development by allowing us to access the route pattern without needing to re-match the request, improving performance and reducing code complexity.
With each of these improvements, there are fewer reasons to choose third-party routers. While the choice between the standard package and third-party routers used to be more obvious, today I would recommend starting with the standard net/http package.
Paginated APIs, often encountered when dealing with servers and databases, break down large datasets into manageable chunks or pages. This methodology not only optimizes resource usage but also enhances the overall user experience by delivering content progressively. However, integrating and managing paginated data in a mobile app can be a challenge, from maintaining loading states to ensuring a seamless transition between pages and supporting content filtering.
\nIn this guide I will show you how to work with paginated APIs in Android with Jetpack Compose and Paging 3 libraries.
\n
\n![\"Paging](\"https://developer.android.com/static/topic/libraries/architecture/images/paging3-library-architecture.svg\")
The key components of the Paging 3 architecture include
\nWe start with the data layer and define a Retrofit API for fetching movies from network service.
\ninterface MoviesApi {\n @GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"api_key\") api_key: String,\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n ): Response<MoviesNetworkResponse>\n}Next, we will leverage the Room library, which is compatible with Paging 3, to create a local paging source.
\n@Dao\ninterface MoviesDao {\n @Query(\"SELECT * FROM movies ORDER BY id ASC\")\n fun getMovies(): DataSource.Factory<Int, MovieDbEntity>\n}It's important to highlight that we utilize DataSource.Factory as the return type instead of PagingSource. This choice is made specifically because DataSource.Factory offers the map() method, enabling us to map MovieDbEntity instances into domain layer models.
Now we need to put together local and remote data sources in Remote mediator.
\ngetRemoteKey().nextPage) is stored in the database after each successful network request.val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)For bidirectional pagination support, provide the preceding page number for the PREPEND load type. The REFRESH load type is employed when content is refreshed, initiating loading from the initial page.
\nCall our network data source method to fetch a page of data from the server.
\nval movies = moviesDataSource.discover(page)The RemoteMediator code will appear as follows:
\nclass MoviesRemoteMediator(\n private val repository: MoviesRepository,\n private val moviesDataSource: NetworkMoviesDataSource,\n) : RemoteMediator<Int, Movie>() {\n\n override suspend fun load(loadType: LoadType, state: PagingState<Int, Movie>): MediatorResult {\n\n return try {\n\n val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)\n\n val movies = moviesDataSource.getMovies(page)\n\n val nextPage = if (movies.isNotEmpty()) page + 1 else null\n\n repository.insertMovies(movies, nextPage, loadType == LoadType.REFRESH)\n\n Success(endOfPaginationReached = movies.isEmpty())\n\n } catch (e: Exception) {\n MediatorResult.Error(e)\n }\n }\n}RemoteMediator uses the repository to store movies list and next page number in the database.\nRepository also takes care of clearing the database table when data is refreshed:
\nclass MoviesRepositoryImpl(\n private val database: MoviesDatabase\n): MoviesRepository {\n\n override fun getMovies() = database.movies().getMovies().map { it.toDomain() }\n\n override suspend fun insertMovies(movies: List<Movie>, nextPage: Int?, clear: Boolean){\n\n database.withTransaction {\n if (clear) {\n database.movies().clear()\n }\n\n database.movies().putMovies(movies.map { it.toDbEntity() })\n\n if (nextPage != null) {\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, nextPage))\n }\n }\n }\n}The Pager object serves as a bridge between remote and local data sources. It is managing the page loading, prefetching, and handling the transitions between various data sources to provide a seamless and responsive flow of paginated content within your application. You can configure the page loading behavior with the following parameters:
\nclass MoviesScreenViewModel(\n moviesSource: MoviesRemoteMediator,\n repository: MoviesRepository,\n private val movieDetails: MovieDetailsApi,\n) : ViewModel() {\n val moviesFlow = Pager(\n config = PagingConfig(\n pageSize = PAGE_SIZE,\n prefetchDistance = PREFETCH_DISTANCE,\n initialLoadSize = INITIAL_LOAD,\n ),\n remoteMediator = moviesSource,\n pagingSourceFactory = repository.getMovies().asPagingSourceFactory()\n ).flow.cachedIn(viewModelScope)\n}Now, let's explore how to consume the paginated data in your Compose UI.
\nUse the collectAsLazyPagingItems extension function to collect the PagingData and observe changes.
val movies = viewModel.moviesFlow.collectAsLazyPagingItems()Use the LazyColumn or LazyVerticalGrid to efficiently display paginated data in your Compose UI.
LazyVerticalGrid(\n modifier = Modifier\n .fillMaxSize()\n .padding(16.dp),\n columns = GridCells.Fixed(3),\n horizontalArrangement = Arrangement.spacedBy(16.dp),\n verticalArrangement = Arrangement.spacedBy(16.dp),\n ) {\n\n items(\n count = movies.itemCount,\n key = movies.itemKey { it.id },\n ) { index ->\n\n movies[index]?.let { movie ->\n MovieCardSmall(\n movie = movie,\n onClick = { onClick(movie.id) },\n )\n }\n }\n }\n\nNote the usage of the key property for items. The key property uniquely identifies each item, allowing Compose to efficiently update and recycle composables as data changes.
\n
LazyPagingItems offers insights into the ongoing data loading process, allowing you to effectively manage errors during pagination. By examining the loadState.append property, you can determine the loading state when appending new items and present a user-friendly indication of the current status.
\nFor instance, the following code snippet demonstrates how to react to different loading states during item appending:
\nwhen (movies.loadState.append) {\n is LoadState.Loading -> { ProgressFooter() }\n is LoadState.Error -> { ErrorFooter(onRetryClick = { movies.retry() } }\n}Similarly, you can assess the loading status when the list data is refreshed by checking the loadState.refresh property. Extension functions isError() and isLoading() provide convenient checks for error and loading states, respectively.
\nfun LazyPagingItems<Movie>.isError(): Boolean = \n loadState.refresh is LoadState.Error && itemSnapshotList.isEmpty()\n\nfun LazyPagingItems<Movie>.isLoading(): Boolean = \n loadState.refresh is LoadState.Loading && itemSnapshotList.isEmpty()Incorporate these checks into your UI logic to display appropriate screens based on the loading or error states. The following code snippet demonstrates how to handle loading and error states along with providing a retry option:
\nwhen {\n movies.isLoading() -> LoadingScreen()\n movies.isError() -> ErrorScreen { movies.retry() }\n else -> MoviesGrid(movies = movies, onClick = viewModel::onMovieClick)\n}Additionally, LazyPagingItems offers a convenient retry() method, allowing you to retry the last failed data loading attempt with ease.
Now lets add filters to our movie list to see paginated list of movies filtered by genre.
\n![\"Filters](\"/static/paging_demo_filters-ff80a7606fb30961e2bda56bd6ed9985.gif\")
We start with requesting genres from the server and saving them in our database. We will keep genre data and selection in the database so that we can observe it from our domain and UI layers.
\ninterface MoviesApi {\n @GET(\"/3/genre/movie/list?language=en-US\")\n suspend fun genres(@Query(\"api_key\") api_key: String): Response<GenresNetworkResponse>\n}\n\n@Dao\ninterface GenresDao {\n\n @Insert(onConflict = OnConflictStrategy.REPLACE)\n suspend fun putGenres(genres: List<GenreDbEntity>)\n\n @Upsert(entity = GenreDbEntity::class)\n suspend fun updateGenres(genres: List<GenreDbUpdateEntity>)\n\n @Query(\"SELECT * FROM genres\")\n suspend fun getGenres(): List<GenreDbEntity>\n\n @Query(\"SELECT * FROM genres\")\n fun observeGenres(): Flow<List<GenreDbEntity>>\n}Now we can add with_genres parameter to our network request:
@GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n @Query(\"api_key\") api_key: String,\n ): Response<MoviesNetworkResponse>Remote mediator will get selected genres from the repository
\n ...\n val genres = repository.getGenreSelection()\n val movies = moviesDataSource.getMovies(page, genres)\n ...Every time user selects a new filter we stave it in the database and restart pagination by clearing movies in the database and resetting the page to 1.
\nclass MoviesScreenViewModel {\n\n fun onGenreClick(genreItem: GenreItem) = viewModelScope.launch {\n genresProvider.setSelection(genreItem.id, !genreItem.isSelected)\n repository.clearMovies()\n }\n \n}class MoviesRepositoryImpl {\n\n override suspend fun clearMovies() = withContext(Dispatchers.IO) {\n database.withTransaction {\n database.movies().clear()\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, 1))\n }\n }\n\n}
Source code: https://github.com/auto1-oss/android-compose-paging\n
Official documentation: https://developer.android.com/topic/libraries/architecture/paging/v3-overview
Sense of Security
\nTreat the injured first
\nTrust your tools
\nCommunication via Radio
\nWhat I present today is how to use police techniques and their mindset to detect and solve bugs in software, so that you can be the best software developer that society can provide.
\nFor some years, I worked as a police officer in Brazil and I can say that it is a challenging career. The first step was to pass a national test, where knowledge of various laws was required. Criminal laws, traffic laws, and even the laws of physics.
\nThe second step was to pass a battery of medical exams. Almost perfect health was necessary to be able to practice the profession, in addition to having most of the parts of the human body.
\nThe third step was the psychological examination. It was necessary to have a clear mind and not have impulses different from those expected from a person who must protect other people.
\nAs a fourth part, there was also the fitness test, a test in which career candidates had to prove they had sufficient strength and speed to carry out daily police tasks. Running, pull-ups, sit-ups and long jump are examples of what was required.
\nThe last stage was the Police Academy, where for three months those from now on known as students would learn and prove the learning of several new skills: first aid, use of firearms (pistols, carbines, submachine guns, etc.), non-lethal weapons, martial arts, public policy, and many other disciplines.
\nAll this so that society receives the best police officer that its people can provide. So, what about using some of the following principles and techniques to become the best software developer that society can provide?
\nWithout needing to be a security expert, you already know that it is impossible to have a completely safe city. Even if we reached the point of having perfect police officers, which in itself is impossible, we would not have a police officer for every citizen. In general, the recommendation is 300 police officers for every 100 thousand inhabitants.
\nAware of this limitation, that it is not possible to guarantee complete security for all citizens, police agencies work with the concept of a sense of security. Through strategic actions to combat crime, citizens' feeling that they are safe is increased. This mental condition gives them peace of mind when working, studying, and carrying out their daily tasks.
\nThis sense of security also affects criminals. The common visibility of police officers carrying out their duties in strategic places, as well as effective actions to combat crime, make criminals stay away from that place.
\nNotice that ending 100% of crime is impossible, what we do well is limit its actions. From here we can extract our first lesson when dealing with software: perfect code, totally bug-free, is impossible.
\nDespite this conclusion, the fact that it is impossible does not mean that we should simply ignore the problem. On the contrary, we can, like the police, create strategic actions to combat Bugs. To the point that Bugs themselves will run away from our code.
\nTest Driven Development - TDD is a strategic action that provides a feeling of security in your code. Before the crime occurs, you position police officers in a specific region. Before the Bug occurs, you position the test in a specific place in your code.
\nRobert C. Martin, in his book \"Clean Code: A HandBook of Agile Software Craftsmanship\", teaches us that writing unit tests before writing production code is just the tip of the iceberg in defining TDD. There must be a continuous cycle of improvement, in which there is always a test covering the code you are developing and this security that the code is protected allows you to play with it, and work with it. After all, whatever problem may happen, there is a police-test there to protect you.
\nOne of the functions of Highway Police Officers is to provide first aid in the event of traffic accidents. There are often serious injuries that must be treated as urgently as possible, as even 1 minute can make the difference between life and death.
\nThe first step when arriving at a traffic accident site is to mark the location appropriately to prevent other accidents. We must use lighting devices and other appliances to make it clear to other vehicles that there is a danger there.
\nThe police officer must then check the condition of all the injured to prioritize them according to the severity of each situation.
\nOnce the treatment of each injured person has been completed and, if necessary, they have been taken to the hospital, it is time to survey the accident site. At this stage, the police officer begins to collect all the information relating to the accident to have sufficient information so that a decision can be made on the cause of the accident and, when applicable, any culprits.
\nThis is the mental model that you and I must adopt when we are faced with a Bug. Although several procedures have been adopted to prevent it from occurring, it is a fatality that can happen. It's like when the highway was well designed, the vehicles were up to date with maintenance, but even so, due to carelessness or misunderstanding on the part of the driver, an accident happens.
\nAs in the case of traffic accidents, the first step to take is to secure the environment so that the Bug does not cause other problems. If possible, we can revert the application version to one in which the Bug did not appear. We can use a feature flag to temporarily disable a feature. Regardless of the method used to stop the bleeding, the important thing is that we must focus on treating the injured first rather than looking for culprits.
\nWe have to agree with David Thomas and Andrew Hunt, in their work \"The Pragmatic Programmer: your journey to mastery\" that facing Bugs is a sensitive subject for most developers. Instead of treating the issue as a problem to be solved, some approach it with denial, pointing fingers, lame excuses, or simply apathy. It's exactly the opposite of what is expected!
\nInstead of looking for the culprit in a traffic accident, we must first assist the victims and stop the bleeding. In software, instead of finding out who inserted the Bug, we must solve it.
\nWhen treating an injured person in an emergency, one approach is to use the Glasgow Scale to determine the patient's consciousness. On this scale, one of the components is the verbal response.
\nThe injured person is asked simple questions, and by analyzing the responses, we can assess the level of consciousness and consequently the level of urgency of their care.
\nYou and I must carefully read every error message that is displayed during the Bug, it may contain vital information so that we can resolve the problem.
\nYou may be thinking, but what if there is no error message? If we are just faced with an incorrect result? Well, this also happens in traffic accidents, it is not uncommon for there to be an unconscious victim who cannot answer questions.
\nWhen we discussed the sense of security, we talked about the importance of tests in our application. Go and build a test that can reproduce that Bug, so you can extract the information you need to solve it.
\nThink of these tests like the motor tests that highway patrol officers perform on unconscious victims to check their level of consciousness: do the eyes open in response to pain? Is there a pupillary response to light stimulation? Mentally translate this into software language: Does the Bug repeat itself when I enter this and that parameter? And if I insert this new argument, does it happen too?
\nTo decide on a Bug, data is needed. If you don't receive this data, you must extract it.
\nPolice work is very complex and your day-to-day job involves the fight for life and protection of the most important assets for society. There would be no point in having a modern and efficient selection and training process for the development of police officers if they were not given the appropriate tools for their work.
\nAnd when I say appropriate, I don't mean they are minimally capable of performing the task, as the result delivered is directly related to the quality and safety of the tools. For an excellent result, not only the police officers but their tools must also be excellent.
\nIn the police agency I worked for, one of the work tools used was an Austrian pistol known worldwide for its safety and efficiency. The Glock 17 pistol was developed to meet the strict criteria of the Austrian army who wanted to replace the model used during the Second World War.
\nIn addition to requirements on durability, efficiency, and reliability, safety against accidental firing was one of the pivots of the new model.
\nBut how did a knife factory manage to deliver the pistol model that would prove to be the safest? Well, it can be said that, in addition to intense testing, the secret to reliability was simplicity and maintainability. The person responsible for the project designed the equipment with as few parts as possible, minimizing complexity. Today the Glock pistol is made with an average of 35 parts, which is a much smaller quantity than other pistols on the market.
\nWith a simpler design, so-called first-level maintenance is much easier. Equipment users can easily perform maintenance procedures, making the equipment's efficiency last for decades.
\nOf course, you must already remember the principle with the acronym KISS, which stands for Keep it Simple (omitting the last word because I don't want to offend the reader). The secret of that renowned pistol model was not in the used materials, but in being simple. The secret of maintaining your software in good shape is to keep it simple, having a specific component for each task. When a problem arises, you will know exactly which component to look for, as each component has a responsibility, this is the principle of Single Responsibility.
\nBecause of all this, when you are at a shooting range practicing your aim, the police officer knows that if he is not hitting the target, most likely, the error is in his procedure and not in the equipment.
\nAllan Kelly, in one of his contributions to the Book \"97 Things Every Programmer Should Know\", corroborates this idea that we should trust our tools when dealing with Bugs. Considering our tools are widely used, mature, and in use by multiple companies, we have little reason to doubt their quality.
\nOf course, a prototype, version 0.1, is much more fragile than a library that has been on the market for several years. We are also not saying that it is impossible to have bugs in compilers.
\nBut considering how rare compiler bugs are, what we are saying is that in the search for a Bug solution, we should trust our tools and look at our piece of code first.
\nImagine it as an efficient distribution of efforts. The library or compiler code has already been reviewed and tested for many years by countless people. As for that little piece of code of yours, only God knows who besides you has looked at it on your team.
\nOn one of those days, I managed to solve an unpleasant bug that occurred in one of our services. It was a non-deterministic error in our test suite, that is, the Bug only pushed out its head when no one was looking.
\nThis type of non-deterministic error generally involves three types of origin: multithread, time sensitivity, and dissynchrony between construction and cleaning of test data.
\nAnalyzing the body of the failed test, after several hours, of course, I began to strongly distrust the way we created timestamps with a native Java library. After all, the test seemed perfect to me, if there was an error it must have been somewhere else.
\nBut, aware of this lesson that we should trust our tools, especially since the component in question had been in Java for 9 years, I ignored this distrust. This saved me a lot of time and embarrassment, as I'm going to share that the error was elsewhere.
\nOur test suite ran all tests in a single thread, so it wouldn't be a concurrency issue. But it also ran each test in a deterministic but unpredictable order. Therefore, the source of the intermittent problem must be in some specific order in which the tests were performed.
\nAfter several hours of analyzing the logs, I was able to identify two tests that failed when run in sequence. It turned out that, although our tests performed a test data cleaning routine after each execution, a simple typo had silently overwritten the cleaning method in one of the tests, which generated data inconsistency if such tests were executed in a specific order.
\nI remember that one of the first systems I developed when I worked in the police was one to replace the sending of statistics through radio communication. Many years ago, each police station was supposed to send daily work statistics via radio. Data such as number of people and vehicles inspected, car accidents, etc.
\nThis took a lot of time as the audio quality of radio communications was not always ideal. The interlocutors spent a lot of time transmitting and confirming each piece of information. All this using Q-Code
\nThe Q-Code is a standardized collection of three-letter codes used in radio communication. The use of Q-code even in voice transmissions makes communication safer and more efficient. Each set of three letters, always starting with the letter Q, has a specific meaning that is known to the interlocutors.
\nI could see that two of those codes were widely used among police officers in radio communication: QAP and QSL.
\nQAP was used when you wanted to transmit a message. Let's say someone wanted to talk to me, so he said on the radio: \"Bruno, QAP?\". This could be translated as \"Bruno, are you ready to hear a message?\"
\nMy answer could be QAP or for example QRM. If I responded with QAP, I was signaling that yes, I was ready to receive the message. If you responded with QRM, I would be informing you that at that moment the communication is being interfered with.
\nDuring communication, the second most used Q-Code appears, QSL. At the end of each message, the sender ends it with the QSL. This means that he is asking the receiver to confirm whether they received the message. It would be something like: \"There is a suspicious vehicle heading towards your team, QSL?\" If I had understood the entire message, I should respond with a simple \"QSL\". In the absence of a response, the sender repeats the message until receiving confirmation from the receiver.
\nEach message is concluded with a QSL, and its reception is confirmed with a QSL. Does this remind you of any transmission control protocol out here, used on the internet?
\nThe main thing here is that a large amount of knowledge is condensed into three letters, in a language that provides interlocutors with efficiency and security. Eric Evans, the author of \"Domain-Driven Design: Tackling Complexity in the Heart of Software\", would agree with me that there is a principle here that should also be adopted in software development.
\nA language specific to the domain, which considers possible interference from the environment, a ubiquitous language that is known by all interlocutors, can be the difference between life and death, between success and damnation of a project.
\nWhen a project does not have its own language for that domain, this creates the need for numerous translations between interlocutors. Developers have to translate for domain experts. These, in turn, have to translate their demands to developers. Even among developers, there may be a need for translations.
\nAll these translations make concepts confusing and the consequences of a lack of mutual understanding can be disastrous.
\nThe importance of this in trying to arrest a Bug goes much further than compilation errors or exceptions. It goes through the swampy characteristic that the software may be logically working almost perfectly, but it is not delivering what was expected by the stakeholder.
\nLet's look at the following pieces of code:
\nif (order.status != completed) {\n // (...)\n}if (order.isFullyRefunded()) {\n // (...)\n}Here we have a small example of two ways of expressing the same situation. In the second piece of code, a common language between developers and domain experts was used. Here, possible logical errors can be easily noticed, without the need to keep in mind what the possible states of the Order are and what the expected state is for that case.
\nSee the following code:
\nPayment newPayment = payment.add(10);What do you expect from this code? Does it add ten dollars to the payment? Ten transactions, inbound or outbound? Will the method create a new payment object or will it just change the value of the payment variable and create a new reference called newPayment?
\nA lesson also taken from the book \"Clean Code: A HandBook of Agile Software Craftsmanship\" shows that clear communication must also take place between software developers. Functions must communicate exactly what they are doing.
\nIf you have to look at the function's implementation to know what it's actually doing, that's a big sign that you need to rename it.
\nI'm very grateful that you've made it this far, you've certainly learned a lot about police activity and how some of its principles can be applied to combat and remove Bugs from your software.
\nBy applying this knowledge you will be able to work more confidently in your code, giving you peace in your daily life and also allowing you to experiment and discover new things.
\nYour aim will be improved and you will get straight to the point, avoiding damage instead of first looking for the culprits. You become bolder and go further because you know you can trust your tools. This leaves you more time to refine your code and seek excellence.
\nFinally, not only will your code be safe and efficient, but its result will also be in line with what your customers expect because through effective communication you can accurately deliver what was demanded from you.
\nThe result of all this is that Bug is in jail, lives are no longer in danger, and you can go home at the end of your shift to enjoy the rest of the day with your family which is looking forward to welcoming their hero back.
","fields":{"slug":"/bug-police-officer/","tags":["debugging","testing","TDD","DDD"]}}}]},"authorArticles":{"totalCount":2,"edges":[{"node":{"id":"2cda3518-ba96-5257-97f4-6f0ec1f69eb1","frontmatter":{"category":"Engineering","title":"Bug Police Officer","date":"2023-11-21","summary":"What I present today is how to use police techniques and their mindset to detect and solve bugs in software, so that you can be the best software developer that society can provide.","thumbnail":{"relativePath":"pages/bug-police-officer/bug-police-officer.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAAC20lEQVQ4y32S208TURCHF6rvvPpI/Ad8MzFIIlESaAuoAZVehEppd7fQFqjhIgiBQhEUaoFy6wUCghRrCFKQi4orhgQ0XOIFA8FoqOKTaHyg7e54zpYWUOsks2fOmd98Z85mCOIPqyNtUQf33Xp7XOZl+7oEua3AfvpgzkTZo4hIVq9oC8PQGluh7M4hpdYh0fl2aFFfh1a1AcQX2oGUWV0ol2si7bH/auAQsFptE+C4VesUZUnscPxMMxyLa2FXikXsKnIc4zOca9M5hVhbs1cT0fKzuniBkbTnSqU2iBeaA/KMZg47jmVSB9RSDjXW6BXdgv8+mf8vpDMar02qzlMG+gGkKz1cunI06Lkergid3VF1xR3URrRG2k48HZziRf31wxVrzCI01I4HZNQYh73RNBF4P7sAvUZXGdZMOMajcU1E2G3aRjDuJ4I9oAV+fQPY2fB9fbsE2OHHhg9+bkOP0dWENdP9k4JbtIP4C2qinfzq0LQTN8i+I8HTE0eZoak18H8B3/dN1rezyeKYGZx8h5L8pVjbr7EcYoTpnryG8AVLhuqYLk2Dtir/nnfrzQqA7xMLvs/s1uoylObd91ZRvboPRZUxIX2oNtxpB+osZDV0j44mR7wfKy+CRVsHGsM0++oxAwsTL8BgGGUTVAxc0r0EinJ7jXSvLlQXZmBqCTUQHBW6h1YWMhBPLkKfvsi/XprAJimfQbLCw4kVY9xZ5Qxco0ZZCenxn6MYyCmcAwSlcS1m8B3yHzXw066nh8dTtfOQSD3fldETsF12Emq1Fi6BmueSNLNcJjXDjZVL4GEJCSJybjcFaQtQDd8ZYuwDCQgOtWpgMl35CMRZrkCi3MMN5athuVgIomwPpGS7QCx3Q7WmGKrIm1ySfCSAZhNwTfCtQOwDY8v5DrVXO9wZGZ2Qkmb2J6fe5SipGV6XXIE8tKI9CNPMIEy1otjKiZAGa3ENz0MMzPoNzyOqI3a+TN4AAAAASUVORK5CYII=","width":460,"height":325,"src":"/static/19a57f273a6f7441d951955ce11fae27/b3029/bug-police-officer.png","srcSet":"/static/19a57f273a6f7441d951955ce11fae27/b3029/bug-police-officer.png 1x"}}},"authorName":"Bruno Morais","authorDescription":"Bruno is a Senior Software Engineer at AUTO1 Group.","authorAvatar":{"relativePath":"pages/bug-police-officer/avatar.jpeg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAQDAQX/xAAXAQEBAQEAAAAAAAAAAAAAAAABAwIE/9oADAMBAAIQAxAAAAGmbSadPSZMPIBLBy1//8QAHBAAAwABBQAAAAAAAAAAAAAAAQIDIQAQERIj/9oACAEBAAEFAqOVEql2047LLDc7A+i5H//EABcRAQEBAQAAAAAAAAAAAAAAAAEAEBL/2gAIAQMBAT8BC5jP/8QAGREAAwADAAAAAAAAAAAAAAAAAAECEBET/9oACAECAQE/AW9HSSsf/8QAGhAAAQUBAAAAAAAAAAAAAAAAAQACEBEhQf/aAAgBAQAGPwLBZRa4VFYt5Jj/xAAcEAADAAIDAQAAAAAAAAAAAAAAAREhURAxYXH/2gAIAQEAAT8ho6G+i0ElVBP0c6AzfMaH3OHLLoTIz//aAAwDAQACAAMAAAAQcC98/8QAGREAAwADAAAAAAAAAAAAAAAAABExEFGB/9oACAEDAQE/EGQbRPcf/8QAGxEAAgIDAQAAAAAAAAAAAAAAAAERMVFhscH/2gAIAQIBAT8QuM2dElucejs//8QAGxABAQEBAAMBAAAAAAAAAAAAAREAITFBYXH/2gAIAQEAAT8QEiRCoPrgpVComQJxnQidLfOE1gCkvzQ9j4TPv80LcHh6dw+UV3//2Q==","width":50,"height":50,"src":"/static/8d7223890dda036e0121799406a35506/d2d31/avatar.jpeg","srcSet":"/static/8d7223890dda036e0121799406a35506/d2d31/avatar.jpeg 1x,\n/static/8d7223890dda036e0121799406a35506/0b804/avatar.jpeg 1.5x,\n/static/8d7223890dda036e0121799406a35506/753c3/avatar.jpeg 2x,\n/static/8d7223890dda036e0121799406a35506/31ca8/avatar.jpeg 3x"}}},"headerImage":{"relativePath":"pages/bug-police-officer/bug-police-officer.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsSAAALEgHS3X78AAAC20lEQVQ4y32S208TURCHF6rvvPpI/Ad8MzFIIlESaAuoAZVehEppd7fQFqjhIgiBQhEUaoFy6wUCghRrCFKQi4orhgQ0XOIFA8FoqOKTaHyg7e54zpYWUOsks2fOmd98Z85mCOIPqyNtUQf33Xp7XOZl+7oEua3AfvpgzkTZo4hIVq9oC8PQGluh7M4hpdYh0fl2aFFfh1a1AcQX2oGUWV0ol2si7bH/auAQsFptE+C4VesUZUnscPxMMxyLa2FXikXsKnIc4zOca9M5hVhbs1cT0fKzuniBkbTnSqU2iBeaA/KMZg47jmVSB9RSDjXW6BXdgv8+mf8vpDMar02qzlMG+gGkKz1cunI06Lkergid3VF1xR3URrRG2k48HZziRf31wxVrzCI01I4HZNQYh73RNBF4P7sAvUZXGdZMOMajcU1E2G3aRjDuJ4I9oAV+fQPY2fB9fbsE2OHHhg9+bkOP0dWENdP9k4JbtIP4C2qinfzq0LQTN8i+I8HTE0eZoak18H8B3/dN1rezyeKYGZx8h5L8pVjbr7EcYoTpnryG8AVLhuqYLk2Dtir/nnfrzQqA7xMLvs/s1uoylObd91ZRvboPRZUxIX2oNtxpB+osZDV0j44mR7wfKy+CRVsHGsM0++oxAwsTL8BgGGUTVAxc0r0EinJ7jXSvLlQXZmBqCTUQHBW6h1YWMhBPLkKfvsi/XprAJimfQbLCw4kVY9xZ5Qxco0ZZCenxn6MYyCmcAwSlcS1m8B3yHzXw066nh8dTtfOQSD3fldETsF12Emq1Fi6BmueSNLNcJjXDjZVL4GEJCSJybjcFaQtQDd8ZYuwDCQgOtWpgMl35CMRZrkCi3MMN5athuVgIomwPpGS7QCx3Q7WmGKrIm1ySfCSAZhNwTfCtQOwDY8v5DrVXO9wZGZ2Qkmb2J6fe5SipGV6XXIE8tKI9CNPMIEy1otjKiZAGa3ENz0MMzPoNzyOqI3a+TN4AAAAASUVORK5CYII=","width":600,"height":424,"src":"/static/19a57f273a6f7441d951955ce11fae27/b7ca8/bug-police-officer.png","srcSet":"/static/19a57f273a6f7441d951955ce11fae27/b7ca8/bug-police-officer.png 1x"}}}},"html":"Sense of Security
\nTreat the injured first
\nTrust your tools
\nCommunication via Radio
\nWhat I present today is how to use police techniques and their mindset to detect and solve bugs in software, so that you can be the best software developer that society can provide.
\nFor some years, I worked as a police officer in Brazil and I can say that it is a challenging career. The first step was to pass a national test, where knowledge of various laws was required. Criminal laws, traffic laws, and even the laws of physics.
\nThe second step was to pass a battery of medical exams. Almost perfect health was necessary to be able to practice the profession, in addition to having most of the parts of the human body.
\nThe third step was the psychological examination. It was necessary to have a clear mind and not have impulses different from those expected from a person who must protect other people.
\nAs a fourth part, there was also the fitness test, a test in which career candidates had to prove they had sufficient strength and speed to carry out daily police tasks. Running, pull-ups, sit-ups and long jump are examples of what was required.
\nThe last stage was the Police Academy, where for three months those from now on known as students would learn and prove the learning of several new skills: first aid, use of firearms (pistols, carbines, submachine guns, etc.), non-lethal weapons, martial arts, public policy, and many other disciplines.
\nAll this so that society receives the best police officer that its people can provide. So, what about using some of the following principles and techniques to become the best software developer that society can provide?
\nWithout needing to be a security expert, you already know that it is impossible to have a completely safe city. Even if we reached the point of having perfect police officers, which in itself is impossible, we would not have a police officer for every citizen. In general, the recommendation is 300 police officers for every 100 thousand inhabitants.
\nAware of this limitation, that it is not possible to guarantee complete security for all citizens, police agencies work with the concept of a sense of security. Through strategic actions to combat crime, citizens' feeling that they are safe is increased. This mental condition gives them peace of mind when working, studying, and carrying out their daily tasks.
\nThis sense of security also affects criminals. The common visibility of police officers carrying out their duties in strategic places, as well as effective actions to combat crime, make criminals stay away from that place.
\nNotice that ending 100% of crime is impossible, what we do well is limit its actions. From here we can extract our first lesson when dealing with software: perfect code, totally bug-free, is impossible.
\nDespite this conclusion, the fact that it is impossible does not mean that we should simply ignore the problem. On the contrary, we can, like the police, create strategic actions to combat Bugs. To the point that Bugs themselves will run away from our code.
\nTest Driven Development - TDD is a strategic action that provides a feeling of security in your code. Before the crime occurs, you position police officers in a specific region. Before the Bug occurs, you position the test in a specific place in your code.
\nRobert C. Martin, in his book \"Clean Code: A HandBook of Agile Software Craftsmanship\", teaches us that writing unit tests before writing production code is just the tip of the iceberg in defining TDD. There must be a continuous cycle of improvement, in which there is always a test covering the code you are developing and this security that the code is protected allows you to play with it, and work with it. After all, whatever problem may happen, there is a police-test there to protect you.
\nOne of the functions of Highway Police Officers is to provide first aid in the event of traffic accidents. There are often serious injuries that must be treated as urgently as possible, as even 1 minute can make the difference between life and death.
\nThe first step when arriving at a traffic accident site is to mark the location appropriately to prevent other accidents. We must use lighting devices and other appliances to make it clear to other vehicles that there is a danger there.
\nThe police officer must then check the condition of all the injured to prioritize them according to the severity of each situation.
\nOnce the treatment of each injured person has been completed and, if necessary, they have been taken to the hospital, it is time to survey the accident site. At this stage, the police officer begins to collect all the information relating to the accident to have sufficient information so that a decision can be made on the cause of the accident and, when applicable, any culprits.
\nThis is the mental model that you and I must adopt when we are faced with a Bug. Although several procedures have been adopted to prevent it from occurring, it is a fatality that can happen. It's like when the highway was well designed, the vehicles were up to date with maintenance, but even so, due to carelessness or misunderstanding on the part of the driver, an accident happens.
\nAs in the case of traffic accidents, the first step to take is to secure the environment so that the Bug does not cause other problems. If possible, we can revert the application version to one in which the Bug did not appear. We can use a feature flag to temporarily disable a feature. Regardless of the method used to stop the bleeding, the important thing is that we must focus on treating the injured first rather than looking for culprits.
\nWe have to agree with David Thomas and Andrew Hunt, in their work \"The Pragmatic Programmer: your journey to mastery\" that facing Bugs is a sensitive subject for most developers. Instead of treating the issue as a problem to be solved, some approach it with denial, pointing fingers, lame excuses, or simply apathy. It's exactly the opposite of what is expected!
\nInstead of looking for the culprit in a traffic accident, we must first assist the victims and stop the bleeding. In software, instead of finding out who inserted the Bug, we must solve it.
\nWhen treating an injured person in an emergency, one approach is to use the Glasgow Scale to determine the patient's consciousness. On this scale, one of the components is the verbal response.
\nThe injured person is asked simple questions, and by analyzing the responses, we can assess the level of consciousness and consequently the level of urgency of their care.
\nYou and I must carefully read every error message that is displayed during the Bug, it may contain vital information so that we can resolve the problem.
\nYou may be thinking, but what if there is no error message? If we are just faced with an incorrect result? Well, this also happens in traffic accidents, it is not uncommon for there to be an unconscious victim who cannot answer questions.
\nWhen we discussed the sense of security, we talked about the importance of tests in our application. Go and build a test that can reproduce that Bug, so you can extract the information you need to solve it.
\nThink of these tests like the motor tests that highway patrol officers perform on unconscious victims to check their level of consciousness: do the eyes open in response to pain? Is there a pupillary response to light stimulation? Mentally translate this into software language: Does the Bug repeat itself when I enter this and that parameter? And if I insert this new argument, does it happen too?
\nTo decide on a Bug, data is needed. If you don't receive this data, you must extract it.
\nPolice work is very complex and your day-to-day job involves the fight for life and protection of the most important assets for society. There would be no point in having a modern and efficient selection and training process for the development of police officers if they were not given the appropriate tools for their work.
\nAnd when I say appropriate, I don't mean they are minimally capable of performing the task, as the result delivered is directly related to the quality and safety of the tools. For an excellent result, not only the police officers but their tools must also be excellent.
\nIn the police agency I worked for, one of the work tools used was an Austrian pistol known worldwide for its safety and efficiency. The Glock 17 pistol was developed to meet the strict criteria of the Austrian army who wanted to replace the model used during the Second World War.
\nIn addition to requirements on durability, efficiency, and reliability, safety against accidental firing was one of the pivots of the new model.
\nBut how did a knife factory manage to deliver the pistol model that would prove to be the safest? Well, it can be said that, in addition to intense testing, the secret to reliability was simplicity and maintainability. The person responsible for the project designed the equipment with as few parts as possible, minimizing complexity. Today the Glock pistol is made with an average of 35 parts, which is a much smaller quantity than other pistols on the market.
\nWith a simpler design, so-called first-level maintenance is much easier. Equipment users can easily perform maintenance procedures, making the equipment's efficiency last for decades.
\nOf course, you must already remember the principle with the acronym KISS, which stands for Keep it Simple (omitting the last word because I don't want to offend the reader). The secret of that renowned pistol model was not in the used materials, but in being simple. The secret of maintaining your software in good shape is to keep it simple, having a specific component for each task. When a problem arises, you will know exactly which component to look for, as each component has a responsibility, this is the principle of Single Responsibility.
\nBecause of all this, when you are at a shooting range practicing your aim, the police officer knows that if he is not hitting the target, most likely, the error is in his procedure and not in the equipment.
\nAllan Kelly, in one of his contributions to the Book \"97 Things Every Programmer Should Know\", corroborates this idea that we should trust our tools when dealing with Bugs. Considering our tools are widely used, mature, and in use by multiple companies, we have little reason to doubt their quality.
\nOf course, a prototype, version 0.1, is much more fragile than a library that has been on the market for several years. We are also not saying that it is impossible to have bugs in compilers.
\nBut considering how rare compiler bugs are, what we are saying is that in the search for a Bug solution, we should trust our tools and look at our piece of code first.
\nImagine it as an efficient distribution of efforts. The library or compiler code has already been reviewed and tested for many years by countless people. As for that little piece of code of yours, only God knows who besides you has looked at it on your team.
\nOn one of those days, I managed to solve an unpleasant bug that occurred in one of our services. It was a non-deterministic error in our test suite, that is, the Bug only pushed out its head when no one was looking.
\nThis type of non-deterministic error generally involves three types of origin: multithread, time sensitivity, and dissynchrony between construction and cleaning of test data.
\nAnalyzing the body of the failed test, after several hours, of course, I began to strongly distrust the way we created timestamps with a native Java library. After all, the test seemed perfect to me, if there was an error it must have been somewhere else.
\nBut, aware of this lesson that we should trust our tools, especially since the component in question had been in Java for 9 years, I ignored this distrust. This saved me a lot of time and embarrassment, as I'm going to share that the error was elsewhere.
\nOur test suite ran all tests in a single thread, so it wouldn't be a concurrency issue. But it also ran each test in a deterministic but unpredictable order. Therefore, the source of the intermittent problem must be in some specific order in which the tests were performed.
\nAfter several hours of analyzing the logs, I was able to identify two tests that failed when run in sequence. It turned out that, although our tests performed a test data cleaning routine after each execution, a simple typo had silently overwritten the cleaning method in one of the tests, which generated data inconsistency if such tests were executed in a specific order.
\nI remember that one of the first systems I developed when I worked in the police was one to replace the sending of statistics through radio communication. Many years ago, each police station was supposed to send daily work statistics via radio. Data such as number of people and vehicles inspected, car accidents, etc.
\nThis took a lot of time as the audio quality of radio communications was not always ideal. The interlocutors spent a lot of time transmitting and confirming each piece of information. All this using Q-Code
\nThe Q-Code is a standardized collection of three-letter codes used in radio communication. The use of Q-code even in voice transmissions makes communication safer and more efficient. Each set of three letters, always starting with the letter Q, has a specific meaning that is known to the interlocutors.
\nI could see that two of those codes were widely used among police officers in radio communication: QAP and QSL.
\nQAP was used when you wanted to transmit a message. Let's say someone wanted to talk to me, so he said on the radio: \"Bruno, QAP?\". This could be translated as \"Bruno, are you ready to hear a message?\"
\nMy answer could be QAP or for example QRM. If I responded with QAP, I was signaling that yes, I was ready to receive the message. If you responded with QRM, I would be informing you that at that moment the communication is being interfered with.
\nDuring communication, the second most used Q-Code appears, QSL. At the end of each message, the sender ends it with the QSL. This means that he is asking the receiver to confirm whether they received the message. It would be something like: \"There is a suspicious vehicle heading towards your team, QSL?\" If I had understood the entire message, I should respond with a simple \"QSL\". In the absence of a response, the sender repeats the message until receiving confirmation from the receiver.
\nEach message is concluded with a QSL, and its reception is confirmed with a QSL. Does this remind you of any transmission control protocol out here, used on the internet?
\nThe main thing here is that a large amount of knowledge is condensed into three letters, in a language that provides interlocutors with efficiency and security. Eric Evans, the author of \"Domain-Driven Design: Tackling Complexity in the Heart of Software\", would agree with me that there is a principle here that should also be adopted in software development.
\nA language specific to the domain, which considers possible interference from the environment, a ubiquitous language that is known by all interlocutors, can be the difference between life and death, between success and damnation of a project.
\nWhen a project does not have its own language for that domain, this creates the need for numerous translations between interlocutors. Developers have to translate for domain experts. These, in turn, have to translate their demands to developers. Even among developers, there may be a need for translations.
\nAll these translations make concepts confusing and the consequences of a lack of mutual understanding can be disastrous.
\nThe importance of this in trying to arrest a Bug goes much further than compilation errors or exceptions. It goes through the swampy characteristic that the software may be logically working almost perfectly, but it is not delivering what was expected by the stakeholder.
\nLet's look at the following pieces of code:
\nif (order.status != completed) {\n // (...)\n}if (order.isFullyRefunded()) {\n // (...)\n}Here we have a small example of two ways of expressing the same situation. In the second piece of code, a common language between developers and domain experts was used. Here, possible logical errors can be easily noticed, without the need to keep in mind what the possible states of the Order are and what the expected state is for that case.
\nSee the following code:
\nPayment newPayment = payment.add(10);What do you expect from this code? Does it add ten dollars to the payment? Ten transactions, inbound or outbound? Will the method create a new payment object or will it just change the value of the payment variable and create a new reference called newPayment?
\nA lesson also taken from the book \"Clean Code: A HandBook of Agile Software Craftsmanship\" shows that clear communication must also take place between software developers. Functions must communicate exactly what they are doing.
\nIf you have to look at the function's implementation to know what it's actually doing, that's a big sign that you need to rename it.
\nI'm very grateful that you've made it this far, you've certainly learned a lot about police activity and how some of its principles can be applied to combat and remove Bugs from your software.
\nBy applying this knowledge you will be able to work more confidently in your code, giving you peace in your daily life and also allowing you to experiment and discover new things.
\nYour aim will be improved and you will get straight to the point, avoiding damage instead of first looking for the culprits. You become bolder and go further because you know you can trust your tools. This leaves you more time to refine your code and seek excellence.
\nFinally, not only will your code be safe and efficient, but its result will also be in line with what your customers expect because through effective communication you can accurately deliver what was demanded from you.
\nThe result of all this is that Bug is in jail, lives are no longer in danger, and you can go home at the end of your shift to enjoy the rest of the day with your family which is looking forward to welcoming their hero back.
","fields":{"slug":"/bug-police-officer/","tags":["debugging","testing","TDD","DDD"]}}},{"node":{"id":"d53c49b4-fba1-5071-b078-79bbf0f23344","frontmatter":{"category":"Engineering","title":"A wild Thread appeared! Multithreaded Testing and Thread-safety","date":"2023-02-24","summary":"Today we'll talk a little about how threads work on computers and how to extract more value from concurrent tasks. For this, I will present techniques on how to develop and especially how to test code that uses multiple threads, avoiding the main pitfalls that involve this theme. Of course, along the way, you'll also learn more about Java's Concurrent package, non-blocking algorithms, immutability, and more. However, we need to start by talking a little bit about butter.","thumbnail":null,"authorName":"Bruno Morais","authorDescription":"Bruno is a Senior Software Engineer at AUTO1 Group.","authorAvatar":{"relativePath":"pages/a-wild-thread-appeared-multithreaded-testing-and-thread-safety/avatar.jpeg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAQDAQX/xAAXAQEBAQEAAAAAAAAAAAAAAAABAwIE/9oADAMBAAIQAxAAAAGmbSadPSZMPIBLBy1//8QAHBAAAwABBQAAAAAAAAAAAAAAAQIDIQAQERIj/9oACAEBAAEFAqOVEql2047LLDc7A+i5H//EABcRAQEBAQAAAAAAAAAAAAAAAAEAEBL/2gAIAQMBAT8BC5jP/8QAGREAAwADAAAAAAAAAAAAAAAAAAECEBET/9oACAECAQE/AW9HSSsf/8QAGhAAAQUBAAAAAAAAAAAAAAAAAQACEBEhQf/aAAgBAQAGPwLBZRa4VFYt5Jj/xAAcEAADAAIDAQAAAAAAAAAAAAAAAREhURAxYXH/2gAIAQEAAT8ho6G+i0ElVBP0c6AzfMaH3OHLLoTIz//aAAwDAQACAAMAAAAQcC98/8QAGREAAwADAAAAAAAAAAAAAAAAABExEFGB/9oACAEDAQE/EGQbRPcf/8QAGxEAAgIDAQAAAAAAAAAAAAAAAAERMVFhscH/2gAIAQIBAT8QuM2dElucejs//8QAGxABAQEBAAMBAAAAAAAAAAAAAREAITFBYXH/2gAIAQEAAT8QEiRCoPrgpVComQJxnQidLfOE1gCkvzQ9j4TPv80LcHh6dw+UV3//2Q==","width":50,"height":50,"src":"/static/8d7223890dda036e0121799406a35506/d2d31/avatar.jpeg","srcSet":"/static/8d7223890dda036e0121799406a35506/d2d31/avatar.jpeg 1x,\n/static/8d7223890dda036e0121799406a35506/0b804/avatar.jpeg 1.5x,\n/static/8d7223890dda036e0121799406a35506/753c3/avatar.jpeg 2x,\n/static/8d7223890dda036e0121799406a35506/31ca8/avatar.jpeg 3x"}}},"headerImage":null},"html":"Today we'll talk a little about how threads work on computers and how to extract more value from concurrent tasks. For this, I will present techniques on how to develop and especially how to test code that uses multiple threads, avoiding the main pitfalls that involve this theme.
\nOf course, along the way, you'll also learn more about Java's Concurrent package, non-blocking algorithms, immutability, and more. However, we need to start by talking a little bit about butter.
\nButter is a product that is part of everyday life for many people. Present in cookies, cakes, bread, sweets, and snacks, as well as other culinary items that make our mouths water just imagining.
\nButter is not recent; we have historical records of its manufacturing process over 2500 BC. Most often made with cow's milk but it can also be produced with the milk of other mammals such as sheep, goats, and buffalo.
\nSalt has been added to butter since ancient times as a method of preservation, but over time salt has lost this importance as the supply chain has become generally refrigerated. Today, so to speak, salt is added only for taste.
\nSometimes food colorings are added to butter, in Brazil, for example, it is common to use Annatto seeds.
\nAnd one thing I really like to eat is bread and butter. In fact, a delicious slice of bread just out of the oven with a little butter on top, which starts to melt on the hot bread. Oh, how delicious!
\nSadly, my wife happens to like hazelnut cream on top of her bread. And this created a problem with our breakfast, now we have to get two knives dirty to apply the toppings that we like. A knife for the butter, and another knife for the hazelnut cream.
\nHowever, our concern for the environment and its natural resources, which are not unlimited, made us rethink this process.
\nLet's turn this problem into a software. After all, Moore's law has given us processors with more and more cores, and as nowadays virtually all computers have multiple execution cores, a computer must be able to simulate my kitchen.
\nFirst, we need a straightforward Bread that takes 50 milliseconds to be ready (what a blazing fast oven, huh!?)
\npublic class Bread {\n private String name = \"bread\";\n\n public Bread() {\n try {\n TimeUnit.MILLISECONDS.sleep(50);\n } catch (InterruptedException e) {\n Thread.currentThread().interrupt();\n }\n }\n\n public void setTopping(String toppingName) {\n this.name += \" with \" + toppingName;\n }\n\n @Override\n public String toString() {\n return this.name;\n }\n}Now we need a naive knife that can apply the topping to our bread.
\npublic class Knife {\n private String topping;\n\n public Knife setTopping(String topping) {\n this.topping = topping;\n return this;\n\n }\n\n public Knife applyToppingTo(Bread bread) {\n if (topping != null) {\n bread.setTopping(topping);\n topping = null;\n }\n return this;\n }\n}We can verify that everything is working using a test. I'm using the AssertJ library, a java assertion library which I do recommend that you have a look on at, as it could enhance your tests and improve readability.
\n@Test\nvoid givenKnifeWhenApplyToppingToThenChangeBreadName() {\n var knife = new Knife();\n var bread = new Bread();\n\n knife.setTopping(\"butter\").applyToppingTo(bread);\n\n assertThat(bread).hasToString(\"bread with butter\");\n}It seems good, lets's see what happens when we are hungry and want 10 pieces of bread made at the same time! For that, we will need a basket where we will put all the cooked bread.
\n@Test\nvoid givenHungryThenCook10BreadsAtTheSameTime() {\n var basket = new ArrayList<Bread>();\n var knife = new Knife();\n for (int i = 0; i < 10; i++) {\n var bread = new Bread();\n knife.setTopping(\"butter\")\n .applyToppingTo(bread);\n basket.add(bread);\n }\n assertThat(basket)\n .isNotEmpty()\n .allSatisfy(bread -> assertThat(bread).hasToString(\"bread with butter\"));\n}Oops!, it worked, but it took too much time. Each bread takes 50 milliseconds to be ready, so now the whole process is taking 500 milliseconds. If we consider the real-world cook time for bread, it could take hours to cook all those pieces of bread like this.
\nI know what to do, we just need more workforce! I will call my wife to help me. As I said, the majority of current computer machines have more than one core, which allows us to work in parallel. Joshua Bloch, in his book Effective Java, warns that we should prefer executors, tasks, and streams to threads, but for simplicity let me demonstrate this problem using threads by hand.
\n@Test\nvoid givenHungryThenCook10BreadsWithMyWife() {\n var basket = new ArrayList<Bread>();\n var knife = new Knife();\n Runnable preparation = () -> {\n var bread = new Bread();\n knife.setTopping(\"butter\")\n .applyToppingTo(bread);\n basket.add(bread);\n };\n for (int i = 0; i < 5; i++) {\n var me = new Thread(preparation);\n var myWife = new Thread(preparation);\n me.start();\n myWife.start();\n }\n assertThat(basket)\n .isNotEmpty()\n .allSatisfy(bread -> assertThat(bread).hasToString(\"bread with butter\"));\n}But wait, the test is failing, what is going wrong?
\njava.lang.AssertionError: \nExpecting actual not to be emptyIt is like the basket is empty, but we didn't change the process at all despite adding the new threads. Yes, it is a common problem that every software engineer faces when starting to work with multiple threads (or multi-thread apps).
\nIn fact, there is a thread that is responsible for running the test suite which executes each step on our test in sequential order. But when we delegate part of the process to other threads, it follows its flow without waiting for them to finish their job. When it reaches the assertion, the previously created threads have not finished creating their bread. That is why our basket is empty.
\nPlease, don't listen to this evil inner thought suggesting that it is just a matter of creating a Thread.sleep() or something like that, so that the assertion waits for the other threads. It will give you a slow and fragile test. On one hand, if you give it a big delay time, it will slow down your tests. On the other hand, with a small delay, the result could vary depending on the processing power of the machine where the tests are being executed.
\nThe answer for cases like those is to use some tools from java's concurrent package. We have to ensure that both workers start to do their job at the same time. As a second requirement, we need that the test thread verifies the assertion only after the job is done by the other threads. For this, we can use a CountDownLatch like those below. CountDownLatch is a synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes. It does it by holding the thread until the count reaches 0.
\n@Test\nvoid givenHungryThenCook10BreadsWithMyWife() throws InterruptedException {\n var basket = new ArrayList<Bread>();\n var knife = new Knife();\n var startGate = new CountDownLatch(1);\n var endGate = new CountDownLatch(10);\n Runnable preparation = () -> {\n try {\n startGate.await();\n } catch (InterruptedException e) {\n throw new RuntimeException();\n }\n var bread = new Bread();\n knife.setTopping(\"butter\")\n .applyToppingTo(bread);\n basket.add(bread);\n endGate.countDown();\n };\n for (int i = 0; i < 5; i++) {\n var me = new Thread(preparation);\n var myWife = new Thread(preparation);\n me.start();\n myWife.start();\n }\n\n startGate.countDown();\n endGate.await();\n assertThat(basket)\n .hasSize(10)\n .allSatisfy(bread -> assertThat(bread).hasToString(\"bread with butter\"));\n}The startGate will hold all threads so that they start virtually at the same time when the test execution reaches the startGate.countDown(). After that, the assertion will not be verified before the endGate reaches zero, which will only happen when all threads have finished their tasks.
\nThe bad news is that we have a problem with this current implementation. If you are unlucky enough (or simply repeat this test a few times), you will face the following error:
\njava.lang.AssertionError: \nExpected size:<10> but was:<6> in:\n<[bread with butter,\n bread with butter,\n bread with butter,\n bread with butter,\n bread with butter,\n bread with butter]>It happened because we are using as our basket an ArrayList, which states in its documentation:
\n\"Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally.\"
\nIn other words, this implementation of the List interface is not thread-safe. Once more, to face this issue, we have at our disposal other collection implementations on the concurrent package. Instead of ArrayList, we could use CopyOnWriteArrayList or a ConcurrentLinkedQueue.
\nAs the first one is more efficient in use cases where mutations are less frequent than other operations, let's stick with ConcurrentLinkedQueue.
\nvar basket = new ConcurrentLinkedQueue<>();But as soon as you execute the test repeatedly, you will see errors like those:
\njava.lang.AssertionError: \nExpecting all elements of:\n <[bread with butter,\n bread with butter,\n bread with butter,\n bread with butter,\n bread with null,\n bread with null,\n bread with butter,\n bread with butter,\n bread with butter,\n bread with butter]>\nto satisfy given requirements, but these elements did not:\n\n <bread with null> \nExpecting actual's toString() to return:\n <"bread with butter">\nbut was:\n <bread with null>We should have a problem in another part of the code, could you imagine where it is?
\nThings get worse when my wife starts to make her bread with hazelnut cream:
\n@Test\nvoid givenHungryThenCook10BreadsWithMyWife() throws InterruptedException {\n var basket = new ConcurrentLinkedQueue<>();\n var knife = new Knife();\n var startGate = new CountDownLatch(1);\n var endGate = new CountDownLatch(10);\n\n Function<String, Runnable> preparation = topping -> () -> {\n try {\n startGate.await();\n } catch (InterruptedException e) {\n throw new RuntimeException(e);\n }\n var bread = new Bread();\n knife.setTopping(topping).applyToppingTo(bread);\n basket.add(bread);\n endGate.countDown();\n };\n\n for (int i = 0; i < 5; i++) {\n var me = new Thread(preparation.apply(\"butter\"));\n me.start();\n var myWife = new Thread(preparation.apply(\"hazelnut cream\"));\n myWife.start();\n }\n\n startGate.countDown();\n endGate.await();\n assertThat(basket)\n .hasSize(10)\n .haveExactly(5, new Condition<>(bread -> bread.toString().contains(\"butter\"), \"butter bread\"))\n .haveExactly(5, new Condition<>(bread -> bread.toString().contains(\"hazelnut\"), \"hazelnutcream bread\"));\n}First, notice the use of a new helper method from AssertJ, we specified that the collection should have exactly 5 items that satisfy a condition.
\nThe Condition entity from the AssertJ library helps us to create a whole new set of assertions. If you are interested, please take a look at a delicious example in the appendix of this article.
\nBut let's come back to our bread story. After executing the test now with hazelnut cream, sometimes we face errors that are even strange:
\njava.lang.AssertionError: \nExpecting elements:\n<[bread with hazelnut cream,\n bread with hazelnut cream,\n bread with hazelnut cream,\n bread with hazelnut cream,\n bread with butter,\n bread with hazelnut cream,\n bread with butter,\n bread with butter,\n bread with hazelnut cream,\n bread with hazelnut cream]>\n to have exactly 5 times <butter bread>Why do we have only 3 bread with butter, but 7 bread with hazelnut cream?
\nBefore advancing, let's recap what we have seen until now:
\nMost today's computers have multiple cores, allowing us to get even more value from concurrent tasks;
\nWorking with multiple threads creates some difficulties when we depend on timing to verify our tests' assertions, nothing that couldn't be handled using tools from java's Concurrent package, like CountDownLatch;
\nAssertJ is a java assertion library that helps us to improve our tests also on their readability.
\nWe should keep in mind that not all classes are thread-safe, like ArrayList for example. When many threads will share access to a common collection, we should use one of the thread-safe Collection implementations from the Concurrent package;
\nOk, with all that knowledge in mind let's figure out what is the reason for the weird errors that we are facing.
\nFirst of all, this is a typical symptom of a Race Condition, or more specifically, a Data Race. Quoting Brian Goetz, in the book Java Concurrency in Practice
\n\"A race condition occurs when the correctness of a computation depends on the relative timing or interleaving of multiple threads by the runtime; in other words, when getting the right answer relies on lucky timing.\"
\nOn our baking issue, we have multiple threads fighting to access the same shared property. My wife and I are fighting for toppings at the same knife.
\nIn the real world, one could think of it like this:
\nI get my knife and put butter on that;
\nJust when I leave the knife to take the bread, my wife gets that knife and puts hazelnut cream on it;
\nI take back the knife and, with tears in my eyes, realize that I'm topping my bread with some strange brown cream;
\nNow my wife with her bread in hand stares at an empty knife, trying to figure out who is sabotaging her.
\nNow the fix of our code is just a matter of adding a synchronized block using the knife as the lock. With this, only one thread at a time can take the knife and execute the code inside it. \"Oh! This is the solution for everything, we should synchronize everything!\" Think twice. Synchronization isn't for free; it creates a performance problem, as now all other threads are blocked waiting for the availability of the lock, which in our case is the knife. In our kitchen, we could say that when my wife touches the knife I become blocked, waiting for her. If the phone rings, she should say: \"darling, please answer the phone\". But I will be blocked, I'll not be able to do anything except wait for her to leave the knife. A blocked thread goes to the TIMED_WAITING state, and during this time it can not be used. It is really advisable to keep track of how many threads our applications are using because threads are a scarce resource. But there is light at the end of the tunnel, at least in this concern you should know that Java 19 is coming up with the concept of Virtual threads, in its Project Loom. This new feature will cut the one-to-one relationship between Java threads and OS threads, which means that more than one Java virtual thread will be able to use the same OS thread. With that, when some virtual thread gets blocked by an I/O operation, for example, another virtual thread can perform calculations on that same OS thread. But, as we said, until there we should use our threads with wisdom. \"But, what to do?\", you could ask. Well, if you need better performance you can use Nonblocking Synchronization. Nonblocking algorithms are widely used by JVMs implementations, Operational Systems, and in Java itself to implement locks and other concurrent data structures. Those Nonblocking algorithms are way more complicated to design and implement, but they provide better performance by avoiding the thread block itself and give us other benefits like being immune to Deadlock. To provide a simple example of Deadlock, imagine that my wife and I are in the living room. The remote controller of the TV and the sofa are Locks. To watch TV, a person needs to sit on the sofa and press the button on the remote controller. A Deadlock would occur if I get the remote controller and my wife sits on the sofa. She will be blocked waiting for the remote controller, and I'll be blocked waiting for the sofa. And we will be like that \"until death does us part.\" Let's go back to the Nonblocking algorithm. First of all, we need a way to ensure atomicity to our tasks. And it is possible because modern computer processors have special instructions for managing access to shared data. A common approach is the implementation of the compare-and-swap (CAS) instruction. It guarantees us that the access to the value and its update will occur as a single task. Java's Concurrent package provides us with some so-called atomic classes. Let's take AtomicInteger as an example, its documentation states that it is \"An int value that may be updated atomically\". The following code shows us the same algorithm, but only the last one achieves thread-safe by delegating its thread-safety to an AtomicInteger: And how does AtomicInteger ensure atomicity? Using the CAS algorithm! Its incrementAndGet method executes a code that could be simplified like this: To start, it gets the current value and stores it in a local variable. Then using the compareAndSetInt method it first checks if, at that precise moment, the current value is the same that was earlier fetched. If it is so, it proceeds with the update and then returns True. If the current value was changed by another thread, the compareAndSetInt method returns False and restarts the process. It is worth saying that the compareAndSetInt method is a native method, which uses the most efficient instructions provided by the underlying hardware. With this new knowledge, what about trying it on our bread? Let's remove the blocking synchronization and create the faster bread with butter that a computer could have (and some with hazelnut cream too, for the sake of my wife's happiness). We can do this using an AtomicReference. Of course, we did that after removing the synchronized block from our test. Good, fast, and easy, and without thread blocking! But, please pay close attention to this piece of code. We have some problems there as it is some kind of adaptation of the canonical CAS algorithm. The hard part of building nonblocking algorithms is to isolate the shared memory into an atomic single change. But in our baker issue, we have a two-step action: first, we set the topping to the knife, and then we add the topping to the bread. So we have two states to keep track of: the knife and the bread. As we said, this code works and does not block the thread, but doesn't give us all the advantages of the CAS algorithm, because we still can have a Livelock issue. A Livelock issue can occur even when the thread is not blocked, if it is waiting for an operation that will never succeed. In our code, we used the topping value as a kind of lock. The first thread that moves it from null to another value, it will put all other threads waiting for that value to become null again. It works, but if any thread fails to add the topping to the bread, which moves the topping value back to null, all the other threads will be waiting forever. The control mechanism, when more than one thread fights for the knife topping, is done by retrying again immediately, like in the CAS. But the canonical CAS will not fail indefinitely if any other thread fails. It will just finish its job as soon as no other thread touches the shared memory when it is still working. \"So, what to do!? I'm reading this to find answers, not questions!\" Ok, I'll let you know. The other side of the coin, in the context of thread safety, is Immutability. If we use an immutable object, virtually all problems with atomicity are gone. That's because all problems that we are facing here are based on threads fighting to access the same shared memory. If the memory that they want to access cannot be modified, those risks are gone and thread safety is achieved. We can summarise that an Immutable object is one that can't be changed after its creation. Java introduced the Record, which is a simple way to create immutable objects. We just need to provide the type and name of the fields. The public constructor, equals, hashCode, and toString methods, along with the private, final fields, are all given for free to us, by the compiler. Take a look at the following code: Some could argue that immutable objects are a waste of resources, but it isn't true in the majority of cases. Dealing with immutable objects offers us performance advantages when it frees us from using synchronization or creating defensive copies, it even reduces the impact on generational garbage collection. But talking about defensive copies, is the record really immutable? Look at this test: This task will fail with the following log: We have to keep in mind that in the way that we created the Parent record we have kept a reference to its internals. A record is still unmodifiable, but to achieve immutability you have to avoid exposing its inner state. In our example, we can avoid that by overriding the constructor and coping the list of children's names: Now the test is successful, but we are still publishing the internal state of the record. The getter method for the property childrenNames is publishing the reference to the record's inner state, allowing it to be modified: This code will fail as well: To avoid that you have at least two options: override the getter and return a copy of our list, or the one that I prefer: save our internal state as an unmodifiable list: Like this, every time someone tries to update our inner state, they will get a UnsupportedOperationException, which I believe is what we should expect from trying to modify something supposed to be immutable. What we have seen so far? We learned more about Race Conditions, in real life and in the software, and the main solution tool from Java: the monitor pattern, using the synchronized keyword; After understanding the downside of locking, we were introduced to the compare-and-swap (CAS) nonblocking algorithm, which despite being hard to develop can lead us to better performance and protect us from Deadlock For the cases when we are unable to isolate our shared memory, or even in other scenarios where we just need more protection, we talked about immutability and how immutable objects can grant us thread safety. We then revisited the java Record, which is a fairly easy way to create an immutable object, but that is not immune to bad programming technic. To really achieve immutability in our objects we saw how we should avoid publishing its inner state. Oh, it was an amazing journey, we started with a brief introduction about threads, how to properly test them, and we met some new friends from Java's Concurrency package. With new algorithms and tools under our belt, we are ready to travel to the multithreaded world riding our thread-safe software. I'll now eat my bread with butter, see you soon! Baking delicious cakes with AssertJ's Condition entity:\n@Test\nvoid givenHungryThenCook10BreadsWithMyWife() throws InterruptedException {\n var basket = new ConcurrentLinkedQueue<>();\n var knife = new Knife();\n var startGate = new CountDownLatch(1);\n var endGate = new CountDownLatch(10);\n\n Function<String, Runnable> preparation = topping -> () -> {\n try {\n startGate.await();\n } catch (InterruptedException e) {\n throw new RuntimeException(e);\n }\n synchronized (knife) {\n var bread = new Bread();\n knife.setTopping(topping).applyToppingTo(bread);\n basket.add(bread);\n endGate.countDown();\n }\n };\n\n // (...) omitted for brevity\n}Thread blocking
\nNonblocking Algorithms
\n\nclass Counter {\n private Integer count = 0;\n\n public void incrementCount() {\n this.count = count + 1;\n }\n\n public int getCount() {\n return count;\n }\n}\n\nclass ThreadSafeCounter {\n private AtomicInteger count = new AtomicInteger();\n\n public void incrementCount() {\n this.count.incrementAndGet();\n }\n\n public int getCount() {\n return count.get();\n }\n}\nprivate int value;\n\npublic final int incrementAndGet(int expectedValue) {\n int oldValue;\n do {\n oldValue = value;\n } while (!compareAndSetInt(oldValue, oldValue + 1));\n return oldValue;\n}\npublic class Knife {\n private final AtomicReference<String> topping = new AtomicReference<>();\n\n public Knife setTopping(String topping) {\n while (!this.topping.compareAndSet(null, topping)) {}\n return this;\n }\n\n public Knife applyToppingTo(Bread bread) {\n String currentTopping;\n if ((currentTopping = topping.getAndUpdate(s -> null)) != null)\n bread.setTopping(currentTopping);\n return this;\n }\n}Immutability
\n\npublic record Knife (String topping) {\n\n public Knife setTopping(String topping) {\n return new Knife(topping);\n }\n public Knife applyToppingTo(Bread bread) {\n if (topping != null)\n bread.setTopping(topping);\n return new Knife(null);\n }\n}\n@Test\nvoid givenReferenceToRecordThenItShouldBeImmutable() {\n record Parent(String name, List<String> childrenNames){};\n var childrenNames = new ArrayList<>(List.of(\"Haru\"));\n var parent = new Parent(\"Bruno\", childrenNames);\n\n var parentBeforeNewChild = parent.toString();\n childrenNames.add(\"Ayumi\");\n\n assertThat(parent).hasToString(parentBeforeNewChild);\n}\njava.lang.AssertionError: \nExpecting actual's toString() to return:\n <"Parent[name=Bruno, childrenNames=[Haru]]">\nbut was:\n <Parent[name=Bruno, childrenNames=[Haru, Ayumi]]>\nrecord Parent(String name, List<String> childrenNames){\n public Parent(String name, List<String> childrenNames) {\n this.name = name;\n this.childrenNames = new ArrayList<>(childrenNames);\n }\n};\n@Test\nvoid givenReferenceToRecordThenItShouldBeImmutable() {\n record Parent(String name, List<String> childrenNames){\n public Parent(String name, List<String> childrenNames) {\n this.name = name;\n this.childrenNames = new ArrayList<>(childrenNames);\n }\n }\n var childrenNames = new ArrayList<>(List.of(\"Haru\"));\n var parent = new Parent(\"Bruno\", childrenNames);\n\n var parentBeforeNewChild = parent.toString();\n parent.childrenNames().add(\"Ayumi\");\n\n assertThat(parent).hasToString(parentBeforeNewChild);\n}\njava.lang.AssertionError: \nExpecting actual's toString() to return:\n <"Parent[name=Bruno, childrenNames=[Haru]]">\nbut was:\n <Parent[name=Bruno, childrenNames=[Haru, Ayumi]]>\nrecord Parent(String name, List<String> childrenNames){\n public Parent(String name, List<String> childrenNames) {\n this.name = name;\n this.childrenNames = List.copyOf(childrenNames);\n }\n}Summary so far
\n\n
\nConclusion
\nAppendix
\n","fields":{"slug":"/a-wild-thread-appeared-multithreaded-testing-and-thread-safety/","tags":["Thread-safety","testing","nonblocking_algorithms","immutability"]}}}]}},"pageContext":{"slug":"/rate-limiters/","tags":["auto1","engineering","RateLimiter","Thread","Scheduling"],"category":"Engineering","author":"Bruno Morais","date":"2022-07-18"}}@Test\nvoid testTheCake() {\n record Cake(String name, boolean hasFruit, int weightInKg){};\n\n var chocolateCake = new Cake(\"Chocolate\", false, 5);\n var strawberryCake = new Cake(\"Strawberry\", true, 3);\n\n var bakery = List.of(chocolateCake, strawberryCake);\n\n var heavyCakes = new Condition<Cake>(cake -> cake.weightInKg >= 3, \"heavy cake\");\n var bigChocolate = new Condition<Cake>(cake ->\n cake.name.contains(\"Chocolate\") &&\n cake.weightInKg >= 5\n , \"big chocolate\");\n\n assertThat(bakery)\n .haveExactly(2, heavyCakes)\n .haveExactly(1, bigChocolate);\n}