![\"Paging](\"/static/paging_demo-a325402259bf9864dced311c9eafa7c8.gif\")
BACKGROUND
\nThis post is in addition to the previous article about plugin development but with a focus on UI based plugin implementations. Sometimes it can be really useful to have an UI and make it possible to interact with it by doing some checkboxes selections or maybe to have some input areas for text. Let's investigate how to develop a simple plugin with input text area as input entry point and some new generated classes, based on input text as a result. As a source code example we will have a look at source code based on my pet project that also was used as a development tool in AUTO1 to generate non GraphQl based POJO classes.
\nUSER STORY
\nAs an Android developer I need to interact with backend data. It can be usually presented as JSON data or if we can speak about new and modern practices it also can be presented as a GraphQl scheme. GraphQl is a really powerful framework with its own classes generation framework. But let's go back one step and imagine that we have to do some data presentation classes updated for the project without GraphQl support. To do it, we need to find the target class, compare its fields with new JSON fields and update it. It can be easy if we have only one field changed, but for multiple inner fields and classes with its own inner fields it can be too complicated. We can find some online services to convert JSON to POJO and copy/paste output files, but let's try to integrate this technology to our IDE! And our job will be only to copy/paste new JSON to some window input field! Like this one:
\nTo make it possible, let's think about our problem and try to do small decomposition into logical steps. Lets formulate our goals by imaging simple user stories:
\nAs a user I want to input JSON text into the input field.
\nAs a user I want to transform JSON text into POJO files.
\nAs a user I want to see the result notification.
\nWe will skip JSON to POJO transformation for this article, but we will learn how to develop a plugin with required user interface. Let's think about option number two as about some library that we already have and can use as a part of our small project.
\nTECHNOLOGY STACK
\nBefore we will start let's have a look on basis plugin implementation. There are 2 base options to develop new plugin:
\n\nWe will speak about gradle implementation. Gradle is a really powerful framework and it can be really helpful if we want to simplify our building, releasing and testing. The project structure is a common structure for gradle based projects:
\nmy_gradle_plugin\n\n├── build.gradle\n\n├── gradle\n\n│ └── wrapper\n\n│ ├── gradle-wrapper.jar\n\n│ └── gradle-wrapper.properties\n\n├── gradlew\n\n├── gradlew.bat\n\n├── settings.gradle\n\n└── src\n\n├── main\n\n│ ├── java\n\n│ └── resources\n\n│ └── META-INF\n\n│ └── plugin.xml\n\n└── test\n\n├── java\n\n└── resources\n\n Most important topic for us will be the plugin.xml file. We already discussed some details in the previous article and we can be focused on some UI related topics.
\nDuring the plugin development we can use Java/Kotlin and all utility tools that we want like jUnit testing and dependency injection frameworks. In our current example Koin and JUnit can be found. So, now we have and know about project structure, some utility tools, language to write the code and now we are ready to think how it can be possible to create a user interface and connect it with our IDE. Let's remember about our major goals:
\nGOAL 1 - As a user I want to input JSON text into the input field.\n
\nGOAL 2 - As a user I want to see the result notification.\n
and see how we can implement it.
\nGOAL 1
\nFirst of all, let's learn how to connect our plugin with the IDE user interface. As we can see the IDE has a lot of action groups to do something (create new files for example). According to our idea to have some logic that can help us to convert JSON to POJO it would be great to have a new item in the new file list. It will be something like an action “Create new POJO file”.
\nTo connect our logic with the IDE by following this path we have to remember about the plugin.xml file that was discussed early. We can use this configuration file to describe our plugin and how it can be used.
\nLet's find the actions section and define new action to show our input field dialog view.
\n<actions>\n\n<action id=\"GeneratePOJOAction\" class=\"GeneratePOJOAction\"\n\nicon=\"/icons/pojo.png\"\n\ntext=\"Generate POJO from JSON\"\n\ndescription=\"GeneratePOJOAction\">\n\n<add-to-group group-id=\"NewGroup\" anchor=\"last\"/>\n\n</action>\n\n</actions>As we can see we can use some properties to describe how users will be available to interact with our plugin. We can define a custom icon for our action, define the position inside of the items list (“NewGroup” anchor “last”) and we can define the class to execute when action will be invoked by the user.
\nCustom action class should extend the abstract AnAction class with the implementation of actionPerformed() method.
\nimport com.intellij.openapi.actionSystem.AnAction\n\nimport com.intellij.openapi.actionSystem.AnActionEvent\n\nclass GeneratePOJOAction : AnAction() {\n\noverride fun actionPerformed(e: AnActionEvent) { \n\n//TODO: Add your UI implementation here \n\n} \n}Inside of actionPerformed() method implementation we can add the logic to show a dialog window with the input field for JSON text. To show dialog view we can easily use DialogBuilder from plugin development SDK and Java swing to create a user interface.
\nMore information about actions can be found on the official documentation page.
\nGOAL 2
\nLet's imagine now that we already implemented logic to show the input text dialog with some selectable options to transform this text. We also added some logic of transformation - that can be some library to work with it or maybe our own implementation. And as the result of that transformation some new POJO files were generated added into our project (We can interact directly with the project or we can use project representation, provided by the plugin development SDK).
\nBut what about the next steps? We also need to notify the user about the result: it can be a success result notification, or maybe some error case notification. To implement it we can use default tools, provided by plugin development SDK. We can show some information to the user by using popup, notifications or messages.
\nLet's imagine the following case: we have a text input field and we have some logic to convert/transform the text from this input field. If we can speak about JSON to POJO transformation, we have to understand that any random text cannot be converted to POJO correctly. Target text should satisfy some conditions and also should have the correct JSON structure and it means that we have to not only show some notification if text structure is not valid, but we have to block UI and break our transformation flow.
\nMessages can really help us with it! According to the official documentation:
\nThe Messages class provides a way to show simple message boxes, input dialogs (modal dialogs with a text field), and chooser dialogs (modal dialogs with a combo box). The function of different methods of the class should be clear from their names. When running on macOS, the message boxes shown by the Messages class use the native UI.
\nLets see the following code:
\ncompanion object {\n\nprivate const val BUTTON_TITLE = \"OK\"\n\nprivate const val DEFAULT_OPTION_INDEX = -1\n\nprivate val DEFAULT_ICON = null\n\n}\n\nfun onPluginExceptionHandled(exception: RoboPluginException) {\n\nMessages.showDialog(\n\nexception.message, exception.title,\n\narrayOf(BUTTON_TITLE),\n\nDEFAULT_OPTION_INDEX,\n\nDEFAULT_ICON\n\n)\n\n}As we can see we can use the show dialog method, provided by SDK. We can define some dialog parameters, like title, message, button and of course we can define our custom plugin icon. If the image is not defined, standart Java icon or IDE icon would be shown. More information about images support can be found here. Also as it was already discussed here we can add support for internationalization instead of hardcoding only english string values.
\nAnd as the result we can see this amazing small UI blocking window with our message text:
\nIt looks now that we found a solution to cover the error cases. Now we can prevent users from wrong input and show notification about it. Let's think about the success use case. Let's imagine that the input text was correct and we processed our JSON to POJO transformation as expected. Maybe some new files were generated, maybe it was only one new file. It will be a really good user experience if we can find a possibility to inform our users about it. Maybe one of the possible ways can be to show notification messages. It can be a small popup view with some text, like this one:
\nand maybe we also can print some information to the event log:
\n12:00 Response.java was updated.\n \n12:00 POJO generation: SuccessIt looks now that we are ready to totally satisfy our user story requirements! Let's do the final steps and implement the following code:
\ncompanion object{\n\nprivate const val MESSAGE_SUCCESS = \"POJO generation: Success\"\n\nprivate const val GROUP_ID = \"RoboPOJO Generator\"\n\n}\n\nfun showSuccessMessage() {\n\nshowMessage(\n\nproject = ProjectManager.getInstance().openProjects.first(),\n\nmessage = MESSAGE_SUCCESS,\n\ntype = NotificationType.INFORMATION,\n\n)\n\n}\n\nprivate fun showMessage(\n\nmessage: String, type: NotificationType, project: Project\n\n) = NotificationGroupManager.getInstance()\n\n.getNotificationGroup(GROUP_ID)\n\n.createNotification(message, type)\n\n.notify(project)\n\n As we can see we can use the create notification method, provided by SDK. Unlike dialog related logic, we also have to define the project that we want to use to show the notification. As an example let's get only the first project (most popular use case), but we have to understand that users can open not only one project in the IDE and we have to take care about it too. So, as we can see it is also possible to define message text. Also we can define the message type (it can be INFORMATION, WARNING or ERROR) and we have to take care about the notification group that was defined inside of our plugin.xml file:
\n<extensions defaultExtensionNs=\"com.intellij\">\n\n<notificationGroup id=\"RoboPOJO Generator\"\n\ndisplayType=\"BALLOON\"\n\nkey=\"com.robohorse.robopojogenerator\"/>\n\n</extensions>FEW WORDS AT THE END
\nAs we can see it was not really a big deal to develop some small and hopefully useful tools for daily use or to help your teammates. The major idea is just to understand that integration with IDE can be really simple and when we will finish with it, we can be focused on the functionality of our tool. Perhaps in the future when we will have a stable version with some really cool features we can share our tool not only with teammates that we are working with directly, but maybe it also can be helpful for some developers from some random parts of the world. And maybe together we can simplify our daily job!
","fields":{"slug":"/IntelliJ IDEA plugin development - focus on UI based plugins/","tags":["auto1","engineering","android","mobile","intellij","idea"]}},"categoryArticles":{"edges":[{"node":{"id":"cb45e6dc-ba61-576d-adc2-ba609b0e722a","frontmatter":{"category":"Engineering","title":"Go 1.23: the new Pattern property in http.Request","date":"2024-11-26","summary":"A short review of the new Pattern property in net/http.Request","thumbnail":null,"authorName":"Andrei Gusev","authorDescription":"Senior Software Engineer at AUTO1 Group","authorAvatar":{"relativePath":"pages/go-123-the-new-pattern-property-in-http-request/avatar_Andrei.jpg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAUBBgT/xAAWAQEBAQAAAAAAAAAAAAAAAAAAAQL/2gAMAwEAAhADEAAAAaeTsi2mDmfWV0wzf//EAB0QAAICAQUAAAAAAAAAAAAAAAECAAMEEBESEyH/2gAIAQEAAQUCtYiUvyE71ZkuQNuzaYvuQTP/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAEDAQE/AR//xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/AR//xAAcEAACAgIDAAAAAAAAAAAAAAABEQACEBIhMTL/2gAIAQEABj8C4KnprGnRhqdnHWpWKPH/xAAdEAACAgMAAwAAAAAAAAAAAAABEQAhEEFRMXGB/9oACAEBAAE/IRoFjyQHGILdksEYAYLXeSuANz4mvAhI736hHP/aAAwDAQACAAMAAAAQNBc+/8QAFhEAAwAAAAAAAAAAAAAAAAAAABAR/9oACAEDAQE/ECv/xAAUEQEAAAAAAAAAAAAAAAAAAAAg/9oACAECAQE/EB//xAAcEAEBAAMBAAMAAAAAAAAAAAABEQAhUTEQYaH/2gAIAQEAAT8Q0H5agDhjJzCu2+nxvKetJ9/fd9x0k6GIzjdTPf69KX9xcAAKlBR2wEDDmf/Z","width":50,"height":50,"src":"/static/479e481648d69b84a342bd2575a225d4/d2d31/avatar_Andrei.jpg","srcSet":"/static/479e481648d69b84a342bd2575a225d4/d2d31/avatar_Andrei.jpg 1x,\n/static/479e481648d69b84a342bd2575a225d4/0b804/avatar_Andrei.jpg 1.5x,\n/static/479e481648d69b84a342bd2575a225d4/753c3/avatar_Andrei.jpg 2x,\n/static/479e481648d69b84a342bd2575a225d4/31ca8/avatar_Andrei.jpg 3x"}}},"headerImage":null},"html":"In Go 1.23, a new Pattern property in net/http.Request was added.\nIt contains the route pattern used to handle the request. This improvement may have gone unnoticed by many, but it can\nsimplify request processing and improve performance in cases where we need to obtain the matched routing pattern\nfor the current request. Let’s see how we can use this feature.
At AUTO1, we have more than 200 microservices, some of which are written in Go and include an HTTP server.\nIt is critically important for us to collect metrics from these services in order to quickly respond to incidents,\nmake data-driven decisions, and optimize system performance.\nFor example, we collect http_requests_total metric:
requestsTotal = promauto.NewCounterVec(prometheus.CounterOpts{\n\t\tName: \"http_requests_total\",\n\t\tHelp: \"total request count\",\n\t}, []string{\"path\", \"method\", \"code\"})Obviously, for the path label, we should store the path template, not the specific value, for example, /test/{id} instead of /test/123.\nOtherwise, we risk a cardinality explosion, which can significantly increase the amount of stored data. You can read how to choose labels properly in \"Prometheus best practices\".
Previously, to extract the route pattern, we had to pass the multiplexer to a middleware function:
\nfunc metricsMiddleware(next http.Handler, mux *http.ServeMux) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\t_, pattern := mux.Handler(r)\n\t\t\n\t\t...\n\t\trequestsTotal.WithLabelValues(pattern, method, statusCode).Inc()\n\t}\n}In this approach, we matched the request to the route a second time, and that negatively affected performance.
\nStarting with Go 1.23, we can get the route pattern directly from the request, without passing the multiplexer to the middleware:
\nfunc middleware(f http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tpattern := r.Pattern\n\t\t\n\t\t...\n\t\trequestsTotal.WithLabelValues(pattern, method, statusCode).Inc()\n\t})\n}This significantly simplifies the middleware, making the code cleaner and more readable. We are no longer dependent on ServeMux and no longer need to re-match the request.
It is important to remember that Request.Pattern is only available after the request has been matched to a route. For example:
func main() {\n\tmux := http.NewServeMux()\n\tmux.Handle(\"/test/{id}\", routeMiddleware(handler()))\n\n\tsrv := http.Server{\n\t\tAddr: \":8080\",\n\t\tHandler: globalMiddleware(mux),\n\t}\n\n\tif err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) {\n\t\tlog.Fatalf(\"server error: %s\", err)\n\t}\n}\n\nfunc routeMiddleware(h http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tlog.Printf(\"ROUTE pattern: %s\", r.Pattern)\n\t\th.ServeHTTP(w, r)\n\t})\n}\n\nfunc globalMiddleware(h http.Handler) http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\tlog.Printf(\"GLOBAL pattern: %s\", r.Pattern)\n\t\th.ServeHTTP(w, r)\n\t\tlog.Printf(\"GLOBAL pattern after: %s\", r.Pattern)\n\t})\n}If we send a request GET http://localhost:8080/test/123, in logs we will see:
2024/11/10 13:20:37 GLOBAL pattern: \n2024/11/10 13:20:37 ROUTE pattern: /test/{id}\n2024/11/10 13:20:37 GLOBAL pattern after: /test/{id}This is the expected behavior. The multiplexer implements the http.Handler interface, and inside the ServeHTTP method, it searches for the handler that matches the request. Starting from Go 1.23, it fills the Pattern property of the request with the matching route pattern:
// net/http/server.go\nfunc (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {\n\t...\n\tvar h Handler\n\tif use121 {\n\t\th, _ = mux.mux121.findHandler(r)\n\t} else {\n\t\th, r.Pattern, r.pat, r.matches = mux.findHandler(r)\n\t}\n\th.ServeHTTP(w, r)\n}Thus, the r.Pattern is filled only after the ServeHTTP method of the multiplexer has been called, and the logs confirm this behavior.
In the last two Go releases, the net/http package has received significant improvements that reduced the gap with third-party packages, like gorilla/mux. Now, net/http.ServeMux supports HTTP methods and wildcards in routing patterns, bringing its functionality closer to what third-party packages previously provided.
Additionally, the introduction of the net/http.Request.Pattern property in Go 1.23 has simplified middleware development by allowing us to access the route pattern without needing to re-match the request, improving performance and reducing code complexity.
With each of these improvements, there are fewer reasons to choose third-party routers. While the choice between the standard package and third-party routers used to be more obvious, today I would recommend starting with the standard net/http package.
Paginated APIs, often encountered when dealing with servers and databases, break down large datasets into manageable chunks or pages. This methodology not only optimizes resource usage but also enhances the overall user experience by delivering content progressively. However, integrating and managing paginated data in a mobile app can be a challenge, from maintaining loading states to ensuring a seamless transition between pages and supporting content filtering.
\nIn this guide I will show you how to work with paginated APIs in Android with Jetpack Compose and Paging 3 libraries.
\n
\n![\"Paging](\"https://developer.android.com/static/topic/libraries/architecture/images/paging3-library-architecture.svg\")
The key components of the Paging 3 architecture include
\nWe start with the data layer and define a Retrofit API for fetching movies from network service.
\ninterface MoviesApi {\n @GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"api_key\") api_key: String,\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n ): Response<MoviesNetworkResponse>\n}Next, we will leverage the Room library, which is compatible with Paging 3, to create a local paging source.
\n@Dao\ninterface MoviesDao {\n @Query(\"SELECT * FROM movies ORDER BY id ASC\")\n fun getMovies(): DataSource.Factory<Int, MovieDbEntity>\n}It's important to highlight that we utilize DataSource.Factory as the return type instead of PagingSource. This choice is made specifically because DataSource.Factory offers the map() method, enabling us to map MovieDbEntity instances into domain layer models.
Now we need to put together local and remote data sources in Remote mediator.
\ngetRemoteKey().nextPage) is stored in the database after each successful network request.val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)For bidirectional pagination support, provide the preceding page number for the PREPEND load type. The REFRESH load type is employed when content is refreshed, initiating loading from the initial page.
\nCall our network data source method to fetch a page of data from the server.
\nval movies = moviesDataSource.discover(page)The RemoteMediator code will appear as follows:
\nclass MoviesRemoteMediator(\n private val repository: MoviesRepository,\n private val moviesDataSource: NetworkMoviesDataSource,\n) : RemoteMediator<Int, Movie>() {\n\n override suspend fun load(loadType: LoadType, state: PagingState<Int, Movie>): MediatorResult {\n\n return try {\n\n val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)\n\n val movies = moviesDataSource.getMovies(page)\n\n val nextPage = if (movies.isNotEmpty()) page + 1 else null\n\n repository.insertMovies(movies, nextPage, loadType == LoadType.REFRESH)\n\n Success(endOfPaginationReached = movies.isEmpty())\n\n } catch (e: Exception) {\n MediatorResult.Error(e)\n }\n }\n}RemoteMediator uses the repository to store movies list and next page number in the database.\nRepository also takes care of clearing the database table when data is refreshed:
\nclass MoviesRepositoryImpl(\n private val database: MoviesDatabase\n): MoviesRepository {\n\n override fun getMovies() = database.movies().getMovies().map { it.toDomain() }\n\n override suspend fun insertMovies(movies: List<Movie>, nextPage: Int?, clear: Boolean){\n\n database.withTransaction {\n if (clear) {\n database.movies().clear()\n }\n\n database.movies().putMovies(movies.map { it.toDbEntity() })\n\n if (nextPage != null) {\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, nextPage))\n }\n }\n }\n}The Pager object serves as a bridge between remote and local data sources. It is managing the page loading, prefetching, and handling the transitions between various data sources to provide a seamless and responsive flow of paginated content within your application. You can configure the page loading behavior with the following parameters:
\nclass MoviesScreenViewModel(\n moviesSource: MoviesRemoteMediator,\n repository: MoviesRepository,\n private val movieDetails: MovieDetailsApi,\n) : ViewModel() {\n val moviesFlow = Pager(\n config = PagingConfig(\n pageSize = PAGE_SIZE,\n prefetchDistance = PREFETCH_DISTANCE,\n initialLoadSize = INITIAL_LOAD,\n ),\n remoteMediator = moviesSource,\n pagingSourceFactory = repository.getMovies().asPagingSourceFactory()\n ).flow.cachedIn(viewModelScope)\n}Now, let's explore how to consume the paginated data in your Compose UI.
\nUse the collectAsLazyPagingItems extension function to collect the PagingData and observe changes.
val movies = viewModel.moviesFlow.collectAsLazyPagingItems()Use the LazyColumn or LazyVerticalGrid to efficiently display paginated data in your Compose UI.
LazyVerticalGrid(\n modifier = Modifier\n .fillMaxSize()\n .padding(16.dp),\n columns = GridCells.Fixed(3),\n horizontalArrangement = Arrangement.spacedBy(16.dp),\n verticalArrangement = Arrangement.spacedBy(16.dp),\n ) {\n\n items(\n count = movies.itemCount,\n key = movies.itemKey { it.id },\n ) { index ->\n\n movies[index]?.let { movie ->\n MovieCardSmall(\n movie = movie,\n onClick = { onClick(movie.id) },\n )\n }\n }\n }\n\nNote the usage of the key property for items. The key property uniquely identifies each item, allowing Compose to efficiently update and recycle composables as data changes.
\n
LazyPagingItems offers insights into the ongoing data loading process, allowing you to effectively manage errors during pagination. By examining the loadState.append property, you can determine the loading state when appending new items and present a user-friendly indication of the current status.
\nFor instance, the following code snippet demonstrates how to react to different loading states during item appending:
\nwhen (movies.loadState.append) {\n is LoadState.Loading -> { ProgressFooter() }\n is LoadState.Error -> { ErrorFooter(onRetryClick = { movies.retry() } }\n}Similarly, you can assess the loading status when the list data is refreshed by checking the loadState.refresh property. Extension functions isError() and isLoading() provide convenient checks for error and loading states, respectively.
\nfun LazyPagingItems<Movie>.isError(): Boolean = \n loadState.refresh is LoadState.Error && itemSnapshotList.isEmpty()\n\nfun LazyPagingItems<Movie>.isLoading(): Boolean = \n loadState.refresh is LoadState.Loading && itemSnapshotList.isEmpty()Incorporate these checks into your UI logic to display appropriate screens based on the loading or error states. The following code snippet demonstrates how to handle loading and error states along with providing a retry option:
\nwhen {\n movies.isLoading() -> LoadingScreen()\n movies.isError() -> ErrorScreen { movies.retry() }\n else -> MoviesGrid(movies = movies, onClick = viewModel::onMovieClick)\n}Additionally, LazyPagingItems offers a convenient retry() method, allowing you to retry the last failed data loading attempt with ease.
Now lets add filters to our movie list to see paginated list of movies filtered by genre.
\n![\"Filters](\"/static/paging_demo_filters-ff80a7606fb30961e2bda56bd6ed9985.gif\")
We start with requesting genres from the server and saving them in our database. We will keep genre data and selection in the database so that we can observe it from our domain and UI layers.
\ninterface MoviesApi {\n @GET(\"/3/genre/movie/list?language=en-US\")\n suspend fun genres(@Query(\"api_key\") api_key: String): Response<GenresNetworkResponse>\n}\n\n@Dao\ninterface GenresDao {\n\n @Insert(onConflict = OnConflictStrategy.REPLACE)\n suspend fun putGenres(genres: List<GenreDbEntity>)\n\n @Upsert(entity = GenreDbEntity::class)\n suspend fun updateGenres(genres: List<GenreDbUpdateEntity>)\n\n @Query(\"SELECT * FROM genres\")\n suspend fun getGenres(): List<GenreDbEntity>\n\n @Query(\"SELECT * FROM genres\")\n fun observeGenres(): Flow<List<GenreDbEntity>>\n}Now we can add with_genres parameter to our network request:
@GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n @Query(\"api_key\") api_key: String,\n ): Response<MoviesNetworkResponse>Remote mediator will get selected genres from the repository
\n ...\n val genres = repository.getGenreSelection()\n val movies = moviesDataSource.getMovies(page, genres)\n ...Every time user selects a new filter we stave it in the database and restart pagination by clearing movies in the database and resetting the page to 1.
\nclass MoviesScreenViewModel {\n\n fun onGenreClick(genreItem: GenreItem) = viewModelScope.launch {\n genresProvider.setSelection(genreItem.id, !genreItem.isSelected)\n repository.clearMovies()\n }\n \n}class MoviesRepositoryImpl {\n\n override suspend fun clearMovies() = withContext(Dispatchers.IO) {\n database.withTransaction {\n database.movies().clear()\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, 1))\n }\n }\n\n}
Source code: https://github.com/auto1-oss/android-compose-paging\n
Official documentation: https://developer.android.com/topic/libraries/architecture/paging/v3-overview
Sense of Security
\nTreat the injured first
\nTrust your tools
\nCommunication via Radio
\nWhat I present today is how to use police techniques and their mindset to detect and solve bugs in software, so that you can be the best software developer that society can provide.
\nFor some years, I worked as a police officer in Brazil and I can say that it is a challenging career. The first step was to pass a national test, where knowledge of various laws was required. Criminal laws, traffic laws, and even the laws of physics.
\nThe second step was to pass a battery of medical exams. Almost perfect health was necessary to be able to practice the profession, in addition to having most of the parts of the human body.
\nThe third step was the psychological examination. It was necessary to have a clear mind and not have impulses different from those expected from a person who must protect other people.
\nAs a fourth part, there was also the fitness test, a test in which career candidates had to prove they had sufficient strength and speed to carry out daily police tasks. Running, pull-ups, sit-ups and long jump are examples of what was required.
\nThe last stage was the Police Academy, where for three months those from now on known as students would learn and prove the learning of several new skills: first aid, use of firearms (pistols, carbines, submachine guns, etc.), non-lethal weapons, martial arts, public policy, and many other disciplines.
\nAll this so that society receives the best police officer that its people can provide. So, what about using some of the following principles and techniques to become the best software developer that society can provide?
\nWithout needing to be a security expert, you already know that it is impossible to have a completely safe city. Even if we reached the point of having perfect police officers, which in itself is impossible, we would not have a police officer for every citizen. In general, the recommendation is 300 police officers for every 100 thousand inhabitants.
\nAware of this limitation, that it is not possible to guarantee complete security for all citizens, police agencies work with the concept of a sense of security. Through strategic actions to combat crime, citizens' feeling that they are safe is increased. This mental condition gives them peace of mind when working, studying, and carrying out their daily tasks.
\nThis sense of security also affects criminals. The common visibility of police officers carrying out their duties in strategic places, as well as effective actions to combat crime, make criminals stay away from that place.
\nNotice that ending 100% of crime is impossible, what we do well is limit its actions. From here we can extract our first lesson when dealing with software: perfect code, totally bug-free, is impossible.
\nDespite this conclusion, the fact that it is impossible does not mean that we should simply ignore the problem. On the contrary, we can, like the police, create strategic actions to combat Bugs. To the point that Bugs themselves will run away from our code.
\nTest Driven Development - TDD is a strategic action that provides a feeling of security in your code. Before the crime occurs, you position police officers in a specific region. Before the Bug occurs, you position the test in a specific place in your code.
\nRobert C. Martin, in his book \"Clean Code: A HandBook of Agile Software Craftsmanship\", teaches us that writing unit tests before writing production code is just the tip of the iceberg in defining TDD. There must be a continuous cycle of improvement, in which there is always a test covering the code you are developing and this security that the code is protected allows you to play with it, and work with it. After all, whatever problem may happen, there is a police-test there to protect you.
\nOne of the functions of Highway Police Officers is to provide first aid in the event of traffic accidents. There are often serious injuries that must be treated as urgently as possible, as even 1 minute can make the difference between life and death.
\nThe first step when arriving at a traffic accident site is to mark the location appropriately to prevent other accidents. We must use lighting devices and other appliances to make it clear to other vehicles that there is a danger there.
\nThe police officer must then check the condition of all the injured to prioritize them according to the severity of each situation.
\nOnce the treatment of each injured person has been completed and, if necessary, they have been taken to the hospital, it is time to survey the accident site. At this stage, the police officer begins to collect all the information relating to the accident to have sufficient information so that a decision can be made on the cause of the accident and, when applicable, any culprits.
\nThis is the mental model that you and I must adopt when we are faced with a Bug. Although several procedures have been adopted to prevent it from occurring, it is a fatality that can happen. It's like when the highway was well designed, the vehicles were up to date with maintenance, but even so, due to carelessness or misunderstanding on the part of the driver, an accident happens.
\nAs in the case of traffic accidents, the first step to take is to secure the environment so that the Bug does not cause other problems. If possible, we can revert the application version to one in which the Bug did not appear. We can use a feature flag to temporarily disable a feature. Regardless of the method used to stop the bleeding, the important thing is that we must focus on treating the injured first rather than looking for culprits.
\nWe have to agree with David Thomas and Andrew Hunt, in their work \"The Pragmatic Programmer: your journey to mastery\" that facing Bugs is a sensitive subject for most developers. Instead of treating the issue as a problem to be solved, some approach it with denial, pointing fingers, lame excuses, or simply apathy. It's exactly the opposite of what is expected!
\nInstead of looking for the culprit in a traffic accident, we must first assist the victims and stop the bleeding. In software, instead of finding out who inserted the Bug, we must solve it.
\nWhen treating an injured person in an emergency, one approach is to use the Glasgow Scale to determine the patient's consciousness. On this scale, one of the components is the verbal response.
\nThe injured person is asked simple questions, and by analyzing the responses, we can assess the level of consciousness and consequently the level of urgency of their care.
\nYou and I must carefully read every error message that is displayed during the Bug, it may contain vital information so that we can resolve the problem.
\nYou may be thinking, but what if there is no error message? If we are just faced with an incorrect result? Well, this also happens in traffic accidents, it is not uncommon for there to be an unconscious victim who cannot answer questions.
\nWhen we discussed the sense of security, we talked about the importance of tests in our application. Go and build a test that can reproduce that Bug, so you can extract the information you need to solve it.
\nThink of these tests like the motor tests that highway patrol officers perform on unconscious victims to check their level of consciousness: do the eyes open in response to pain? Is there a pupillary response to light stimulation? Mentally translate this into software language: Does the Bug repeat itself when I enter this and that parameter? And if I insert this new argument, does it happen too?
\nTo decide on a Bug, data is needed. If you don't receive this data, you must extract it.
\nPolice work is very complex and your day-to-day job involves the fight for life and protection of the most important assets for society. There would be no point in having a modern and efficient selection and training process for the development of police officers if they were not given the appropriate tools for their work.
\nAnd when I say appropriate, I don't mean they are minimally capable of performing the task, as the result delivered is directly related to the quality and safety of the tools. For an excellent result, not only the police officers but their tools must also be excellent.
\nIn the police agency I worked for, one of the work tools used was an Austrian pistol known worldwide for its safety and efficiency. The Glock 17 pistol was developed to meet the strict criteria of the Austrian army who wanted to replace the model used during the Second World War.
\nIn addition to requirements on durability, efficiency, and reliability, safety against accidental firing was one of the pivots of the new model.
\nBut how did a knife factory manage to deliver the pistol model that would prove to be the safest? Well, it can be said that, in addition to intense testing, the secret to reliability was simplicity and maintainability. The person responsible for the project designed the equipment with as few parts as possible, minimizing complexity. Today the Glock pistol is made with an average of 35 parts, which is a much smaller quantity than other pistols on the market.
\nWith a simpler design, so-called first-level maintenance is much easier. Equipment users can easily perform maintenance procedures, making the equipment's efficiency last for decades.
\nOf course, you must already remember the principle with the acronym KISS, which stands for Keep it Simple (omitting the last word because I don't want to offend the reader). The secret of that renowned pistol model was not in the used materials, but in being simple. The secret of maintaining your software in good shape is to keep it simple, having a specific component for each task. When a problem arises, you will know exactly which component to look for, as each component has a responsibility, this is the principle of Single Responsibility.
\nBecause of all this, when you are at a shooting range practicing your aim, the police officer knows that if he is not hitting the target, most likely, the error is in his procedure and not in the equipment.
\nAllan Kelly, in one of his contributions to the Book \"97 Things Every Programmer Should Know\", corroborates this idea that we should trust our tools when dealing with Bugs. Considering our tools are widely used, mature, and in use by multiple companies, we have little reason to doubt their quality.
\nOf course, a prototype, version 0.1, is much more fragile than a library that has been on the market for several years. We are also not saying that it is impossible to have bugs in compilers.
\nBut considering how rare compiler bugs are, what we are saying is that in the search for a Bug solution, we should trust our tools and look at our piece of code first.
\nImagine it as an efficient distribution of efforts. The library or compiler code has already been reviewed and tested for many years by countless people. As for that little piece of code of yours, only God knows who besides you has looked at it on your team.
\nOn one of those days, I managed to solve an unpleasant bug that occurred in one of our services. It was a non-deterministic error in our test suite, that is, the Bug only pushed out its head when no one was looking.
\nThis type of non-deterministic error generally involves three types of origin: multithread, time sensitivity, and dissynchrony between construction and cleaning of test data.
\nAnalyzing the body of the failed test, after several hours, of course, I began to strongly distrust the way we created timestamps with a native Java library. After all, the test seemed perfect to me, if there was an error it must have been somewhere else.
\nBut, aware of this lesson that we should trust our tools, especially since the component in question had been in Java for 9 years, I ignored this distrust. This saved me a lot of time and embarrassment, as I'm going to share that the error was elsewhere.
\nOur test suite ran all tests in a single thread, so it wouldn't be a concurrency issue. But it also ran each test in a deterministic but unpredictable order. Therefore, the source of the intermittent problem must be in some specific order in which the tests were performed.
\nAfter several hours of analyzing the logs, I was able to identify two tests that failed when run in sequence. It turned out that, although our tests performed a test data cleaning routine after each execution, a simple typo had silently overwritten the cleaning method in one of the tests, which generated data inconsistency if such tests were executed in a specific order.
\nI remember that one of the first systems I developed when I worked in the police was one to replace the sending of statistics through radio communication. Many years ago, each police station was supposed to send daily work statistics via radio. Data such as number of people and vehicles inspected, car accidents, etc.
\nThis took a lot of time as the audio quality of radio communications was not always ideal. The interlocutors spent a lot of time transmitting and confirming each piece of information. All this using Q-Code
\nThe Q-Code is a standardized collection of three-letter codes used in radio communication. The use of Q-code even in voice transmissions makes communication safer and more efficient. Each set of three letters, always starting with the letter Q, has a specific meaning that is known to the interlocutors.
\nI could see that two of those codes were widely used among police officers in radio communication: QAP and QSL.
\nQAP was used when you wanted to transmit a message. Let's say someone wanted to talk to me, so he said on the radio: \"Bruno, QAP?\". This could be translated as \"Bruno, are you ready to hear a message?\"
\nMy answer could be QAP or for example QRM. If I responded with QAP, I was signaling that yes, I was ready to receive the message. If you responded with QRM, I would be informing you that at that moment the communication is being interfered with.
\nDuring communication, the second most used Q-Code appears, QSL. At the end of each message, the sender ends it with the QSL. This means that he is asking the receiver to confirm whether they received the message. It would be something like: \"There is a suspicious vehicle heading towards your team, QSL?\" If I had understood the entire message, I should respond with a simple \"QSL\". In the absence of a response, the sender repeats the message until receiving confirmation from the receiver.
\nEach message is concluded with a QSL, and its reception is confirmed with a QSL. Does this remind you of any transmission control protocol out here, used on the internet?
\nThe main thing here is that a large amount of knowledge is condensed into three letters, in a language that provides interlocutors with efficiency and security. Eric Evans, the author of \"Domain-Driven Design: Tackling Complexity in the Heart of Software\", would agree with me that there is a principle here that should also be adopted in software development.
\nA language specific to the domain, which considers possible interference from the environment, a ubiquitous language that is known by all interlocutors, can be the difference between life and death, between success and damnation of a project.
\nWhen a project does not have its own language for that domain, this creates the need for numerous translations between interlocutors. Developers have to translate for domain experts. These, in turn, have to translate their demands to developers. Even among developers, there may be a need for translations.
\nAll these translations make concepts confusing and the consequences of a lack of mutual understanding can be disastrous.
\nThe importance of this in trying to arrest a Bug goes much further than compilation errors or exceptions. It goes through the swampy characteristic that the software may be logically working almost perfectly, but it is not delivering what was expected by the stakeholder.
\nLet's look at the following pieces of code:
\nif (order.status != completed) {\n // (...)\n}if (order.isFullyRefunded()) {\n // (...)\n}Here we have a small example of two ways of expressing the same situation. In the second piece of code, a common language between developers and domain experts was used. Here, possible logical errors can be easily noticed, without the need to keep in mind what the possible states of the Order are and what the expected state is for that case.
\nSee the following code:
\nPayment newPayment = payment.add(10);What do you expect from this code? Does it add ten dollars to the payment? Ten transactions, inbound or outbound? Will the method create a new payment object or will it just change the value of the payment variable and create a new reference called newPayment?
\nA lesson also taken from the book \"Clean Code: A HandBook of Agile Software Craftsmanship\" shows that clear communication must also take place between software developers. Functions must communicate exactly what they are doing.
\nIf you have to look at the function's implementation to know what it's actually doing, that's a big sign that you need to rename it.
\nI'm very grateful that you've made it this far, you've certainly learned a lot about police activity and how some of its principles can be applied to combat and remove Bugs from your software.
\nBy applying this knowledge you will be able to work more confidently in your code, giving you peace in your daily life and also allowing you to experiment and discover new things.
\nYour aim will be improved and you will get straight to the point, avoiding damage instead of first looking for the culprits. You become bolder and go further because you know you can trust your tools. This leaves you more time to refine your code and seek excellence.
\nFinally, not only will your code be safe and efficient, but its result will also be in line with what your customers expect because through effective communication you can accurately deliver what was demanded from you.
\nThe result of all this is that Bug is in jail, lives are no longer in danger, and you can go home at the end of your shift to enjoy the rest of the day with your family which is looking forward to welcoming their hero back.
","fields":{"slug":"/bug-police-officer/","tags":["debugging","testing","TDD","DDD"]}}}]},"authorArticles":null},"pageContext":{"slug":"/IntelliJ IDEA plugin development - focus on UI based plugins/","tags":["auto1","engineering","android","mobile","intellij","idea"],"category":"Engineering","author":"Vadim Shchenev","date":"2021-07-10"}}