![\"Paging](\"/static/paging_demo-a325402259bf9864dced311c9eafa7c8.gif\")
Paginated APIs, often encountered when dealing with servers and databases, break down large datasets into manageable chunks or pages. This methodology not only optimizes resource usage but also enhances the overall user experience by delivering content progressively. However, integrating and managing paginated data in a mobile app can be a challenge, from maintaining loading states to ensuring a seamless transition between pages and supporting content filtering.
\nIn this guide I will show you how to work with paginated APIs in Android with Jetpack Compose and Paging 3 libraries.
\n
\n![\"Paging](\"https://developer.android.com/static/topic/libraries/architecture/images/paging3-library-architecture.svg\")
The key components of the Paging 3 architecture include
\nWe start with the data layer and define a Retrofit API for fetching movies from network service.
\ninterface MoviesApi {\n @GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"api_key\") api_key: String,\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n ): Response<MoviesNetworkResponse>\n}Next, we will leverage the Room library, which is compatible with Paging 3, to create a local paging source.
\n@Dao\ninterface MoviesDao {\n @Query(\"SELECT * FROM movies ORDER BY id ASC\")\n fun getMovies(): DataSource.Factory<Int, MovieDbEntity>\n}It's important to highlight that we utilize DataSource.Factory as the return type instead of PagingSource. This choice is made specifically because DataSource.Factory offers the map() method, enabling us to map MovieDbEntity instances into domain layer models.
Now we need to put together local and remote data sources in Remote mediator.
\ngetRemoteKey().nextPage) is stored in the database after each successful network request.val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)For bidirectional pagination support, provide the preceding page number for the PREPEND load type. The REFRESH load type is employed when content is refreshed, initiating loading from the initial page.
\nCall our network data source method to fetch a page of data from the server.
\nval movies = moviesDataSource.discover(page)The RemoteMediator code will appear as follows:
\nclass MoviesRemoteMediator(\n private val repository: MoviesRepository,\n private val moviesDataSource: NetworkMoviesDataSource,\n) : RemoteMediator<Int, Movie>() {\n\n override suspend fun load(loadType: LoadType, state: PagingState<Int, Movie>): MediatorResult {\n\n return try {\n\n val page = when (loadType) {\n LoadType.REFRESH -> 1\n LoadType.PREPEND -> null\n LoadType.APPEND -> repository.getRemoteKey()?.nextPage\n } ?: return Success(endOfPaginationReached = true)\n\n val movies = moviesDataSource.getMovies(page)\n\n val nextPage = if (movies.isNotEmpty()) page + 1 else null\n\n repository.insertMovies(movies, nextPage, loadType == LoadType.REFRESH)\n\n Success(endOfPaginationReached = movies.isEmpty())\n\n } catch (e: Exception) {\n MediatorResult.Error(e)\n }\n }\n}RemoteMediator uses the repository to store movies list and next page number in the database.\nRepository also takes care of clearing the database table when data is refreshed:
\nclass MoviesRepositoryImpl(\n private val database: MoviesDatabase\n): MoviesRepository {\n\n override fun getMovies() = database.movies().getMovies().map { it.toDomain() }\n\n override suspend fun insertMovies(movies: List<Movie>, nextPage: Int?, clear: Boolean){\n\n database.withTransaction {\n if (clear) {\n database.movies().clear()\n }\n\n database.movies().putMovies(movies.map { it.toDbEntity() })\n\n if (nextPage != null) {\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, nextPage))\n }\n }\n }\n}The Pager object serves as a bridge between remote and local data sources. It is managing the page loading, prefetching, and handling the transitions between various data sources to provide a seamless and responsive flow of paginated content within your application. You can configure the page loading behavior with the following parameters:
\nclass MoviesScreenViewModel(\n moviesSource: MoviesRemoteMediator,\n repository: MoviesRepository,\n private val movieDetails: MovieDetailsApi,\n) : ViewModel() {\n val moviesFlow = Pager(\n config = PagingConfig(\n pageSize = PAGE_SIZE,\n prefetchDistance = PREFETCH_DISTANCE,\n initialLoadSize = INITIAL_LOAD,\n ),\n remoteMediator = moviesSource,\n pagingSourceFactory = repository.getMovies().asPagingSourceFactory()\n ).flow.cachedIn(viewModelScope)\n}Now, let's explore how to consume the paginated data in your Compose UI.
\nUse the collectAsLazyPagingItems extension function to collect the PagingData and observe changes.
val movies = viewModel.moviesFlow.collectAsLazyPagingItems()Use the LazyColumn or LazyVerticalGrid to efficiently display paginated data in your Compose UI.
LazyVerticalGrid(\n modifier = Modifier\n .fillMaxSize()\n .padding(16.dp),\n columns = GridCells.Fixed(3),\n horizontalArrangement = Arrangement.spacedBy(16.dp),\n verticalArrangement = Arrangement.spacedBy(16.dp),\n ) {\n\n items(\n count = movies.itemCount,\n key = movies.itemKey { it.id },\n ) { index ->\n\n movies[index]?.let { movie ->\n MovieCardSmall(\n movie = movie,\n onClick = { onClick(movie.id) },\n )\n }\n }\n }\n\nNote the usage of the key property for items. The key property uniquely identifies each item, allowing Compose to efficiently update and recycle composables as data changes.
\n
LazyPagingItems offers insights into the ongoing data loading process, allowing you to effectively manage errors during pagination. By examining the loadState.append property, you can determine the loading state when appending new items and present a user-friendly indication of the current status.
\nFor instance, the following code snippet demonstrates how to react to different loading states during item appending:
\nwhen (movies.loadState.append) {\n is LoadState.Loading -> { ProgressFooter() }\n is LoadState.Error -> { ErrorFooter(onRetryClick = { movies.retry() } }\n}Similarly, you can assess the loading status when the list data is refreshed by checking the loadState.refresh property. Extension functions isError() and isLoading() provide convenient checks for error and loading states, respectively.
\nfun LazyPagingItems<Movie>.isError(): Boolean = \n loadState.refresh is LoadState.Error && itemSnapshotList.isEmpty()\n\nfun LazyPagingItems<Movie>.isLoading(): Boolean = \n loadState.refresh is LoadState.Loading && itemSnapshotList.isEmpty()Incorporate these checks into your UI logic to display appropriate screens based on the loading or error states. The following code snippet demonstrates how to handle loading and error states along with providing a retry option:
\nwhen {\n movies.isLoading() -> LoadingScreen()\n movies.isError() -> ErrorScreen { movies.retry() }\n else -> MoviesGrid(movies = movies, onClick = viewModel::onMovieClick)\n}Additionally, LazyPagingItems offers a convenient retry() method, allowing you to retry the last failed data loading attempt with ease.
Now lets add filters to our movie list to see paginated list of movies filtered by genre.
\n![\"Filters](\"/static/paging_demo_filters-ff80a7606fb30961e2bda56bd6ed9985.gif\")
We start with requesting genres from the server and saving them in our database. We will keep genre data and selection in the database so that we can observe it from our domain and UI layers.
\ninterface MoviesApi {\n @GET(\"/3/genre/movie/list?language=en-US\")\n suspend fun genres(@Query(\"api_key\") api_key: String): Response<GenresNetworkResponse>\n}\n\n@Dao\ninterface GenresDao {\n\n @Insert(onConflict = OnConflictStrategy.REPLACE)\n suspend fun putGenres(genres: List<GenreDbEntity>)\n\n @Upsert(entity = GenreDbEntity::class)\n suspend fun updateGenres(genres: List<GenreDbUpdateEntity>)\n\n @Query(\"SELECT * FROM genres\")\n suspend fun getGenres(): List<GenreDbEntity>\n\n @Query(\"SELECT * FROM genres\")\n fun observeGenres(): Flow<List<GenreDbEntity>>\n}Now we can add with_genres parameter to our network request:
@GET(\"/3/discover/movie?language=en-US&sort_by=popularity.desc\")\n suspend fun discover(\n @Query(\"page\") page: Int,\n @Query(\"with_genres\") genres: String,\n @Query(\"api_key\") api_key: String,\n ): Response<MoviesNetworkResponse>Remote mediator will get selected genres from the repository
\n ...\n val genres = repository.getGenreSelection()\n val movies = moviesDataSource.getMovies(page, genres)\n ...Every time user selects a new filter we stave it in the database and restart pagination by clearing movies in the database and resetting the page to 1.
\nclass MoviesScreenViewModel {\n\n fun onGenreClick(genreItem: GenreItem) = viewModelScope.launch {\n genresProvider.setSelection(genreItem.id, !genreItem.isSelected)\n repository.clearMovies()\n }\n \n}class MoviesRepositoryImpl {\n\n override suspend fun clearMovies() = withContext(Dispatchers.IO) {\n database.withTransaction {\n database.movies().clear()\n database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, 1))\n }\n }\n\n}
Source code: https://github.com/auto1-oss/android-compose-paging\n
Official documentation: https://developer.android.com/topic/libraries/architecture/paging/v3-overview
BACKGROUND
\nThis post is in addition to the previous article about plugin development but with a focus on UI based plugin implementations. Sometimes it can be really useful to have an UI and make it possible to interact with it by doing some checkboxes selections or maybe to have some input areas for text. Let's investigate how to develop a simple plugin with input text area as input entry point and some new generated classes, based on input text as a result. As a source code example we will have a look at source code based on my pet project that also was used as a development tool in AUTO1 to generate non GraphQl based POJO classes.
\nUSER STORY
\nAs an Android developer I need to interact with backend data. It can be usually presented as JSON data or if we can speak about new and modern practices it also can be presented as a GraphQl scheme. GraphQl is a really powerful framework with its own classes generation framework. But let's go back one step and imagine that we have to do some data presentation classes updated for the project without GraphQl support. To do it, we need to find the target class, compare its fields with new JSON fields and update it. It can be easy if we have only one field changed, but for multiple inner fields and classes with its own inner fields it can be too complicated. We can find some online services to convert JSON to POJO and copy/paste output files, but let's try to integrate this technology to our IDE! And our job will be only to copy/paste new JSON to some window input field! Like this one:
\nTo make it possible, let's think about our problem and try to do small decomposition into logical steps. Lets formulate our goals by imaging simple user stories:
\nAs a user I want to input JSON text into the input field.
\nAs a user I want to transform JSON text into POJO files.
\nAs a user I want to see the result notification.
\nWe will skip JSON to POJO transformation for this article, but we will learn how to develop a plugin with required user interface. Let's think about option number two as about some library that we already have and can use as a part of our small project.
\nTECHNOLOGY STACK
\nBefore we will start let's have a look on basis plugin implementation. There are 2 base options to develop new plugin:
\n\nWe will speak about gradle implementation. Gradle is a really powerful framework and it can be really helpful if we want to simplify our building, releasing and testing. The project structure is a common structure for gradle based projects:
\nmy_gradle_plugin\n\n├── build.gradle\n\n├── gradle\n\n│ └── wrapper\n\n│ ├── gradle-wrapper.jar\n\n│ └── gradle-wrapper.properties\n\n├── gradlew\n\n├── gradlew.bat\n\n├── settings.gradle\n\n└── src\n\n├── main\n\n│ ├── java\n\n│ └── resources\n\n│ └── META-INF\n\n│ └── plugin.xml\n\n└── test\n\n├── java\n\n└── resources\n\n Most important topic for us will be the plugin.xml file. We already discussed some details in the previous article and we can be focused on some UI related topics.
\nDuring the plugin development we can use Java/Kotlin and all utility tools that we want like jUnit testing and dependency injection frameworks. In our current example Koin and JUnit can be found. So, now we have and know about project structure, some utility tools, language to write the code and now we are ready to think how it can be possible to create a user interface and connect it with our IDE. Let's remember about our major goals:
\nGOAL 1 - As a user I want to input JSON text into the input field.\n
\nGOAL 2 - As a user I want to see the result notification.\n
and see how we can implement it.
\nGOAL 1
\nFirst of all, let's learn how to connect our plugin with the IDE user interface. As we can see the IDE has a lot of action groups to do something (create new files for example). According to our idea to have some logic that can help us to convert JSON to POJO it would be great to have a new item in the new file list. It will be something like an action “Create new POJO file”.
\nTo connect our logic with the IDE by following this path we have to remember about the plugin.xml file that was discussed early. We can use this configuration file to describe our plugin and how it can be used.
\nLet's find the actions section and define new action to show our input field dialog view.
\n<actions>\n\n<action id=\"GeneratePOJOAction\" class=\"GeneratePOJOAction\"\n\nicon=\"/icons/pojo.png\"\n\ntext=\"Generate POJO from JSON\"\n\ndescription=\"GeneratePOJOAction\">\n\n<add-to-group group-id=\"NewGroup\" anchor=\"last\"/>\n\n</action>\n\n</actions>As we can see we can use some properties to describe how users will be available to interact with our plugin. We can define a custom icon for our action, define the position inside of the items list (“NewGroup” anchor “last”) and we can define the class to execute when action will be invoked by the user.
\nCustom action class should extend the abstract AnAction class with the implementation of actionPerformed() method.
\nimport com.intellij.openapi.actionSystem.AnAction\n\nimport com.intellij.openapi.actionSystem.AnActionEvent\n\nclass GeneratePOJOAction : AnAction() {\n\noverride fun actionPerformed(e: AnActionEvent) { \n\n//TODO: Add your UI implementation here \n\n} \n}Inside of actionPerformed() method implementation we can add the logic to show a dialog window with the input field for JSON text. To show dialog view we can easily use DialogBuilder from plugin development SDK and Java swing to create a user interface.
\nMore information about actions can be found on the official documentation page.
\nGOAL 2
\nLet's imagine now that we already implemented logic to show the input text dialog with some selectable options to transform this text. We also added some logic of transformation - that can be some library to work with it or maybe our own implementation. And as the result of that transformation some new POJO files were generated added into our project (We can interact directly with the project or we can use project representation, provided by the plugin development SDK).
\nBut what about the next steps? We also need to notify the user about the result: it can be a success result notification, or maybe some error case notification. To implement it we can use default tools, provided by plugin development SDK. We can show some information to the user by using popup, notifications or messages.
\nLet's imagine the following case: we have a text input field and we have some logic to convert/transform the text from this input field. If we can speak about JSON to POJO transformation, we have to understand that any random text cannot be converted to POJO correctly. Target text should satisfy some conditions and also should have the correct JSON structure and it means that we have to not only show some notification if text structure is not valid, but we have to block UI and break our transformation flow.
\nMessages can really help us with it! According to the official documentation:
\nThe Messages class provides a way to show simple message boxes, input dialogs (modal dialogs with a text field), and chooser dialogs (modal dialogs with a combo box). The function of different methods of the class should be clear from their names. When running on macOS, the message boxes shown by the Messages class use the native UI.
\nLets see the following code:
\ncompanion object {\n\nprivate const val BUTTON_TITLE = \"OK\"\n\nprivate const val DEFAULT_OPTION_INDEX = -1\n\nprivate val DEFAULT_ICON = null\n\n}\n\nfun onPluginExceptionHandled(exception: RoboPluginException) {\n\nMessages.showDialog(\n\nexception.message, exception.title,\n\narrayOf(BUTTON_TITLE),\n\nDEFAULT_OPTION_INDEX,\n\nDEFAULT_ICON\n\n)\n\n}As we can see we can use the show dialog method, provided by SDK. We can define some dialog parameters, like title, message, button and of course we can define our custom plugin icon. If the image is not defined, standart Java icon or IDE icon would be shown. More information about images support can be found here. Also as it was already discussed here we can add support for internationalization instead of hardcoding only english string values.
\nAnd as the result we can see this amazing small UI blocking window with our message text:
\nIt looks now that we found a solution to cover the error cases. Now we can prevent users from wrong input and show notification about it. Let's think about the success use case. Let's imagine that the input text was correct and we processed our JSON to POJO transformation as expected. Maybe some new files were generated, maybe it was only one new file. It will be a really good user experience if we can find a possibility to inform our users about it. Maybe one of the possible ways can be to show notification messages. It can be a small popup view with some text, like this one:
\nand maybe we also can print some information to the event log:
\n12:00 Response.java was updated.\n \n12:00 POJO generation: SuccessIt looks now that we are ready to totally satisfy our user story requirements! Let's do the final steps and implement the following code:
\ncompanion object{\n\nprivate const val MESSAGE_SUCCESS = \"POJO generation: Success\"\n\nprivate const val GROUP_ID = \"RoboPOJO Generator\"\n\n}\n\nfun showSuccessMessage() {\n\nshowMessage(\n\nproject = ProjectManager.getInstance().openProjects.first(),\n\nmessage = MESSAGE_SUCCESS,\n\ntype = NotificationType.INFORMATION,\n\n)\n\n}\n\nprivate fun showMessage(\n\nmessage: String, type: NotificationType, project: Project\n\n) = NotificationGroupManager.getInstance()\n\n.getNotificationGroup(GROUP_ID)\n\n.createNotification(message, type)\n\n.notify(project)\n\n As we can see we can use the create notification method, provided by SDK. Unlike dialog related logic, we also have to define the project that we want to use to show the notification. As an example let's get only the first project (most popular use case), but we have to understand that users can open not only one project in the IDE and we have to take care about it too. So, as we can see it is also possible to define message text. Also we can define the message type (it can be INFORMATION, WARNING or ERROR) and we have to take care about the notification group that was defined inside of our plugin.xml file:
\n<extensions defaultExtensionNs=\"com.intellij\">\n\n<notificationGroup id=\"RoboPOJO Generator\"\n\ndisplayType=\"BALLOON\"\n\nkey=\"com.robohorse.robopojogenerator\"/>\n\n</extensions>FEW WORDS AT THE END
\nAs we can see it was not really a big deal to develop some small and hopefully useful tools for daily use or to help your teammates. The major idea is just to understand that integration with IDE can be really simple and when we will finish with it, we can be focused on the functionality of our tool. Perhaps in the future when we will have a stable version with some really cool features we can share our tool not only with teammates that we are working with directly, but maybe it also can be helpful for some developers from some random parts of the world. And maybe together we can simplify our daily job!
","fields":{"slug":"/IntelliJ IDEA plugin development - focus on UI based plugins/","tags":["auto1","engineering","android","mobile","intellij","idea"]}}}]}},"pageContext":{"slug":"/tags/mobile","tag":"mobile","categories":["Architecture","Coding","DevOps","Engineering","ProjectManagement","QA","Social","TechRadar"]}}