The PostgreSQL team announced recently a new release of the most advanced\nopen source relational database - PostgreSQL 12. As usual it comes with an impressive list of improvements\n(generated columns ♥️), one of them being long awaited by dozens of developers: improved Common Table Expressions.
\nI should first explain what are the Common Table Expressions for those who are unfamiliar with them:\nthe CTE’s, often called “WITH queries”, are SQL constructs giving a possibility of creating temporal data views for a sake of a query execution.
Essentially CTE is an additional query which results can be referenced in the subsequent CTE’s or the main query before which\nit is being placed. It should be clear enough with an example - here’s a sample query with two CTE’s taken from Postgres docs:
\nWITH regional_sales AS (\n SELECT region, SUM(amount) AS total_sales\n FROM orders\n GROUP BY region\n), top_regions AS (\n SELECT region\n FROM regional_sales\n WHERE total_sales > (SELECT SUM(total_sales)/10 FROM regional_sales)\n)\nSELECT region,\n product,\n SUM(quantity) AS product_units,\n SUM(amount) AS product_sales\nFROM orders\nWHERE region IN (SELECT region FROM top_regions)\nGROUP BY region, product;Now we know what they are, but what purpose can they serve us? Well - we could parry here and say: for the same purpose as ordinary database views serve.\nThat’s of course a dramatic simplification - CTE’s are much more powerful and should not be treated as a simple database views.\nNevertheless I would like to keep the collation for the sake of this article.
\nDatabase views are absolutely optional - one can simply substitute them with a subquery and achieve identical results.\nIndeed that’s what modern database engines do these days - once a database view is being used they inline it’s query as a subquery.\nWhy to bother then? We are able to deal completely without database views and even if we did use one the database engine would get rid of it anyway.
\nWhat benefits do views give us then? And why the heck are they inlined?
\nThe most appropriate explanation here is that database views help us achieve better readability.\nThey offer an elegant way of abstracting some parts of a database into a meaningful object, often matching closely with the domain.
\nInstead of creating giant and ugly looking queries it is possible to extract some of its parts into an appealing view which is easier to browse and select data from. \"Divide and conquer\" rule in it’s true form.
\nStill, we didn’t answer the fact that the underlying view’s query is most of the times inlined while it is being referenced. The reason is performance of course. Smart guys found out that lazy evaluation helps the optimizer a lot - by delaying the execution we could take advantage of a context of the actual query.
\nThis in turn allows many clever optimization techniques, like: pushing down predicates (WHERE filters), eliminating unnecessary JOINS, accessing only subset of columns etc.\nIn other words - a database is smart enough to do as little work as possible when evaluating database views, in the context of the issued query.
\nPersonally I love this pattern: aggregating all the data into views and letting a database to optimize my queries - these folks are really good in it and my queries are dead simple too.
\nI have mentioned the CTE’s at the beginning, saying they are able to create temporal data views. I still conform to the comparison with database views - they both are in many cases similar.\nThe main difference is that CTE results are temporary and are reachable only in the context of a query which CTE is being part of.
\nIt may make sense to use a CTE in a place where a regular database view is not justified (e.g. it makes sense only in the context of a query and not in the whole domain), expecting similar behavior.
\nBesides many remarkable advantages there’s at least one disadvantage which disqualifies CTE's for most use cases - before PostgreSQL 12 it was implemented as an optimization fence. What does it mean?
\nEasy to imagine an example with a generic data view aggregating lots of data. If the view is then used to select just few records it could mean a tremendous waste of computation, if the aggregation is executed immediately.\nInstead the aggregation should be executed only on a small subset of data, which can be deduced from the outer query.
\nUnfortunately such counter intuitive behavior was true for CTE’s for a long time - their results were materialized only to be accessible for the rest of the query afterwards.
\nNot to blame anybody - the creators had quite good reasons to implement such behavior (i.e. guarantee of exactly one evaluation, possibility of a recursive CTE’s and more) but this still feels like focusing on corner cases instead of optimizing the happy path.\nThat’s why the community insisted for a long time for changing the status quo by giving the possibility of disabling the fence and unlocking the full potential of CTE’s.
\nThe SQL gods listen to their prayers and here it is - PostgreSQL 12 with updated Common Table Expressions.\nBy default, when few constraints are met, the queries will be inlined allowing joint optimizations.
\nIt is still possible to force the old behavior - by defining the CTE AS MATERIALIZED the engine would execute it immediately.\nIt is also possible to hint the optimizer that we definitely want the CTE to be inlined, e.g. when the CTE is being referenced twice the engine won't inline it by default.
This is truly a game changer for many developers who care about their queries’ readability, allowing them to substitute not very well liked subqueries with elegant “WITH queries”.
Don’t get me wrong - not every subquery should be immediately replaced, they still have their strengths and in some cases they should be chosen over CTE’s. It is just convenient to have two distinct tools in a toolbox, isn’t it?
","fields":{"slug":"/postgres12-a-precious-release/","tags":["postgres","postgres12","release","sql","cte"]}}},{"node":{"id":"cd170267-3e38-5f47-8461-d6f6583c5bed","frontmatter":{"category":"Coding","title":"Our love story with Deadlocks (PostgreSQL Edition)","date":"2019-03-15","summary":"How to get rid of deadlocks","thumbnail":{"relativePath":"pages/event-store-deadlock/thumbnail.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAVCAYAAABG1c6oAAAACXBIWXMAAAsSAAALEgHS3X78AAAFFklEQVQ4y5WUeUzTZxjHi+AWnTuYc1mAKaiAIgoI4tR5henqgSBEoOOmCINxKAIiRPFCpK304CyUUikUtIisHC0UhHoy8ZoaY2Lcodv8w3+WxcyMtu937/ubOLNliXuSN2/yvu/zeb/P8b483t8WvjlkjTopTmCMiQzrXBm4uIyu8T94d0ZGdERYf3Ji/JWIUL4hThB1QZiW8dvuvD0kJzv7aSh/Qy89F8p71QKW+Krr1VrUdF3A4RYzxB0jqNYPQ9Oqg+a0AeJTFhxoNqNCN4zx+49x+f4TCKU9yKruQ233JVTVNcBz7hwlB5s21SFd096FhMo+69xY0YR3gsTmFlVuDS9tmzDd+IGs3aO2zYsVWb3ixXbPeIltUXKlLb+uh6h6x8iCBDFxF1RYhXKjtUVvwEezZpbxYmN2XKnoOI/Z0cdsyzOqEJxRRXyTT6DZOE6iDmqJd7yILM+sQlC6nCz7UkEC0+RwjylHUUM/aeq/ShanVOLjqDL7oVYLkhLifuRlZ2f/mlXdD+YYlK7gDmzIb4SyZwzugnJQZfClawwUsFPGzUvp+HD7YdR0X4ZQpCdUDMlXGiEUpjzh5eTkPs/kgBUkiCpYmCjBF0d1OHP+Dg5qzJxTaHEzfJMrqUoF/IRSrN1VD/mZixi99RBKwxiZtf2wvVw3gpgdEdd5aanC7/aqBuAZJ7KzkGhekCruxND1B8iQnsUIdWofvgX/VBnohVSxCKKOUbQN3YS08wIkp0bJbIHI2thtQbD/okZe2OaN3dLTFtCEW4MzFPBmQEknDp00w21HGVbn1EHVdxUbC1TwS5WyiznVm4vUEIr1pETVTz7JVdlqmlpAayzgec1xKVGeNiIwo9YamCb7SyEFitpHOThTxPK5rUQDll9vmtNd1QbsVw8g8fgpUqTss2dW9WFXTtZTCpzFOmedRF6DrQfaaUFOEAohu2t6UKwygcFZqE3945xCBmSFYd3QbLpGMqVdEOkvWRXtJvj7eKom+/qt3OysR4UqM1Nj90mSEJZDRddFzNxWiq371FAbx7GEFoNVlxWGqf5K3g3T1fvEdPMR8gqLnlOO+8uXsio4oLHhrAU+KTIrKwztQ9I+cgdFzaM43m5Bfl0vPATHQfc4aGC6nKoXEcvdx7bbt29j2htOJorZREfQJHN9pbwa0cfO2OZEHSHLMmuJXPs1KS0uxIOH3xPd8C0SkqdkKrnG9t8pY00O7fC3RNs9aC8+Ino2ZLmIrVu2WHkAHDji6pXmjqFrWLe37Q/d0A20tWhsRaVHJ3bX9k60nrtrL240sgpTUBVYrwqO6IiFttQh7Qj6xx+i6WQrPOfPa2KsKS9UekVGRv6k1XcjYnv47/UNKlju/QJJSw8GBs3IrTawgnFhe9Nw02UGyGWyZ1s+/+zKquVBndQ/5tUPZxLqSkciHd5sDvBdoJk5480TwpSkewUNA1zRgmmOPePFthK1GSlJiZfpOYdJCI2Wx/Px8fkn9F+2v6S4a59qEF4JYittGcJ+nQrdOURHhvW+AE1539nZ8aWDq6srN0+fPt3Bzc3N0cnJycHFxcWxp880la0X5OW0HmgepK9JbF1Bf5758ZKJ+i4LVgQuVrB9Pp/vyHsdCwsPd+LmTSF7taZvsCBFYaXtQ2LFPVBqdOyp8dm+h4fH6wGdnZ0n8zM7v6Dw55PGMSj0o7bGlg6sX/Np/YtwHXj/x9575+3J21P9Fi1E8FI/pkzxX+f/BBhq9luuZDT4AAAAAElFTkSuQmCC","width":315,"height":325,"src":"/static/fff962ac6bfdb5c2156f04c8855b288b/b3029/thumbnail.png","srcSet":"/static/fff962ac6bfdb5c2156f04c8855b288b/b3029/thumbnail.png 1x,\n/static/fff962ac6bfdb5c2156f04c8855b288b/8d141/thumbnail.png 1.5x"}}},"authorName":"Core Services Team","authorDescription":"Core Services team is responsible for infrastructure and platform development at AUTO1","authorAvatar":null,"headerImage":{"relativePath":"pages/event-store-deadlock/header.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAAAsSAAALEgHS3X78AAABoUlEQVQoz2MIjM+pDy/sfhxb3HkyNL/zXlbD9O+h2Y0XPD3c1RmAwM07gElKSooBhiUlxBlklDQYsAGDhIkMDP45Xf+t8hb9L5y66792/NT/wbUr/hTOOPDfN76whIEAMIjv5zRMmsJkmDiJH8QH0gwMftmdD21zZv8vmrT5l0v+7L/pXev/5XSv+hMUneLFzMCgr6ikrCApKckGdJ0QEAtKyynx8jEwcOiG1csZJk62B7rKDojVgIa5gm2JKWyf5V29+n9G19r/zkADg2pX/p+2bNtNDSVZM1ExcRNpaWkzoEFuQBwKxL5SUtJBEqLCrspmni7G6XMcDeInhAMNlDJMnJhpkDhRlCE6Jcc4pWrio/krNr4Pqlzw1zZz2r8J0+dddffwdAJZaGJqxg51HTcQqwANlBfhYmBTNvcWNEqdJQX0tiDQQFXDhEnyQFfKwoKDvTLJ28A9u/+jR/6M7xmemtIgwczMLCYGUkFxTSsjjB2bXbk3pahhE4i98/oXsLiysgoDcixLScswiAtwMQBdyGCUOpMR6EJQZDACXQnGAOs+imBvfQhTAAAAAElFTkSuQmCC","width":500,"height":168,"src":"/static/2d78359581a2f1861a243566e527e27a/7d852/header.png","srcSet":"/static/2d78359581a2f1861a243566e527e27a/7d852/header.png 1x"}}}},"html":"Always try to keep your transactions short and write your queries to deal with records in a deterministic order with respect to other transactions.
\nAlmost every non-trivial system needs some kind of task (job, event, whatever you call it) scheduling functionality at some point. Having more than 250 micro-services, ours was no exception.\nAt some point, it made sense to us to provide such functionality as a service for our developers. With event-store service, one simply can schedule a job to be executed at a specific time in future. There are already several ready to use tools for this specific purpose (e.g. BigBen, Quartz), however, because of several reasons (which are beyond the scope of this post) we decided to implement our own from scratch.
Considering the subject of the post which is Deadlocks, we need a brief introduction on event-store's implementation.
An event on the server side is simply a record in the database. A simplified view of events table is as follow:
| id | \nevent_type | \ndue_time | \nstart_process_time | \nend_process_time | \n
|---|---|---|---|---|
| '5c55ea79-4774-49ae-96e2-975af7a41ce6' | \n'user-cache-flush' | \n'2019-01-01 00:00:00' | \nnull | \nnull | \n
The following is a bird's-eye view of what happens on the service:
\nThread #1
\na. The service periodically fetches a bunch of unprocessed events (those that are due and have no process time, neither start nor end) from the database (and marks them as progressing by filling the start_process_time column with the current timestamp). Let's call this fetch-unprocessed query.
b. The service processes the events and marks them as processed (by filling the end_process_time column with the current timestamp). Let's call this save-processed query.
Thread #2
\na. The service periodically checks the table and looks for stale events (those that are due and have start_process_time but no end_process_time). Let's call this fetch-stale query.
b. The service marks them back as unprocessed (by setting start_process_time to null). Let's call this release-stale query.
The very first MVP was a simple read-modify-write (anti-)pattern (with the help of Spring, JPA, and Hibernate). It is not hard to guess to what issue this implementation is vulnerable: Deadlocks.
\nERROR: deadlock detected\nDetail: Process 5234 waits for ShareLock on transaction 3465; blocked by process 467845.\n Process 467845 waits for ShareLock on transaction 96575; blocked by process 5234.\nHint: See server log for query details.\nWhere: while updating tuple (14954,4) in relation \"events\"ERROR: deadlock detected\nDetail: Process 10438 waits for ExclusiveLock on tuple (14954,4) of relation 19118 of database 19113; blocked by process 31501.\n Process 31501 waits for ShareLock on transaction 763124271; blocked by process 28450.\n Process 28450 waits for ShareLock on transaction 763124277; blocked by process 28873.\n Process 28873 waits for ExclusiveLock on tuple (14954,4) of relation 19118 of database 19113; blocked by process 10438.\nHint: See server log for query details.\nWhere: while locking tuple (6984,19) in relation \"events\"In the beginning it was not a big deal, because deadlock errors were not frequent and we could live with it. However, very soon, after getting more clients, the deadlock issue (among other problems, e.g. lost-updates) started hurting the quality of the service (e.g. firing events twice).
\nFor the first attempt, we refactored the code to get rid of read-modify-write antipattern:
\n-- fetch-unprocessed query\nUPDATE events event_to_update\nSET start_process_time = now()\nFROM (\n SELECT id\n FROM events \n WHERE end_process_time is NULL\n AND start_process_time IS NULL\n AND (due_time BETWEEN ? AND ?)\n) matched_event_for_update\nWHERE event_to_update.id = matched_event_for_update.id\nRETURNING *;\n\n-- release-stale query (including fetch-stale query)\nUPDATE events event_to_release\nSET start_process_time = NULL\nFROM (\n SELECT id\n FROM events\n WHERE end_process_time IS NULL AND start_process_time < ?\n) stuck_event\nWHERE event_to_release.id = stuck_event.id\nRETURNING *;Obviously this is an improvement, though reduced the number of deadlocks, it didn't solve the issue completely.\nThe quickest solution to eliminate deadlocks was to change the isolation level to Serializable. However this is the last solution we wanted since it would hurt the performance and the quality of the service (think what would happen if one service start a transaction, fetch the events to process, but for some reason become stale).
The better way to deal with deadlock is to analyze transactions and queries to prevent transaction interference by:
\nBy the first refactoring, we made transactions a bit shorter. But there's still some room for improvement. To apply the second best practice from the above we refactored the queries as follow (essentially ordering and acquiring a lock):
\n-- fetch-unprocessed query\nUPDATE events event_to_update\nSET start_process_time = now()\nFROM (\n SELECT id\n FROM events \n WHERE ... -- as before\n ORDER BY ID\n FOR NO KEY UPDATE\n) matched_event_for_update\nWHERE ... -- as before\nRETURNING *;\n\n-- release-stale query (including fetch-stale query)\nUPDATE events event_to_release\nSET start_process_time = NULL\nFROM (\n SELECT id\n FROM events\n WHERE ... -- as before\n ORDER BY ID\n FOR NO KEY UPDATE\n) stuck_event\nWHERE ... -- as before\nRETURNING *;However, the deadlocks were still appearing intermittently:
\nERROR: deadlock detected\nDetail: Process 7534 waits for ShareLock on transaction 93897000; blocked by process 13376.\n Process 13376 waits for ShareLock on transaction 93896994; blocked by process 7534.\nHint: See server log for query details.\nWhere: while rechecking updated tuple (134131,3) in relation \"events\"Of course! we still had one transaction/query dealing with records in a non deterministic way: JpaRepository.save(Iterable<S> entities). In the code, after fetching events to be processed, we flag (UPDATE) the process events as processed, and also save (INSERT) new events spawned by the processed events. Here's the catch: the event processing is done in parallel. So the resulting list of events (both processed and newly created ones) is not ordered with respect to other transactions. So the third refactoring was to restore the order of processed events:
/**\n * To process events.\n * The result list will be ordered with respect to the given list.\n * All new events will be added at the end of the result list.\n * @param events events to be processed\n * @return list of processed and created events\n */\nprivate List<Event> doProcessEvents(List<Event> events) {\n ProcessingResult result = eventProcessor.processInParallel(events);\n\n List<Event> successfulEvents = result.getSucceeded(); // may contain new events;\n List<Event> failedEvents = result.getFailed();\n List<Event> allEvents = ListUtils.union(successfulEvents, failedEvents);\n\n Map<UUID, Integer> eventIndices = range(0, events.size()).boxed()\n .collect(CollectorUtils.toMap(i -> events.get(i).getId(), i -> i));\n\n // This should sort according to the original order and put all new events at the end.\n allEvents.sort(comparingInt(event -> eventIndices.getOrDefault(event.getId(), Integer.MAX_VALUE)));\n\n return allEvents;\n}After this last change, we saw the game for deadlocks was over: \"Goodbye Deadlock!\"
","fields":{"slug":"/event-store-deadlock/","tags":["deadlock","sql","transaction","postgresql"]}}},{"node":{"id":"f1483025-8f93-596d-89a4-e459c4718414","frontmatter":{"category":"Coding","title":"Database Transaction Isolation","date":"2018-06-20","summary":"What is transaction isolation level? What are differences? What are the default values for RDBMSs?","thumbnail":{"relativePath":"pages/transactionisolation/thumbnail.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsSAAALEgHS3X78AAABIUlEQVQoz5VS7U7DMAzsa6A1cRynTT/WrTBNiB8gJN7/mczFVWHTWiF+nM750OXsS8XMGjhqbDplaRZuMupWKSZ1zqn3/g5EtIsqBNIAoTRdNeajpuPFIMOs0p+0Pjypqw9AvbADQ7Q8tAUIBg1wwk0PwdGEuO2VUTPcynAGnrE3GlNsFnHnH5wXVGvLQbLmy4ehe/3SfP20OnWTpmHQbh41n9BB32IvquS4OQ4TpLIgj7mhDh6OyyPB1t47c1TYu4UJYyr3dwXtgGpNmSFCgDdOHcbBZO3dw/2I3da/Dlm0nd8ww0ljf14CAsv4gofCppNV7MGhoD1uB8zrHa2wfZUSRuGSvve0K7gZSkrpJvZ6wfo9LE2362ZTUET+vPwfh9/1JBQjnS3wvwAAAABJRU5ErkJggg==","width":677,"height":325,"src":"/static/2691083d1efbcb8a271d80bf9ff6940f/b3029/thumbnail.png","srcSet":"/static/2691083d1efbcb8a271d80bf9ff6940f/b3029/thumbnail.png 1x"}}},"authorName":"Hayk Jhangiryan","authorDescription":"Hayk is Senior Java Engineer at AUTO1 Group.","authorAvatar":{"relativePath":"pages/transactionisolation/avatar.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAACXBIWXMAAAsSAAALEgHS3X78AAACyElEQVQ4y4VUO2gUURR9mZmN8YcItoKVjeXuzsxu9jOzm92NkBBTqDEmjZAEgtqIpSgI1gp+uhS6kEI7wc7GQsFCLERiISGCaGHhF02yu+M5s/cujyC4cJY379573r3n3XdNvV51xsYiJ4rKbqlUyBj8wjAYyWazp4BVYB3YAraBDeAhMBcE/l76lkrFTLVachuNyCGXEbJMrVbx6ADnaWAtn88nvu8nWBMdQcJ9Auv3wCxjWq2aC9IMSF3DzCyyG1YAs9pSYkL2melmLpfT71tKGkUlz2iZJAuCgA5dEjGAwPoV0AbuAy+tfR7YkZjb5CiXR1OutEw5jWQdCfgCnEiSxNg/7LWAj+KTSiHSzBvrAtascrriPCmnDkP4AYQ0Em3TGPHf8P38ARpP/4PsHQ7aU69Hqdi4OJeI47JXLg9IX1tVbUuWCzSsygcJO+L0goGNRpySoB1colarerhEvcCn4tuxCB/TsC5ZdS3Ct2Ho75uZmRoigWbIbgCxt7Q0O8TLsgg17o2R2+JHzzJcV/1IoBlybel42SLsCcc3I6UqoaZ+lUHFYjjCzJSQa+4J4UVLKiX8ScOGVXJXDN+xd7SfZXFYS65U+n2Wz+cOw+ezHadSkfCRdVJiZdlmMG9ZtHMpgWR3Z0eMSvWExjmrbXoCbZ951dIim7Yk6u1IYtlwavChW51Pp01tXOwf01eC9RHs/Rbbnx2Hf4IUh/Q5zdol8H3KG22Pj9edOK54LF1876pd37TELhqdZ+J4MwxDHUvXgDOFQrB7YqLpjI4WdhG8GLYNbCeBK/1+TWNWRG+Pfy5Hj5Dek3IXzX9+8DkL/AIeLC/PDU1NHXdSQk5altNsDkjPAT+ArzKyFnK5bB2IxbYik4hanmfM5GTL6Q/Y2DUc25y0HLToubR8TI39cL4APJOxr/35AXgOXEJnHKQvepP6eiQj119kDrZjqLo8HAAAAABJRU5ErkJggg==","width":50,"height":50,"src":"/static/26a7a327ccc4b335712e5a7086f2b26d/45876/avatar.png","srcSet":"/static/26a7a327ccc4b335712e5a7086f2b26d/45876/avatar.png 1x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/eb85b/avatar.png 1.5x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/4f71c/avatar.png 2x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/9ec3e/avatar.png 3x"}}},"headerImage":{"relativePath":"pages/transactionisolation/header-image.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsSAAALEgHS3X78AAABIUlEQVQoz5VS7U7DMAzsa6A1cRynTT/WrTBNiB8gJN7/mczFVWHTWiF+nM750OXsS8XMGjhqbDplaRZuMupWKSZ1zqn3/g5EtIsqBNIAoTRdNeajpuPFIMOs0p+0Pjypqw9AvbADQ7Q8tAUIBg1wwk0PwdGEuO2VUTPcynAGnrE3GlNsFnHnH5wXVGvLQbLmy4ehe/3SfP20OnWTpmHQbh41n9BB32IvquS4OQ4TpLIgj7mhDh6OyyPB1t47c1TYu4UJYyr3dwXtgGpNmSFCgDdOHcbBZO3dw/2I3da/Dlm0nd8ww0ljf14CAsv4gofCppNV7MGhoD1uB8zrHa2wfZUSRuGSvve0K7gZSkrpJvZ6wfo9LE2362ZTUET+vPwfh9/1JBQjnS3wvwAAAABJRU5ErkJggg==","width":838,"height":402,"src":"/static/2691083d1efbcb8a271d80bf9ff6940f/d5ccc/header-image.png","srcSet":"/static/2691083d1efbcb8a271d80bf9ff6940f/d5ccc/header-image.png 1x"}}}},"html":"Transaction isolation level defines the degree to which one transaction must be isolated from resource or data modifications made by other transactions.
\nLet's have a look at phenomena which can occur during the execution of concurrent transactions:
\nDirty read - A transaction may read data written but not yet committed by other transactions.\n
example: Transaction T1 modifies a row. Transaction T2 then reads that row before T1 performs a COMMIT. If T1 then performs a ROLLBACK, T2 have read a row that was never committed and that may be considered to have never existed.
\nNonrepeatable read - A transaction re-reads data it had previously read and finds that data has been modified by another transaction (that committed since the initial read).\n
example: Transaction T1 reads a row. Transaction T2 then modifies or deletes that row and performs a COMMIT. If T1 then attempts to reread the row, it may receive the modified value or discover that the row has been deleted.
\nPhantom read - A transaction re-executes a query returning a set of rows that satisfy a search condition and finds that the set of rows satisfying the condition has changed due to another recently-committed transaction.\n
example: Transaction T1 reads the set of rows N that satisfy some search condition. Transaction T2 then executes SQL-statements that generate one or more rows that satisfy the search condition used by transaction T1. If transaction T1 then repeats the initial read with the same search condition, it obtains a different set of rows.
\nIsolation is the I in the acronym ACID. In ACID, atomicity and durability are strict requirements. Whereas consistency and isolation are more of configuration (kind of). Besides, they are closely related, so much so that the SQL standard defines four levels of transaction isolation based on the consistency they provide (from least to the most consistency):
\n| ISOLATION LEVEL | \nDIRTY READ | \nNON-REPEATABLE READ | \nPHANTOM READ | \n
|---|---|---|---|
| READ UNCOMMITTED | \npossible | \npossible | \npossible | \n
| READ COMMITTED | \n-- | \npossible | \npossible | \n
| REPEATABLE READ | \n-- | \n-- | \npossible | \n
| SERIALIZABLE | \n-- | \n-- | \n-- | \n
Different isolation levels are defined based on whether they allow (or prevent) the above phenomena (as described by the SQL standard). Note that transaction isolation does not affect the changes made by the same transaction. It means a transaction always sees all the changes made by itself.
\nInside Java services, the transaction isolation level can be controlled by Spring @Transactional annotation which supports isolation attribute. If you do not specify the isolation level in the code explicitly, the default isolation level will be applied, which means that the default isolation level of the underlying datastore will be used.
It is worth to mention that default isolation levels for each system might be different. For example, the default isolation level for MySQL 5.7 (InnoDB) and PostgreSQL 9.5 is respectively, REPEATABLE READ and READ COMMITTED. To check the default isolation level, query:
\nSHOW VARIABLES LIKE 'tx_isolation'; -- on MySQLSHOW default_transaction_isolation'; -- on PostgreSQL