{"data":{"allMarkdownRemark":{"edges":[{"node":{"id":"458b814a-9266-529b-a538-b64d05af8cc6","frontmatter":{"category":"QA","title":"How to O(1) your test preparation","date":"2019-01-29","summary":"Are you sure your tests are fast enough?","thumbnail":null,"authorName":"Alexander Gyulai","authorDescription":"Head of Quality Assurance at AUTO1 Group","authorAvatar":{"relativePath":"pages/faster-automation-tests/avatar.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAACXBIWXMAAAsSAAALEgHS3X78AAACyElEQVQ4y4VUO2gUURR9mZmN8YcItoKVjeXuzsxu9jOzm92NkBBTqDEmjZAEgtqIpSgI1gp+uhS6kEI7wc7GQsFCLERiISGCaGHhF02yu+M5s/cujyC4cJY379573r3n3XdNvV51xsYiJ4rKbqlUyBj8wjAYyWazp4BVYB3YAraBDeAhMBcE/l76lkrFTLVachuNyCGXEbJMrVbx6ADnaWAtn88nvu8nWBMdQcJ9Auv3wCxjWq2aC9IMSF3DzCyyG1YAs9pSYkL2melmLpfT71tKGkUlz2iZJAuCgA5dEjGAwPoV0AbuAy+tfR7YkZjb5CiXR1OutEw5jWQdCfgCnEiSxNg/7LWAj+KTSiHSzBvrAtascrriPCmnDkP4AYQ0Em3TGPHf8P38ARpP/4PsHQ7aU69Hqdi4OJeI47JXLg9IX1tVbUuWCzSsygcJO+L0goGNRpySoB1colarerhEvcCn4tuxCB/TsC5ZdS3Ct2Ho75uZmRoigWbIbgCxt7Q0O8TLsgg17o2R2+JHzzJcV/1IoBlybel42SLsCcc3I6UqoaZ+lUHFYjjCzJSQa+4J4UVLKiX8ScOGVXJXDN+xd7SfZXFYS65U+n2Wz+cOw+ezHadSkfCRdVJiZdlmMG9ZtHMpgWR3Z0eMSvWExjmrbXoCbZ951dIim7Yk6u1IYtlwavChW51Pp01tXOwf01eC9RHs/Rbbnx2Hf4IUh/Q5zdol8H3KG22Pj9edOK54LF1876pd37TELhqdZ+J4MwxDHUvXgDOFQrB7YqLpjI4WdhG8GLYNbCeBK/1+TWNWRG+Pfy5Hj5Dek3IXzX9+8DkL/AIeLC/PDU1NHXdSQk5altNsDkjPAT+ArzKyFnK5bB2IxbYik4hanmfM5GTL6Q/Y2DUc25y0HLToubR8TI39cL4APJOxr/35AXgOXEJnHKQvepP6eiQj119kDrZjqLo8HAAAAABJRU5ErkJggg==","width":50,"height":50,"src":"/static/26a7a327ccc4b335712e5a7086f2b26d/45876/avatar.png","srcSet":"/static/26a7a327ccc4b335712e5a7086f2b26d/45876/avatar.png 1x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/eb85b/avatar.png 1.5x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/4f71c/avatar.png 2x,\n/static/26a7a327ccc4b335712e5a7086f2b26d/9ec3e/avatar.png 3x"}}},"headerImage":{"relativePath":"pages/faster-automation-tests/header.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsSAAALEgHS3X78AAABtklEQVQY0wGrAVT+AAACBAAEBwAECAQECA0ECRoECCwFCEEFCVYECGsECHkECYYECZUGDKIKEKEMEaMLEqcQHK4aJKo5PjghPwBTAwdiAgRvAQOBAQONAAORAQWOAweFBAmCCA18Cw9zDhFlEhRMEhI4ERdGHC5XOlZyXHx9hqiGosqfkLcAXQEEUwIFSgkNNgsQJA0SHxMWGBMVERUVCBMSARAPAA8SAQ0VAAsjHy1Rd36i0NTk9Pv//f3+/8zj14CnAAcSFgUTFgcQEAoQDgsODAYICAUGCQUFCwQIEwQIGwIJLBAbRz9Md5CevNje7f/q8PvT4c6Yu21CexENPgAIBQEHAwMBAAUAAAgABBQCCRkDDB0CDB8LEy8tN1xub465qb/uwdb3r8zdi7i4cKZ5VYsgJlkABSkBAhYAAAIQAAUTAwobBxAgBA4ZCBAfISY4WFtznZau0rTI76rD5YGoxlaSkzyAVS9xGiRcABM+AQklBQkXAQQLAAYOGAsVHw4ZIBUaJ0dDWJGOoM3Azty0xtGIqbZXiaA7fY0qdUwkZREbUgAQPQAGJAMEFQMGEgACBwAAAMagbU9WOO54AAAAAElFTkSuQmCC","width":1280,"height":445,"src":"/static/34ed0cae6856ac71c7fea1ac46b6a5a9/26421/header.png","srcSet":"/static/34ed0cae6856ac71c7fea1ac46b6a5a9/26421/header.png 1x"}}}},"html":"

(*) If your career as a QA - like the author’s - started without a computer science degree, you will hopefully love\nto hear and learn about the different theories and notations of runtime analysis.\nCheck wikipedia’s page about O-notation here

\n

Speed… Did you ever think how you can make your tests faster and more reliable? If you're anything like me or my colleagues at AUTO1,\nyou face this challenge all the time. Here at AUTO1 we are constantly looking for new ways to improve our tools and\nto make testing fast and efficient. In this article, I’m going to take test suite that takes ~70 min and reduce the execution time to 1 min!!!\nInteresting, right? :)

\n

Intro

\n

Let’s imagine we are testing some Web Shop Scenario. We need to test that users can do some actions with items they purchased\n(submit feedback, return, submit claim, etc.). And imagine that in this case we have exactly 20 scenarios that go like this:

\n
Steps:\n- Create item, put item in certain state\n- Create user\n- Create user delivery address\n- Create user billing address\n- Create user payment details\n- Sell item to user\n- User does something with an item
\n

Back in times…

\n

Let’s look at the timeline of our test

\n

\n \n \n \n \n

\n

We significantly increased the speed. From ~73 to ~8 minutes. This already looks like a game changer! But can we do more? Spoiler: Yes we can!

\n

Removing dependencies

\n

If we check our test scenario again, we can clearly see that the actual validation of our test scenario happens only\nafter the third minute! And everything we have up until then it’s just creating preconditions.\nWhat we actually want to test is: “User can do certain action with item purchased on our platform”.\nWe should not care HOW we got this item, we should have another test, testing that user should be able to purchase.\nThis just creates dependency for us that makes our tests:

\n
    \n
  1. Slower
    2. \n
  2. Less stable. It becomes less stable because we have to go through multiple steps (in some cases maybe through multiple\napplications) with creating item, creating user, adding user info and so on. If something will fail in the middle (which can happen easily)\nit doesn’t really mean that user couldn't leave a feedback on the item.
    3. \n
\n

P.S. While writing this article I tried to re-create all these conditions and I didn’t have any successful build.\nAnd it was always something related to other dependencies.

\n

Solution? There are many possible answers, the one I chose for this example is to move all logic with test data preparation to the API layer.\nLet’s try this and review our timeline again

\n

\n \n \n \n \n

\n

Amazing, this is how our test looks now! two minutes just gone.

\n

And this is how our suite looks now comparing to other runs.\n\n \n \n

Having test preparation using API is a great step here, but does it scale from 20 tests to 1000 without having side-effects?\nAt the end, your test data preparation might look like this:

\n

\"machingun\"

\n

Assuming you have around 40 calls with a reasonable fan-out for every scenario and you usually run tests in multiple threads,\nyou might involuntary start a load test there. Aaaaand it’s time to look for a solution again!

\n

Introducing Q-Service

\n

What if we have our test data waiting for us somewhere? Wouldn’t be amazing?!\nOur solution for this problem was to create a Web Service that would keep the data in a queue for us.\nIt provides couple of huge benefits for us:

\n
    \n
  1. Instead of doing all these calls in runtime, we do 1 single call to our API to get test data. And instead of 1 min in test preparation we got <1 sec!
    2. \n
  2. We don’t overload the system while executing tests. Which also means for us we can run more tests in parallel!
    3. \n
  3. Tests are way more stable. Now we just need to take our test data and check that user can do certain action with that item.
    4. \n
\n

Let’s look at our timeline again. This is how our test scenario changed:

\n

\n \n \n \n \n

\n

We barely can see it on our timeline :)

\n\n

How does it work?

\n

Stay tuned! We will open-source it soon!

\n

Summary

\n

In AUTO1 Group, every day we are looking for smarter ways to do software development and testing.\nAnd the main point in this article is that you should never stop innovating and looking for new solutions to existing problems.\nSomething that works for us, is not necessarily going to work for you,\nbut maybe you will ask yourself: “Where is our next level? Can we do something better?”

","fields":{"slug":"/faster-automation-tests/","tags":["testing","automation"]}}}]}},"pageContext":{"slug":"/tags/automation","tag":"automation","categories":["Architecture","Coding","DevOps","Engineering","ProjectManagement","QA","Social","TechRadar"]}}