Having test preparation using API is a great step here, but does it scale from 20 tests to 1000 without having side-effects?\nAt the end, your test data preparation might look like this:
\n![\"machingun\"](\"/static/machingun-f6e9bbc355090a0e8c6db2e891dfb64a.gif\" "\\"Suitemachingun\\"")
Assuming you have around 40 calls with a reasonable fan-out for every scenario and you usually run tests in multiple threads,\nyou might involuntary start a load test there. Aaaaand it’s time to look for a solution again!
\nIntroducing Q-Service
\nWhat if we have our test data waiting for us somewhere? Wouldn’t be amazing?!\nOur solution for this problem was to create a Web Service that would keep the data in a queue for us.\nIt provides couple of huge benefits for us:
\n- \n
- Instead of doing all these calls in runtime, we do 1 single call to our API to get test data. And instead of 1 min in test preparation we got <1 sec! \n
- We don’t overload the system while executing tests. Which also means for us we can run more tests in parallel! \n
- Tests are way more stable. Now we just need to take our test data and check that user can do certain action with that item. \n
Let’s look at our timeline again. This is how our test scenario changed:
\nWe barely can see it on our timeline :)
\n- \n
- 1 test now takes ~30s to run (instead of 3m 40s). And it’s “slow” and “unstable”(as it’s usually called) UI end-to-end test. \n
- Entire suite takes ~1m \n
- We can easily double the number, if we need, since these tests are producing a very little amount of our resources \n
How does it work?
\nStay tuned! We will open-source it soon!
\nSummary
\nIn AUTO1 Group, every day we are looking for smarter ways to do software development and testing.\nAnd the main point in this article is that you should never stop innovating and looking for new solutions to existing problems.\nSomething that works for us, is not necessarily going to work for you,\nbut maybe you will ask yourself: “Where is our next level? Can we do something better?”
","fields":{"slug":"/faster-automation-tests/","tags":["testing","automation"]}},"categoryArticles":{"edges":[{"node":{"id":"a26825f5-bab1-5ad6-bc95-41f8c672ba4c","frontmatter":{"category":"QA","title":"Efficient Testing - Automate Early","date":"2022-06-01","summary":"Early bird gets the worm - or how preparing in time can reduce the effort needed in manual testing.","thumbnail":null,"authorName":"Joanna Tyka","authorDescription":"Joana is a QA Automation Engineer in our Katowice office.","authorAvatar":{"relativePath":"pages/efficient-testing-2/JoannaT.jpeg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAATABQDASIAAhEBAxEB/8QAGAABAQEBAQAAAAAAAAAAAAAAAAUCBAH/xAAVAQEBAAAAAAAAAAAAAAAAAAABAv/aAAwDAQACEAMQAAABqc+OOWqwSV4DXFH/xAAcEAEAAQQDAAAAAAAAAAAAAAACAwABBCEQETH/2gAIAQEAAQUCSsDHJd8ZS6gie6yaWk/f/8QAFREBAQAAAAAAAAAAAAAAAAAAASD/2gAIAQMBAT8BI//EABQRAQAAAAAAAAAAAAAAAAAAACD/2gAIAQIBAT8BH//EAB0QAAEEAgMAAAAAAAAAAAAAAAEAAhARAyFBUXH/2gAIAQEABj8CLjwtgC4I7TGwPVjqP//EABsQAQEBAAIDAAAAAAAAAAAAAAERABAxIVFx/9oACAEBAAE/IeogXCDAUjcuKkWDGQR7X7rlS/fEZ4Kl4P/aAAwDAQACAAMAAAAQ9AjA/8QAFxEBAQEBAAAAAAAAAAAAAAAAARARMf/aAAgBAwEBPxACOw5P/8QAFxEBAQEBAAAAAAAAAAAAAAAAAAERMf/aAAgBAgEBPxC9Yqv/xAAdEAEAAwADAAMAAAAAAAAAAAABABEhMUFhgaHB/9oACAEBAAE/ELDbyryGiEG8eM0myDEXXZjO3Q44B9sdxnK12+IIWkHfEAGuz9Z//9k=","width":50,"height":50,"src":"/static/d453ba1bcdc86e2db725a048eefd13a6/d2d31/JoannaT.jpeg","srcSet":"/static/d453ba1bcdc86e2db725a048eefd13a6/d2d31/JoannaT.jpeg 1x,\n/static/d453ba1bcdc86e2db725a048eefd13a6/0b804/JoannaT.jpeg 1.5x,\n/static/d453ba1bcdc86e2db725a048eefd13a6/753c3/JoannaT.jpeg 2x,\n/static/d453ba1bcdc86e2db725a048eefd13a6/31ca8/JoannaT.jpeg 3x"}}},"headerImage":null},"html":"Efficient Testing - Automate Early
\nI don't have time for automation - how often do you say that? I said that once and now I feel a bit ashamed thinking about it. To never repeat this mistake I had to find some solution, to not only have enough time for automation but also to be much faster using it. I realized that when I was repeating the same test case manually for the 5th time, it was incredibly boring and each try was taking 15 minutes of my time. Today I would like to share my experience with you on how to make it faster and more pleasant. In the big picture, it is all about being agile, splitting the job between small pieces, done as fast as possible. Before the feature is fully ready for tests we have so many different opportunities to start working on automation.
\nPlanning and reconnaissance
\nAn hour of planning can save you 10 hours of doing - Dale Carnegie
\nAs soon as test scenarios are prepared, you can select candidates for automation and choose which ones should be executed on API level and which on UI. It should be a part of a well-prepared automation strategy (more about test automation strategy here https://auto1.tech/efficient-testing-1/ ). Check whether all necessary API calls are available for you (maybe you need to ask developers to create new endpoints?), what will be the steps to prepare the environment for the test, explore existing automation infrastructure and recognize how many new methods and classes you will need to prepare. Sometimes our features are settled on some other pages or we need to go through other features or applications to do our tests. As these parts already exist, it is a great time to cover it with automation if nobody hasn't done that before. It is also a great time to consider which tools will be needed and find new libraries that will be more effective.
\nTest data preparation
\nTest data means all that needs to be prepared before your real test will start. When you are sure that you know all the preconditions and steps required to arrange the system before beginning the testing, you can recognize how to prepare it just when the first version of the specification is written. For example, let's take an order in the shop. When developers write their code you can write your own which will prepare test data via API, so following our example the order in specified state. To make it more efficient you can store example of the order in the queue (read more about it here https://auto1.tech/faster-automation-tests/). It will be so helpful when you will need to execute the first most basic test and speeds up the whole process significantly. Developers in your team will be also very grateful for the created test data that they can take and quickly test their changes on.
\nPage Object skeletons
\nIf you use Page Object Pattern you can create a page objects skeleton just now based on the design. Prepare all classes storing variables with empty locators, write all methods which will use these locators to interact on the page and prepare bigger steps that will do your manual work. When the front end part of the feature will be implemented you can just fill in locators one by one look at the DOM and adjust steps easily. With unique locators added by developers, it will be almost a pure pleasure.
\n![](\"https://i.postimg.cc/tXm79NqD/image.png\")
Let's take this design of the saved searches page as an example and imagine a test scenario that verifies that saved searches are displayed on the proper page and the user can use a saved search to view the current results of searching. We know that to be able to execute this scenario we need the signed-in user so we can prepare this user just now. Looking at this picture we already know that we have an upper menu that should be already covered with automation (if it isn't covered, the time to do it is just now) and we need to use it to open my account view. We have also a side menu to open the saved searches page from my account context. We know how the saved search will look and which button we will need to click to open search results. Having all this information we can prepare the saved search page class, empty locators for all elements and methods that will interact with the page.
\nDoubts?
\nMany times I saw this approach to automation. I will do everything manually and leave the automation creation at the end or it doesn't make any sense to create automation now, because I will have to change it. Have you seen something similar? But how much time will be needed then to write all automated tests? Two weeks, or more? Who will give this time, when a new urgent feature is upcoming? From my perspective, it is really hard to find that person and it is well-argued. That's why in my humble opinion it is much better to split your time into small sessions before the feature is ready for tests and then have all automation components already done, then just write tests on demand. Isn't it beautiful having valuable tests from the start and be able to use them as regression tests at any time?
\nIn real life
\nI will give you my example: I was testing integration between two systems. Requirements were very agile, five teams were engaged in this feature and multiple layers of integration were created, so we had many bugs by the way. I had to repeat every test scenario 10 times on average. Manually test data preparation for each took 10 minutes, the real testing part between 5 and 10 minutes. After the third attempt, I was tired and frustrated. The more tired and frustrated I was - the more mistakes I made. It was a critical point for me to start writing automated tests. Of course, I had to spend a few hours to have it ready, but what a relief I felt when the 15 minutes scenario was done by my automation in 3 minutes. The other big benefit was that automation doesn't make mistakes and I could run my tests how many times I needed so I've stopped caring about the number of repetitions. I was able to share these tests with my colleague that needed them for testing the other part of this integration. My rough calculations show that I saved a few days and so many hairs which I would have ripped out of my head if I had to do all these tests manually to the end. When I was asked about regression I needed only a few minutes of my effort.
\nConclusion
\nWe have so many opportunities to be well prepared for upcoming features and to reduce the effort needed in manual testing. Creating smart automation will develop our coding skills and help to save time and frustration. I am sure that there are many other ways to receive significant benefits by automating early, we just need to use them.
","fields":{"slug":"/efficient-testing-2/","tags":["auto1","engineering","qa","testing"]}}},{"node":{"id":"43a222cd-fe2c-5cb5-af76-53e7d4ad29ec","frontmatter":{"category":"QA","title":"The journey of a QA engineer in AUTO1 Tech","date":"2022-03-31","summary":"Check out Dhivya's journey in our QA department & how our stack and team contributed.","thumbnail":null,"authorName":"Dhivya Venugopal","authorDescription":"Dhivya is a Junior QA Engineer in the AUTO1 Berlin office.","authorAvatar":{"relativePath":"pages/qa-engineer-journey/dhivya.jpg","childImageSharp":{"resolutions":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAAUABQDASIAAhEBAxEB/8QAGAABAAMBAAAAAAAAAAAAAAAAAAEEBQP/xAAWAQEBAQAAAAAAAAAAAAAAAAABAAL/2gAMAwEAAhADEAAAAXWvULaZ6pqGyQn/xAAcEAACAQUBAAAAAAAAAAAAAAABAgADBBESIRP/2gAIAQEAAQUCpnisScxMquzerXOJXGkTon//xAAVEQEBAAAAAAAAAAAAAAAAAAARIP/aAAgBAwEBPwFj/8QAGBEAAgMAAAAAAAAAAAAAAAAAAAECECH/2gAIAQIBAT8BUTb/AP/EABwQAAEEAwEAAAAAAAAAAAAAAAAQESFBARIxMv/aAAgBAQAGPwIy/KSTY8EDUn//xAAbEAACAwEBAQAAAAAAAAAAAAABEQAxQSFRgf/aAAgBAQABPyHykFbsz0UKMJmkC9nELAfyNrrluBBEa2WJpCJ//9oADAMBAAIAAwAAABCHMMH/xAAZEQEAAgMAAAAAAAAAAAAAAAABABEQITH/2gAIAQMBAT8QBVM1Hrj/xAAXEQEBAQEAAAAAAAAAAAAAAAABEDFB/9oACAECAQE/EE07BhP/xAAcEAEBAAMAAwEAAAAAAAAAAAABEQAhMVFhcaH/2gAIAQEAAT8QUhkEVKfHjA10KQp0jMUp9wY6EihV16w0n7o+2sU+sPC/mXadbXuBNStjW5gkDgZ//9k=","width":50,"height":50,"src":"/static/d37e68c1bb8fcdb464c251c9a15e7dbe/d2d31/dhivya.jpg","srcSet":"/static/d37e68c1bb8fcdb464c251c9a15e7dbe/d2d31/dhivya.jpg 1x,\n/static/d37e68c1bb8fcdb464c251c9a15e7dbe/0b804/dhivya.jpg 1.5x,\n/static/d37e68c1bb8fcdb464c251c9a15e7dbe/753c3/dhivya.jpg 2x,\n/static/d37e68c1bb8fcdb464c251c9a15e7dbe/31ca8/dhivya.jpg 3x"}}},"headerImage":null},"html":"Test data management using SHIFT
\nThe test cases, scenarios, and strategies are maintained in SHIFT, our internal (and soon to be open-sourced) test case management system. Our QAs make use of a test data tool which is integrated to SHIFT. The 100+ types of test data required by the teams are generated using automation suites that run in Jenkins. This test data is then queued in SHIFT and as a result increased the productivity of our QA team by reducing the effort for manual test data creation. Our director of QA has briefed this here https://auto1.tech/faster-automation-tests/.
\nSeamless Integrations
\nOur CI system, Jenkins, is wired to run these tests against our various QA environments. The integration of SHIFT with Slack, Jenkins, and Jira has helped the teams immensely. The status of Jenkins jobs/sanity suites is notified in Slack. The time taken to debug the issue is hugely reduced since SHIFT displays a precise list of failed tests including the screenshot reference, which helps me to quickly understand the exact point of failure. Once I have identified the issue, I fix it and test it locally, then push the branch to our repository. The process is set in such a way that merge can be done only after review, which gives me more confidence and validation for the proposed solution. This helps me address the issues faster and get the system up and running. This is indeed why the QA team at AUTO1 is one of the key pillars of the Engineering team.
\nMy automation testing journey at AUTO1 could not have been this impactful if not for this perfect blend of the best tech stacks and resources, and most importantly, my amazing team. My learning on the job has been very effective and hands-on, and it makes each day filled with new challenges and new learning opportunities that help my work create a real impact.
\nFrom the very many opportunities that came my way and the deep learning curve it has presented me with, I now realize that - the journey has just begun!
","fields":{"slug":"/qa-engineer-journey/","tags":["auto1","engineering","testing","QA","social"]}}},{"node":{"id":"0e79f71e-bac7-5c18-b5ae-ae1cec933364","frontmatter":{"category":"QA","title":"Efficient testing - Test Strategy","date":"2021-10-08","summary":"The first article in a series of articles focusing on efficient testing.","thumbnail":null,"authorName":"Michal Chytroszek","authorDescription":"Michal is the QA lead of the Car Journey associated teams.","authorAvatar":{"relativePath":"pages/efficient-testing-1/Michal.png","childImageSharp":{"resolutions":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAIAAAAC64paAAAACXBIWXMAACE4AAAhOAFFljFgAAAEyklEQVQ4yx2Qe0xTdxzFr9Dee3/30dpCRQqsQofyEEQpT8v6ABwgUPoASpUiMigKHeVReZTy0BYEh6TVtohKsbCNqAiFKPhgY3GYqAPMhkaWzEyXbG5z2ZZliX8sY5clJ998//mcnHMgg83Tah28e+2k7Uxb36Uxy/B4w1mP4bSr8WRvtjQ5I57X47R3uYY1avkxfZX+hL5CKXbUZpbLJVVHiiDLN/+euvfDl4/mHj24dnXh4f1Xv0y/+Htw6fehlbfywxWi2OCx5VfO1bfuJ3/VWFy9o580G/WL5ysHLnpu3ZyFWj//o8O7dnv5q/kHS947M3efPn/8+rfTX/w0+PXG8c7+tChu0+BQdfegXG/pu70+v/6yratjskfjmrhxzjUCGRfedCz8emlqbuaze8vfrV9f+VZfU5knTTZYBuptngOJkbWGWm1pcUpyYk2T0dRmqNOVTnRrHOesDVUlkME+1uC62ugY7xy91uWezN2/dxcORTKgZC5Q5OWLBRFFMmleekJq3E5VrrDq6OHGD6tnzpW2dylOlL4HWV9unH650ff9P2d+3miafiiPZjsVPLcywFPo156zI3k3L1McJ0qKEQkiNIdy7cPOZrNJeZinqhRrVRlQ48Wp5pFZ08h0z/iMuX/AKg/3loXNargLWn/PIX62MFp+MEEiTk3dHaKV729oNqoz99QXBLVo4oqyEqDe8VvOqzfdc/cvTM17LjsvHxEURm01p3F0e4haEV99ME2pSpfnC3X52ztNOaOrb0ZbBet96NDxhPJDBdDd1bVnKzf+fHZlbu3FWH+HVxM2URQ6VcQbzd3mLQmpFocnJsY0lPMrlL4HFUzPpOO6VeIyZelbzCp9N9Qy/Wzx4eKPT9w3nzx3zy/OdmtsGQH6ff7mRMZIQYBKEiMWhEglHJmCa+zQdtts5o/sirreYrNDVtsBtc6s9dx5ar+zapl9fGrp9YBzyKMK7s0MtmWSLYWJkXsTynRHjfYLdYOjdbbrOqu73GwvrOss6XTJDFZI//GDdu9y/8K62btSc35SmJ5dKeTqkvzUsUQEf0dQME+Up9G2n6/q81RYhss6HdqWM7Jqk6p9KFdvgeo/Xapz3pDK1Pxd0SwCp/v44BgRzsEJBN6yxRem03GaDxOGRPkardlR3NhXXG/NLm/KMpyVVrRBxomlhP1SGIIQmi8GAIskEASFaHQUhnGAogBlEgSB+PBC+craroJjZpnOlK4+Li5rFhbqoIr+KwH+fjiG4BhKAJRFYAwcENRPXRzdFEBJgHJYzKwSXf7RxpxSvViuFRYcScophuLfVwZu8wOIL0BpBEZn4YBNiYExCUASgEGgGIqQGMLAkLSMvJziDyT56tQDBXHvZUclSaFIsYIbGECgvghCwwHMJgCbRCmxCJRFUjCgYCaOMgEtIipGmJEnSJVEx6cEvhO2LZgHCVJEASyCxGAEoQMU3mQIqie6dfMBJE61RthMwMThwO3bY+NTouME/KgYNjeY5HCg0CD/QDZB4hRJg2E65bIZEv9f1BBg05RNYiwCYbMZEbH7dkbH8sIjWIFBwI8NZWuqOAyEScIYSqPBvgigAwBjGExNCFA6RdLoNCoIi4EyGei7UbFhuyJDQncw/P0o+/8A3meRLxNk0Z4AAAAASUVORK5CYII=","width":50,"height":50,"src":"/static/52659ecfce9ed71e567bddef5d9c1b15/45876/Michal.png","srcSet":"/static/52659ecfce9ed71e567bddef5d9c1b15/45876/Michal.png 1x,\n/static/52659ecfce9ed71e567bddef5d9c1b15/eb85b/Michal.png 1.5x,\n/static/52659ecfce9ed71e567bddef5d9c1b15/4f71c/Michal.png 2x,\n/static/52659ecfce9ed71e567bddef5d9c1b15/9ec3e/Michal.png 3x"}}},"headerImage":null},"html":"Hi everyone! This article is the first in a series of AUTO1 testing-related posts, I hope you’ll find the content interesting enough to return for the rest in the series :) To start off, let’s ask one of the most important questions:
\nWhat is a Test Strategy?
\nIn AUTO1 the test strategy is a high level, short and concise document which defines:
\n- \n
- \n
Scope of the testing
\n \n - \n
Risks - areas that potentially can be broken
\n \n - \n
Automation strategy
\n \n - \n
Required test data
\n \n - \n
Security
\n \n
Why do we create a Test Strategy?
\nIn agile and fast changing environments we usually do not spend that much time on test documentation as in waterfall. However, even in such cases we cannot underestimate the importance of a test strategy. When you are creating a strategy it becomes your guide, a channel for your thoughts - you have a space to stop for a moment to try to understand what should be verified and do it at a very early stage. It is much easier to go back to the feature if you already wrote down your thoughts and you have some kind of summary.
\nHow do we create a Test Strategy?
\nLet’s go through our thinking process, step by step.
\nScope of the testing
\nStart with defining what exactly has to be tested. You can ask yourself a few questions:
\n- \n
- \n
Who is going to use the feature?
\n \n - \n
What apps will be changed/created?
\n \n - \n
On what devices will it be used?
\n \n - \n
What types of testing will be needed?
\n \n
It allows you to better understand the product you are going to test and the environment setup you’ll need.
\nRisks
\nNow think about all the things that can go wrong. Here the questions will be:
\n- \n
- \n
What apps/areas can be impacted by our change?
\n \n - \n
Are we modifying dependencies of another system?
\n \n - \n
Is my domain knowledge enough to understand a feature?
\n \n - \n
Are there any missing prerequisites which can block testing?
\n \n - \n
Are changes going to be compatible with existing solutions?
\n \n
Answering these questions will help us create a plan in order to mitigate all risks. It can be planning an onboarding session with someone who has knowledge we need, investigating regression testing possibilities for apps or areas in danger or setting up a plan on how to get blockers done.
\nAutomation strategy
\nAutomation is an important part of our testing process. We think about the approach on the test strategy stage. We create our tests on an UI and API level. When we’ll have the technical specifications, which include details of implementation, then it is time to plan what and when will be automated. We ask questions such as:
\n- \n
- \n
Where is the specific logic placed? Is it frontend or backend?
\n \n - \n
Do we have existing automation in this area? If yes, do we have to adjust it?
\n \n - \n
Do we have all prerequisites needed to start writing automation?
\n \n
Required test data
\nAt AUTO1 we have a lot of connections between several areas. It created dependencies which have to be covered in test data preparation. We have to be sure we know the preconditions and steps required to arrange the system before starting testing. Here we can specify if we need additional onboarding or support from other teams.
\nHaving answers to the above questions means we are prepared to plan more detailed test cases and we can use our strategy as some kind of a cheatsheet. However, do not blindly follow what you’ve planned - requirements can change. Review your strategy from time to time, to ensure that it is not outdated.
\nSummary
\nTest strategy shouldn’t take much time but it is important for the next step - creating a test plan. It can be a great guide which will help you to structure your initial ideas and be prepared for upcoming changes faster.
","fields":{"slug":"/efficient-testing-1/","tags":["auto1","engineering","testing","QA"]}}}]},"authorArticles":null},"pageContext":{"slug":"/faster-automation-tests/","tags":["testing","automation"],"category":"QA","author":"Alexander Gyulai","date":"2019-01-29"}}