Test Automation Stack

Cypress is a powerful Automation tool using which we can test our Application by loading it in our choice of browser.

With cypress its easy to automate hundreds and thousands of Tests, and as the tests increases its execution time also increases. To get the test result sooner we can use Cypress Cloud to run the tests in parallel and Cypress Cloud will intelligently distribute these tests across multiple agents.

In this article we will be using Azure Devops for our Tests execution in the CI/CD Pipeline.

The build stages are like below

1. Build React Application
2. Run Component Tests
3. Run End to End Tests

Now let’s quickly setup a pipeline to achieve this in Azure Devops

Pre-requisite: Azure Account and Code repo pushed to Azure Devops

To setup pipeline

1. We need to Navigate to the Project in Azure and Click on Pipelines > New pipeline

2. Select the source code repo (Which is Azure Repo in our case)

3. Select the project for which you want to setup pipeline

4. Select the type of project for which the pipeline is being setup, Since we are going to write the pipeline and don’t want pre-defined, we can Select Starter pipeline.

5. Which will create a yml file like below

6. Delete the script and replace that with below script

For this article, lets focus on the Test stage, where we run our end-to-end tests and in Parallel.

pool:
vmImage: ubuntu-latest

variables:
YARN_CACHE_FOLDER: $(Pipeline.Workspace)/.yarn

resources:
repositories:

• repository: templates
  name: Azure-Demo/pipeline-template
  type: git
  ref: main

stages:

• stage: Build
  displayName: Build React App
  jobs:
• template: baseBuildTemplate.yml@templates
  parameters:
  buildType: “test”
• stage: ComponentTest
  displayName: Component Test
  jobs:
• job: ComponentTestJob
  variables:
  NPM_CACHE_FOLDER: $(Pipeline.Workspace)/.yarn steps:
  • task: Cache@2
    displayName: Cache npm dependencies
    inputs:
    key: ‘yarn | “$(Agent.OS)” | yarn.lock’
    restoreKeys: |
    yarn | “$(Agent.OS)”
    yarn
    path: $(YARN_CACHE_FOLDER)
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘Yarn Install’
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)/FunctionalTests’
    arguments: install
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘run component test ‘
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)/FunctionalTests’
    arguments: component:test
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘generate Report ‘
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)/FunctionalTests’
    arguments: combine:reports
  • task: PublishTestResults@2
    displayName: ‘Publish Test Results’
    inputs:
    testResultsFiles: ‘**/combined-report.xml’
    searchFolder: ‘$(System.DefaultWorkingDirectory)/FunctionalTests/’
• stage: Test
  displayName: Run Automated Tests
  jobs:
• job: RunAppAndTest
  strategy:
  parallel: 2
  steps:
  • task: Cache@2
    inputs:
    key: ‘”yarn” | “$(Agent.OS)” | yarn.lock’
    restoreKeys: |
    yarn | “$(Agent.OS)”
    yarn
    path: $(YARN_CACHE_FOLDER)
    displayName: Cache Yarn packages
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘install dep’
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)/FunctionalTests’
    arguments: install –cache-folder $(YARN_CACHE_FOLDER)
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘install dep’
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)’
    arguments: install –cache-folder $(YARN_CACHE_FOLDER)
  • task: Bash@3
    displayName: ‘Update Yarn version to 2.x’
    inputs:
    targetType: inline
    script: |
    yarn run start&
  • task: geeklearningio.gl-vsts-tasks-yarn.yarn-task.Yarn@3
    displayName: ‘run cypress test’
    inputs:
    projectDirectory: ‘$(System.DefaultWorkingDirectory)/FunctionalTests’
    arguments: run test
  • task: PublishBuildArtifacts@1
    inputs:
    pathToPublish: ‘$(System.DefaultWorkingDirectory)/FunctionalTests/htmlReport’
    artifactName: HtmlReport
  • task: LakshayKaushik.PublishHTMLReports.publishhtmlreport.publishhtmlreport@1
    displayName: publishhtmlreport
    inputs:
    JmeterReportsPath: ‘$(System.DefaultWorkingDirectory)/FunctionalTests/htmlReport’
  • task: JakubRumpca.azure-pipelines-html-report.PublishHtmlReport.PublishHtmlReport@1
    displayName: ‘Publish Html Report’
    inputs:
    reportDir: ‘$(System.DefaultWorkingDirectory)/FunctionalTests/htmlReport/index.html’

If we break down the above code

First we define the Stage called Test, which contains Jobs and Job in-turn contains multiple tasks.

In the above Job we define the strategy as parallel with parallel run count(2) which will run this job and its tasks in 2 agents in parallel

– job: RunAppAndTest
 strategy:
 parallel: 2
 steps:

Now when we Save this and Run the pipeline, we see the pipeline like below and its stages. The Last stage is our interest, where the end-to-end tests run in parallel(Across multiple agents)

Now to distribute the tests across these agents while running in parallel, we need to connect to cypress cloud(Create the project in cypress cloud and use the project id and key in the cypress project where our tests resides)

Below is the cypress command which uses record key and parallel flag to run the tests in parallel, distributing tests across the Azure build agents

yarn cypress run — record — key 4002-ae36–34c515 — parallel — ci-build-id $BUILD_BUILDNUMBER — group ‘Azure CI’

Note that in the above command we have to send the CI build number to Cypress cloud in order to run the tests in Parallel. Hence we for azure we have used $BUILD_BUILDNUMBER in the run command. Which send the build number of the pipeline run to Cypress cloud.

To check the build number sent from pipeline, we can go to Cypress Cloud > Project, click on three dots of the latest run and we can the build id under CI BUILD ID

This is good and works fine when we don’t have tests failing and want to rerun the failed job.

To test this, lets click on the Rerun failed jobs

The run starts but tests will not be executed. This is because the build number is still same and Cypress has this build number already exists as part of the previous run.

In order to resolve this, we need to send the Attempt Number along with Build number to Cypress Cloud.

yarn cypress run — record — key cba34fb–388e4404c515 — parallel — ci-build-id $BUILD_BUILDNUMBER+$SYSTEM_JOBATTEMPT — group ‘Azure CI’

Now when we Rerun Failed Jobs, Cypress Cloud gets Attempt Number along with Build Number and tests will be executed

Hope this article helped you and if it did, do not forget to share and 👏