How to run Robot tests on an Azure DevOps hosted agent?

This post was most recently updated on September 19th, 2021.

2 min read.

This short article simply documents how to install the dependencies for running Robot Framework on Azure DevOps’ hosted agents.

This simple tip comes in handy when you need to run any UI testing for your project after build, as Robot Framework is a great and widely used tool for that. Let’s quickly take a look at the background of the issue, after which I’m sharing a highly copy-pasteable piece of YAML for your convenience. Well – and mine.

Background

Table of Contents

So I got a question about this on another article’s comments section, and figured it might be worthwhile to document this somewhere!

Essentially, you’ll need to be able to download your test definitions to the agent and then execute them using the same framework or tools your tests are written for.

Problem

The hosted agents are highly standardized images with a limited amount of software preinstalled. While Selenium WebDrivers ARE in fact typically preinstalled on at least some of the images (see the post below for more info about the software installed on the images), Robot Framework is not. And that makes sense, as it’s not a good idea to bloat the images with all the bells and whistles.

How to use the right version of the WebDriver on hosted agents in Azure DevOps?

Anyway, back to the task. How to actually run the Robot tests on the machine?

Solution

We’ll need the packages for Robot Framework and some other dependencies. That’s all Python (at least in our configuration).

Luckily, Python is preinstalled, and we can use pip – Package Installer for Python – to install whatever Python packages we need!

So, long story short, this YAML should do the trick:

trigger:
- master

pool:
  vmImage: 'windows-2019'

strategy:
  matrix:
    Python37:
      python.version: '3.7'
  maxParallel: 1

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:

- task: [email protected]
  inputs: 
    versionSpec: '$(python.version)'
    architecture: 'x64'

- script: pip install robotframework robotframework-pabot robotframework-seleniumlibrary webdrivermanager
  displayName: 'Install dependencies'

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' -t 'Your Test' '$(Build.SourcesDirectory)\[YourProjectName]\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: true

- task: [email protected]
  displayName: 'Publish Test Results'
  continueOnError: true
  inputs:
    testResultsFiles: outputxunit.xml

With this, you should be good to go!

Author

Recent Posts

Antti K. Koskela

Developer / Speaker / Consultant at Koskila / Precio Fishbone / Norppandalotti Software Co / Alter - Experience Ideas Ltd

Antti Koskela is a proud digital native nomadic millennial full stack developer (is that enough funny buzzwords? That's definitely enough funny buzzwords!), who works as Solutions Architect for Precio Fishbone, building delightful Digital Workplaces.

He's been a developer from 2004 (starting with PHP and Java), and he's been working on .NET projects, Azure, Office 365, SharePoint and a lot of other stuff. He's also Microsoft MVP for Office Development.

This is his personal professional (e.g. professional, but definitely personal) blog.

Latest posts by Antti K. Koskela (see all)