How to run Robot tests on an Azure DevOps hosted agent?

This short article simply documents how to install the dependencies for running Robot Framework on Azure DevOps' hosted agents.

This simple tip comes in handy when you need to run any UI testing for your project after build, as Robot Framework is a great and widely used tool for that. Let's quickly take a look at the background of the issue, after which I'm sharing a highly copy-pasteable piece of YAML for your convenience. Well - and mine.

Background

So I got a question about this on another article's comments section, and figured it might be worthwhile to document this somewhere!

Essentially, you'll need to be able to download your test definitions to the agent and then execute them using the same framework or tools your tests are written for.

Problem

The hosted agents are highly standardized images with a limited amount of software preinstalled. While Selenium WebDrivers ARE in fact typically preinstalled on at least some of the images (see the post below for more info about the software installed on the images), Robot Framework is not. And that makes sense, as it's not a good idea to bloat the images with all the bells and whistles.

https://www.koskila.net/how-to-use-the-right-version-of-the-webdriver-on-hosted-agents-in-azure-devops/

Anyway, back to the task. How to actually run the Robot tests on the machine?

Solution

We'll need the packages for Robot Framework and some other dependencies. That's all Python (at least in our configuration).

Luckily, Python is preinstalled, and we can use pip - Package Installer for Python - to install whatever Python packages we need!

So, long story short, this YAML should do the trick:

trigger:
- master

pool:
  vmImage: 'windows-2019'

strategy:
  matrix:
    Python37:
      python.version: '3.7'
  maxParallel: 1

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:

- task: UsePythonVersion@0
  inputs: 
    versionSpec: '$(python.version)'
    architecture: 'x64'

- script: pip install robotframework robotframework-pabot robotframework-seleniumlibrary webdrivermanager
  displayName: 'Install dependencies'

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' -t 'Your Test' '$(Build.SourcesDirectory)\[YourProjectName]\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: true

- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  continueOnError: true
  inputs:
    testResultsFiles: outputxunit.xml

With this, you should be good to go!