How to run Robot tests on an Azure DevOps hosted agent?

This short article simply documents how to install the dependencies for running Robot Framework on Azure DevOps’ hosted agents.

This simple tip comes in handy when you need to run any UI testing for your project after build, as Robot Framework is a great and widely used tool for that. Let’s quickly take a look at the background of the issue, after which I’m sharing a highly copy-pasteable piece of YAML for your convenience. Well – and mine.

Background

So I got a question about this on another article’s comments -section, and figured it might be worthwhile to document even this somewhere!

Essentially, you’ll need to be able to download your test definitions to the agent and then execute them using the same framework or tools your tests are written for. In my case, said framework would be Robot.

Problem

The hosted agents are highly standardized images with a limited amount of software preinstalled. While Selenium WebDrivers ARE in fact typically preinstalled on at least some of the images (see the post below for more info about the software installed on the images), Robot Framework is not. And that makes sense, as it’s not a good idea to bloat the images with all the bells and whistles.

Anyway, back to the task. How to actually run the Robot tests on the machine?

Solution

We’ll need the packages for Robot Framework and some other dependencies. That’s all Python (at least in our configuration).

Luckily, Python is preinstalled, and we can use pip – Package Installer for Python – to install whatever Python packages we need!

So, long story short, this YAML should do the trick:

trigger:
- master

pool:
  vmImage: 'windows-2019'

strategy:
  matrix:
    Python37:
      python.version: '3.7'
  maxParallel: 1

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:

- task: [email protected]
  inputs: 
    versionSpec: '$(python.version)'
    architecture: 'x64'

- script: pip install robotframework robotframework-pabot robotframework-seleniumlibrary webdrivermanager
  displayName: 'Install dependencies'

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' -t 'Your Test' '$(Build.SourcesDirectory)\[YourProjectName]\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: true

- task: [email protected]
  displayName: 'Publish Test Results'
  continueOnError: true
  inputs:
    testResultsFiles: outputxunit.xml

With this, you should be good to go!

• Author
• Recent Posts

Antti K. Koskela

Developer / Speaker / Consultant at Koskila / Etteplan Oyj / Norppandalotti Software Co / Alter - Experience Ideas Ltd.

Antti Koskela is a proud digital native nomadic millennial full stack developer (is that enough funny buzzwords? That's definitely enough funny buzzwords!), who works as a Cloud Solutions Architect for Etteplan Oyj, an engineering company that employs something like 700 devs building and fixing anything even half-digital.

He's been a developer from 2004 (starting with PHP and Java), and he's been working on .NET projects, Azure, Office 365, SharePoint and a lot of other stuff. He's also Microsoft MVP for Office Development.

This is his personal professional (e.g. professional, but definitely personal) blog.

Latest posts by Antti K. Koskela (see all)

• How to fix a Logic App deployment that failed with error “ApiNotFound”? - April 22, 2021
• Experiment: DateTime formats - April 21, 2021
• How to fix “LinkedAuthorizationFailed” when deploying an Azure Logic App? - April 15, 2021