How to use the right version of the WebDriver on hosted agents in Azure DevOps?

We were working on getting our Robot Framework tests running during our builds on Azure DevOps. However, using hosted agents, it was quite a struggle to get all of the dependencies on the agent and make sure the tests could access the browser on the build machine.

In our case, we were running the tests in Chrome. For this, we needed Robot Framework and some of its dependencies to be installed on the machine. Luckily, that wasn't too complicated - however, getting the WebDriver that Selenium needs to access Chrome turned out to be tougher!

Basically, you need Python, Robot Framework with its dependencies (these are easy to install with pip), Chrome and a compatible WebDriver on your machine. The emphasis is on "compatible" here, folks. Let's see about your options, here:

How to figure out the right version of the WebDriver?

Microsoft updates the images at their leisure and updates the documentation only if the moon is in the right position. This means that you shouldn't trust whatever information you find online about any versions, and just automate what you can.

Now, I'll show the best option first, and your possible fallbacks afterwards:

Use the preinstalled WebDriver, if it's available

At least some images Microsoft offers have the WebDriver preinstalled. The chrome version might be whatever, but the WebDriver should be compatible.

Funnily enough, while Microsoft might install the WebDriver, they leave out the crucial step: adding it to PATH environment variable. That means that your Robot Framework won't get access to the WebDriver - you'd need to either hardcode the path to your scripts (and nobody wants to do that!) or use the environment variable Microsoft offers for this - yes, they do offer a variable with the location of the WebDriver, but they do not add it to the PATH variable!

Anyway, this variable is usually called "CHROMEWEBDRIVER". Now, in case you don't want to strap that into your Robot Framework tests (and you shouldn't), you can modify your build pipeline to add the value of the variable to PATH instead :)

If this works for you, it's the superior option.

How to add the preinstalled WebDriver's path to PATH environment variable:

pool:
  vmImage: 'windows-2019'

steps:

- pwsh: |
   echo "##vso[task.prependpath]$env:CHROMEWEBDRIVER"
  displayName: Adds ChromeWebDriver for Selenium to PATH 

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: false

The tests should go through just fine since the WebDriver can be found from PATH :) If this worked for you, don't read further - you're all set, and you shouldn't even consider any of the workaround steps below.

Now, in case that didn't work - here's some further investigation:

Install the WebDriver yourself

Augh - installing WebDrivers sucks, but luckily there are tools to help you out - webdrivermanager being one of them (there are at least 3 tools with that name, but let's just roll with it).

Now, this is how you can easily install the WebDriver using WebDriverManager, and it'll get added to the PATH automagically:

# Using WebDriver Manager to download and fix environment variable:
# https://stackoverflow.com/questions/56754918/setting-chrome-driver-executable-path-keeps-causing-my-build-to-fail
- script: webdrivermanager chrome --linkpath AUTO

The tricky part is that the version needs to be the correct one. By default, it'll use the latest one available - but the images can have whichever version of Chrome.

First, let me tell you how it should be done according to the documentation, and then let me tell you how it actually works.

Find out the Chrome version from Microsoft's documentation

Microsoft has great documentation on the different Microsoft-hosted agents and their installed software. It's included at the end of this blog post. Below is an example of the documentation for an image:

Of course, please note that the list of software might be outdated, as even though it's supposed to be automatically updated, I don't think it works very reliably. People on this issue found that out the hard way, just like I did:

https://github.com/microsoft/azure-pipelines-image-generation/issues/1340

So, the actual solution?

How to find out the Chrome version and use it as the version of your WebDriver?

Relying on the documentation backfired on us quite badly, as we spent a few hours trying to get the Chrome WebDriver to load and start the robot tests - but it was complaining about the version of Google Chrome not being 75. We were installing version 75 WebDriver as shown above since that's what Microsoft's documentation claimed was installed on the image.

Well, duh, it really wasn't. But the latest version of the WebDriver (likely for Chome 78) didn't work either. Then, what to do?

How to find out the right version of Google Chrome?

1. Using the PowerShell below, you can find the version of Chrome installed on a machine (like your dev box):

   ((Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo)

   The output should look something like this:

   
   The PowerShell one can use to find out the Chrome version on a machine.

   This applies to most desktop machines - but you can also use it on the hosted agents in your YAML, like this:

2. Use YAML to output the installed Chrome version on the machine:

   - powershell: |    (Get-Item "C:\Program Files\Google\Chrome\Application\chrome.exe").VersionInfo   displayName: 'We want to know the Chrome version installed on this darned machine'

3. Depending on the image you're using, you might need to change "Program Files" to "Program Files (x86)".

   I even made a nice little script to grab the current Chrome version into a variable, and then use it to download the matching WebDriver. That should work - the WebDriver team says so:

   ChromeDriver uses the same version number scheme as Chrome. See https://www.chromium.org/developers/version-numbers/ for more details.

   - - We always provide ChromeDriver for the current Stable and Beta versions of Chrome. However, if you use Chrome from Dev or Canary channel, or build your own custom version of Chrome, It is possible that there is no available ChromeDriver that officially supports it. In this case, please try the following: https://sites.google.com/a/chromium.org/chromedriver/downloads/version-selection

   - powershell: |    $ver = ((Get-Item "C:\Program Files\Google\Chrome\Application\chrome.exe").VersionInfo).ProductVersion    echo "##vso[task.setvariable variable=ChromeVersion;isOutput=true]$ver"   name: GetChromeVersion   displayName: 'We want to know the Chrome version installed on this darned machine'   - script: webdrivermanager chrome:$(GetChromeVersion.ChromeVersion) --linkpath AUTO   displayName: 'Manage Chrome webdriver version'

   However, the patch numbers don't necessarily match - and the WebDriverManager -libraries I've found so far don't have the necessary logic to download the correct version of the WebDriver.

   So the script above doesn't work for all cases. And since we need something that actually DOES work, here's a simple way to set the variable for the build pipeline itself, and just update it if a new major version gets deployed to the agent again (even if the major version is wrong in Microsoft's documentation was a fun surprise...)

4. set a variable for a version of Chrome that's supported by a WebDriver and use that for your tests

   - powershell: |    (Get-Item "C:\Program Files\Google\Chrome\Application\chrome.exe").VersionInfo   displayName: 'We want to know the Chrome version installed on this darned machine'   - script: pip install robotframework robotframework-pabot robotframework-seleniumlibrary webdrivermanager   displayName: 'Install Python dependencies'   - script: webdrivermanager chrome:$(ChromeVersion) --linkpath AUTO   displayName: 'Manage Chrome webdriver version'   - powershell: |    Write-Host "Robot Framework tests"    robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.'   failOnStderr: true   displayName: 'Run Robot Framework Tests'   continueOnError: true

   And the variable is set as so:

   
   Azure DevOps Build Pipeline Variable configuration

   You need to use a Chrome version that has a WebDriver released for it. This website has a list of possible values: https://chromedriver.chromium.org/downloads

With that workaround, you should be good, until Microsoft updates the images. And at that point, you'll see the new version of Chrome in the build output for the task we left at the top for outputting the Chrome version with PowerShell! :)

Table of agent images hosted by Microsoft

The list has been last updated on 26.1.2020. If you want to check out the latest version, you can do that on docs.microsoft.com.

You can simply find the right image from the table above, click the link to the list of installed software, and use that info in your scripts. I mean, if it's updated. Don't rely on that too much!

References

• How to modify PATH variable:

  • https://docs.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash

  • https://stackoverflow.com/questions/52631143/cannot-change-the-path-in-a-script-task-in-azure-pipelines

• How to use variables in Azure DevOps:

  • https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-in-script

• How to output the correct kind of echo statement to force Azure DevOps to run the snippets:

  • https://stackoverflow.com/questions/42861916/powershell-echo-statement-variable-in-one-line/42862164