Always ship Azure DevOps.

How to use the right version of the WebDriver on hosted agents in Azure DevOps?

This post was most recently updated on December 10th, 2019.

Reading Time: 7 minutes.

We were working on getting our Robot Framework tests running during our builds on Azure DevOps. However, using hosted agents, it was quite a struggle to get all of the dependencies on the agent and make sure the tests can access the browser on the build machine.

In our case, we were running the tests in Chrome. For this, we needed Robot Framework and some of its dependencies to be installed on the machine. Luckily, that wasn’t too complicated – however, getting the WebDriver that Selenium needs to access Chrome turned out to be tougher!

Basically, you need Python, Robot Framework with its dependencies (these are easy to install with pip), Chrome and a compatible WebDriver on your machine. The emphasis is on “compatible” here, folks. Let’s see about your options, here:

How to figure out the right version of the WebDriver?

Microsoft updates the images at their leisure, and updates the documentation only if the moon is in the right position. What this kind of means is that you shouldn’t trust whatever information you find online about any versions, and just automate what you can.

Now, I’ll show the best option first, and your possible fallbacks afterwards:

Use the preinstalled WebDriver, if it’s available

At least some images Microsoft offers have the WebDriver preinstalled. Chrome version might be whatever, but the WebDriver should be compatible.

Funnily enough, while Microsoft might install the WebDriver, they leave out the crucial step: adding it to PATH environment variable. That means that your Robot Framework won’t get access to the WebDriver – you’d need to either hardcode the path to your scripts (and nobody wants to do that!) or use the environment variable Microsoft offers for this – yes, they do offer a variable with the location of the WebDriver, but they do not add it to the PATH variable!

Anyway, this variable is usually called “CHROMEWEBDRIVER”. Now, in case you don’t want to strap that into your Robot Framework tests (and you shouldn’t), you can modify your build pipeline to add the value of the variable to PATH instead :)

If this works for you, it’s the superior option.

How to add the preinstalled WebDriver’s path to PATH environment variable:

pool:
  vmImage: 'windows-2019'

steps:

- pwsh: |
   echo "##vso[task.prependpath]$env:CHROMEWEBDRIVER"
  displayName: Adds ChromeWebDriver for Selenium to PATH 

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: false

The tests should go through just fine, since the WebDriver can be found from PATH :) If this worked for you, don’t read further – you’re all set, and you shouldn’t even consider any of the workaround steps below.

Now, in case that didn’t work – here’s some further investigation:

Install the WebDriver yourself

Augh – installing WebDrivers sucks, but luckily there are tools to help you out – webdrivermanager being one of them (there’s at least 3 of tools with that name, but let’s just roll with it).

Now, this is how you can easily install the WebDriver using WebDriverManager, and it’ll get added to the PATH automagically:

# Using WebDriver Manager to download and fix environment variable:
# https://stackoverflow.com/questions/56754918/setting-chrome-driver-executable-path-keeps-causing-my-build-to-fail
- script: webdrivermanager chrome --linkpath AUTO

The tricky part is that the version needs to be the correct one. By default, it’ll use the latest one available – but the images can have whichever version of Chrome.

First, let me tell you how it should be done according to the documentation, and then let me tell you how it actually works.

Find out the Chrome version from Microsoft’s documentation

Microsoft has great documentation on the different Microsoft-hosted agents and their installed software. It’s included at the end of this blog post. Below is an example of the documentation for an image:

Microsoft's documentation detailing the version of Google Chrome installed on a hosted build agent
Microsoft’s documentation detailing the version of Google Chrome installed on a hosted build agent

Of course, please note that the list of software might be outdated, as even though it’s supposed to be automatically updated, I don’t think it works very reliably. People on this issue found that out the hard way, just like I did:

https://github.com/microsoft/azure-pipelines-image-generation/issues/1340

So, the actual solution?

How to find out the Chrome version and use it as the version of your WebDriver?

Relying on the documentation backfired on us quite badly, as we spent a few hours trying to get the Chrome WebDriver to load and start the robot tests – but it was complaining about the version of Google Chrome not being 75. We were installing version 75 WebDriver like shown above, since that’s what Microsoft’s documentation claimed was installed on the image.

Well, duh, it really wasn’t. But the latest version of the WebDriver (likely for Chome 78) didn’t work either. Then, what to do?

How to find out the right version of Google Chrome?

Using the PowerShell below, you can find the version of Chrome installed on a machine (like your dev box):

((Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo)

Output should look something like this:

The PowerShell one can use to find out the Chrome version on a machine.
The PowerShell one can use to find out the Chrome version on a machine.

This applies to most desktop machines – but you can also use it on the hosted agents in your YAML, like this:

How to use YAML to output installed Chrome version on the machine:
- powershell: |
   (Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo 
  displayName: 'We want to know the Chrome version installed on this darned machine'

I even made a nice little script to grab the current Chrome version into a variable, and then use it to download the matching WebDriver. That should work – the WebDriver team says so:

ChromeDriver uses the same version number scheme as Chrome. See https://www.chromium.org/developers/version-numbers for more details.

– – We always provide ChromeDriver for the current Stable and Beta versions of Chrome. However, if you use Chrome from Dev or Canary channel, or build your own custom version of Chrome, It is possible that there is no available ChromeDriver that officially supports it. In this case, please try the following:

https://sites.google.com/a/chromium.org/chromedriver/downloads/version-selection
- powershell: |
   $ver = ((Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo).ProductVersion
   echo "##vso[task.setvariable variable=ChromeVersion;isOutput=true]$ver"
  name: GetChromeVersion
  displayName: 'We want to know the Chrome version installed on this darned machine'

- script: webdrivermanager chrome:$(GetChromeVersion.ChromeVersion) --linkpath AUTO
  displayName: 'Manage Chrome webdriver version'

However, the patch numbers don’t necessarily match – and the WebDriverManager -libraries I’ve found so far don’t have the necessary logic to download the correct version of the WebDriver.

So the script above doesn’t work for all cases. And since we need something that actually DOES work, here’s a simple way to set the variable for the build pipeline itself, and just update it if a new major version gets deployed to the agent again (even the major version being wrong in Microsoft’s documentation was a fun surprise…)

How to set a variable for a version of Chrome that’s supported by a WebDriver and use that for your tests:

- powershell: |
   (Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo 
  displayName: 'We want to know the Chrome version installed on this darned machine'

- script: pip install robotframework robotframework-pabot robotframework-seleniumlibrary webdrivermanager
  displayName: 'Install Python dependencies'

- script: webdrivermanager chrome:$(ChromeVersion) --linkpath AUTO
  displayName: 'Manage Chrome webdriver version'

- powershell: |
   Write-Host "Robot Framework tests"
   robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.'
  failOnStderr: true
  displayName: 'Run Robot Framework Tests'
  continueOnError: true

And the variable is set as so:

Azure DevOps Build Pipeline Variable configuration
Azure DevOps Build Pipeline Variable configuration

You need to use a Chrome version that has a WebDriver released for it – this website has a list for possible values: https://chromedriver.chromium.org/downloads


With that workaround, you should be good, until Microsoft updates the images. And at that point, you’ll see the new version of Chrome in the build output for the task we left at the top for outputting the Chrome version with PowerShell! :)

Table of agent images hosted by Microsoft

ImageClassic Editor Agent SpecificationYAML VM Image LabelIncluded Software
Windows Server 2019 with Visual Studio 2019windows-2019windows-latest OR windows-2019Link
Windows Server 2016 with Visual Studio 2017vs2017-win2016vs2017-win2016Link
Windows Server 2012 R2 with Visual Studio 2015vs2015-win2012r2vs2015-win2012r2Link
Windows Server Core 1803 (for running Windows containers)win1803win1803Link
Ubuntu 18.04ubuntu-18.04ubuntu-latest OR ubuntu-18.04Link
Ubuntu 16.04ubuntu-16.04ubuntu-16.04Link
macOS X Mojave 10.14macOS-10.14macOS-latest OR macOS-10.14Link
macOS X High Sierra 10.13macOS-10.13macOS-10.13Link

You can simply find the right image from the table above, click the link to the list of installed software, and use that info in your scripts. I mean, if it’s updated. It probably isn’t :)

References

Antti K. Koskela

Antti Koskela is a proud digital native nomadic millennial full stack developer (is that enough funny buzzwords? That's definitely enough funny buzzwords!), who works as a Solutions Architect for Valo Intranet, the product that will make you fall in love with your intranet.

He's been a developer from 2004 (starting with PHP and Java), and he's been bending and twisting SharePoint into different shapes since MOSS. Nowadays he's not only working on SharePoint, but also on .NET projects, Azure, Office 365 and a lot of other stuff. He's also Microsoft MVP for Office Development.

This is his personal professional (e.g. professional, but definitely personal) blog.
mm

4
Leave a Reply

avatar
5000
2 Comment threads
2 Thread replies
3 Followers
 
Most reacted comment
Hottest comment thread
3 Comment authors
esaruohommSimonWCC Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
SimonWCC
Guest
SimonWCC

Thanks, you save my day~

esaruoho
Guest

Thanks for creating this post. I added the bits of code you mentioned at the start of the article, to a basic Azure Pipeline YAML file. I ran it, and got

robot : The term ‘robot’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At D:\a\_temp\5fbce742-ff63-4e76-b91d-ee238386d389.ps1:3 char:1
+ robot -x ‘outputxunit.xml’ ‘d:\a\1\s\tests\.’
+ ~~~~~
+ CategoryInfo : ObjectNotFound: (robot:String) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : CommandNotFoundException
PowerShell exited with code ‘1’.

so I might have to figure out how to add robot into the windows-2019 vmware image.