We were working on getting our Robot Framework tests running during our builds on Azure DevOps. However, using hosted agents, it was quite a struggle to get all of the dependencies on the agent and make sure the tests can access the browser on the build machine.
In our case, we were running the tests in Chrome. For this, we needed Robot Framework and some of its dependencies to be installed on the machine. Luckily, that wasn’t too complicated – however, getting the WebDriver that Selenium needs to access Chrome turned out to be tougher!
Basically, you need Python, Robot Framework with its dependencies (these are easy to install with pip), Chrome and a compatible WebDriver on your machine. The emphasis is on “compatible” here, folks. Let’s see about your options, here:
How to figure out the right version of the WebDriver?
Microsoft updates the images at their leisure, and updates the documentation only if the moon is in the right position. What this kind of means is that you shouldn’t trust whatever information you find online about any versions, and just automate what you can.
Now, I’ll show the best option first, and your possible fallbacks afterwards:
Use the preinstalled WebDriver, if it’s available
At least some images Microsoft offers have the WebDriver preinstalled. Chrome version might be whatever, but the WebDriver should be compatible.
Funnily enough, while Microsoft might install the WebDriver, they leave out the crucial step: adding it to PATH environment variable. That means that your Robot Framework won’t get access to the WebDriver – you’d need to either hardcode the path to your scripts (and nobody wants to do that!) or use the environment variable Microsoft offers for this – yes, they do offer a variable with the location of the WebDriver, but they do not add it to the PATH variable!
Anyway, this variable is usually called “CHROMEWEBDRIVER”. Now, in case you don’t want to strap that into your Robot Framework tests (and you shouldn’t), you can modify your build pipeline to add the value of the variable to PATH instead :)
If this works for you, it’s the superior option.
How to add the preinstalled WebDriver’s path to PATH environment variable:
pool: vmImage: 'windows-2019' steps: - pwsh: | echo "##vso[task.prependpath]$env:CHROMEWEBDRIVER" displayName: Adds ChromeWebDriver for Selenium to PATH - powershell: | Write-Host "Robot Framework tests" robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.' failOnStderr: true displayName: 'Run Robot Framework Tests' continueOnError: false
The tests should go through just fine, since the WebDriver can be found from PATH :) If this worked for you, don’t read further – you’re all set, and you shouldn’t even consider any of the workaround steps below.
Now, in case that didn’t work – here’s some further investigation:
Install the WebDriver yourself
Augh – installing WebDrivers sucks, but luckily there are tools to help you out – webdrivermanager being one of them (there’s at least 3 of tools with that name, but let’s just roll with it).
Now, this is how you can easily install the WebDriver using WebDriverManager, and it’ll get added to the PATH automagically:
# Using WebDriver Manager to download and fix environment variable: # https://stackoverflow.com/questions/56754918/setting-chrome-driver-executable-path-keeps-causing-my-build-to-fail - script: webdrivermanager chrome --linkpath AUTO
The tricky part is that the version needs to be the correct one. By default, it’ll use the latest one available – but the images can have whichever version of Chrome.
First, let me tell you how it should be done according to the documentation, and then let me tell you how it actually works.
Find out the Chrome version from Microsoft’s documentation
Microsoft has great documentation on the different Microsoft-hosted agents and their installed software. It’s included at the end of this blog post. Below is an example of the documentation for an image:
Of course, please note that the list of software might be outdated, as even though it’s supposed to be automatically updated, I don’t think it works very reliably. People on this issue found that out the hard way, just like I did:
So, the actual solution?
How to find out the Chrome version and use it as the version of your WebDriver?
Relying on the documentation backfired on us quite badly, as we spent an hour or two trying to get the Chrome WebDriver to load and start the robot tests – but it was complaining about the version of Google Chrome not being 75. We were installing version 75 WebDriver like shown above, since that’s what Microsoft’s documentation claimed was installed on the image.
Well, duh, it really wasn’t. But the latest version of the WebDriver (likely for Chome 78) didn’t work either. Then, what to do?
How to find out the right version of Google Chrome?
Using the PowerShell below, you can find the version of Chrome installed on a machine:
((Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo)
Output should look something like this:
This applies to most desktop machines – but you can also use it on the hosted agents, like this:
How to use YAML to output installed Chrome version on the machine:
- powershell: | (Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo displayName: 'We want to know the Chrome version installed on this darned machine'
I even made a nice little script to grab the current Chrome version into a variable, and then use it to download the matching WebDriver. That should work – the WebDriver team says so:
ChromeDriver uses the same version number scheme as Chrome. See https://www.chromium.org/developers/version-numbers for more details.
– – We always provide ChromeDriver for the current Stable and Beta versions of Chrome. However, if you use Chrome from Dev or Canary channel, or build your own custom version of Chrome, It is possible that there is no available ChromeDriver that officially supports it. In this case, please try the following:https://sites.google.com/a/chromium.org/chromedriver/downloads/version-selection
- powershell: | $ver = ((Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo).ProductVersion echo "##vso[task.setvariable variable=ChromeVersion;isOutput=true]$ver" name: GetChromeVersion displayName: 'We want to know the Chrome version installed on this darned machine' - script: webdrivermanager chrome:$(GetChromeVersion.ChromeVersion) --linkpath AUTO displayName: 'Manage Chrome webdriver version'
However, the patch numbers don’t necessarily match – and the WebDriverManager -libraries I’ve found so far don’t have the necessary logic to download the correct version of the WebDriver.
So the script above doesn’t work for all cases. And since we need something that actually DOES work, here’s a simple way to set the variable for the build pipeline itself, and just update it if a new major version gets deployed to the agent again (even the major version being wrong in Microsoft’s documentation was a fun surprise…)
How to set a variable for a version of Chrome that’s supported by a WebDriver and use that for your tests:
- powershell: | (Get-Item "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe").VersionInfo displayName: 'We want to know the Chrome version installed on this darned machine' - script: webdrivermanager chrome:$(ChromeVersion) --linkpath AUTO displayName: 'Manage Chrome webdriver version' - powershell: | Write-Host "Robot Framework tests" robot -x 'outputxunit.xml' '$(Build.SourcesDirectory)\tests\.' failOnStderr: true displayName: 'Run Robot Framework Tests' continueOnError: true
And the variable is set as so:
You need to use a Chrome version that has a WebDriver released for it – this website has a list for possible values: https://chromedriver.chromium.org/downloads
With that workaround, you should be good, until Microsoft updates the images. And at that point, you’ll see the new version of Chrome in the build output for the task we left at the top for outputting the Chrome version with PowerShell! :)
Table of agent images hosted by Microsoft
|Image||Classic Editor Agent Specification||YAML VM Image Label||Included Software|
|Windows Server 2019 with Visual Studio 2019||windows-2019||Link|
|Windows Server 2016 with Visual Studio 2017||vs2017-win2016||Link|
|Windows Server 2012 R2 with Visual Studio 2015||vs2015-win2012r2||Link|
|Windows Server Core 1803 (for running Windows containers)||win1803||Link|
|macOS X Mojave 10.14||macOS-10.14||Link|
|macOS X High Sierra 10.13||macOS-10.13||Link|
You can simply find the right image from the table above, click the link to the list of installed software, and use that info in your scripts. I mean, if it’s updated. It probably isn’t :)
- How to modify PATH variable:
- How to use variables in Azure DevOps:
- How to output the correct kind of an echo statement to force Azure DevOps to run the snippets:
He's been a developer from 2004 (starting with PHP and Java), and he's been bending and twisting SharePoint into different shapes since MOSS. Nowadays he's not only working on SharePoint, but also on .NET projects, Azure, Office 365 and a lot of other stuff.
This is his personal professional (e.g. professional, but definitely personal) blog.
Latest posts by Antti K. Koskela (see all)
- How to instantiate your DbContext outside your Data project? - November 11, 2019
- Another year, another Hacktoberfest (2019)! - November 1, 2019
- How to resolve “refusing to allow an integration to create or update .github/workflows/main.yml” on GitHub Desktop? - October 29, 2019