bash - it's even more of a mystery than PowerShell

Azure DevOps fails with “Error: The path ‘/home/vsts/work/1/s/your-repo’ does not exist” even though the repository is there for sure?

This post was most recently updated on November 28th, 2022.

4 min read.

Eh, this was a weird one. This article explains how to get around an issue where your build tools fail due to weird “path does not exist” errors when you’re using bash to execute a restore to your project. The build would probably succeed nicely, but it fails due to missing dependencies – in my case, at least the whole node_modules folder was missing. And of course, it is – the restore failed, after all.

But let’s take one step at a time. What did I run into, this time?

Background

I do pretty much all of my dev stuff on a Windows machine. But for performance reasons, pretty much every single one of my Azure DevOps Build / Release pipelines uses Ubuntu agents. And these 2 environments treat file paths quite differently.

Most of the time, this doesn’t cause issues. But occasionally you run into weirdness.

Problem

Windows and Unix-based systems handle file paths quite differently.

In this particular case, my build step was trying to restore a project, and I had a path to a .csproj file passed to the step as a parameter to it.

The steps are going to be pretty much the same even if you just need to grab the path from somewhere else – just adjust the steps accordingly.

We had this in our pipeline:

- task: Bash@3
  displayName: Build
  inputs:
    targetType: inline
    script: >-
      cd $(dirname "${{ parameters.projPath }}")

      $HOME/.dotnet/tools/our-custom-build-tool restore 

      $HOME/.dotnet/tools/our-custom-build-tool build

A similar pipeline – actually, in some cases the exact same step, job, and stage templates – worked for another project. But for this particular one, they failed.

Errors thrown

Okay – the output below is going to be a bit confusing, but it’s a sample of errors that could be caused the earlier issue with restoring npm packages due to bash not understanding our file paths.

  npm ERR! code ELIFECYCLE
  npm ERR! errno 2
  npm ERR! [email protected] after-build: `tsc -v && tsc && gulp after-build "--docs"`
  npm ERR! Exit status 2
  npm ERR! 
  npm ERR! Failed at the [email protected] after-build script.
  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  npm WARN Local package.json exists, but node_modules missing, did you mean to install?
  
  npm ERR! A complete log of this run can be found in:
  npm ERR!     /home/vsts/.npm/_logs/2022-09-28T10_13_13_708Z-debug.log
  npm ERR! code ELIFECYCLE
  npm ERR! errno 2
  npm ERR! [email protected] after-build-release: `npm run after-build -- --docs`
  npm ERR! Exit status 2
  npm ERR! 
  npm ERR! Failed at the [email protected] after-build-release script.
  npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  npm WARN Local package.json exists, but node_modules missing, did you mean to install?
  
  npm ERR! A complete log of this run can be found in:
  npm ERR!     /home/vsts/.npm/_logs/2022-09-28T10_13_13_737Z-debug.log
/home/vsts/work/1/s/our-project.csproj(61,5): error MSB3073: The command "npm run after-build-release" exited with code 2.
Error: dotnet published failed

And before, there was this gem:

Error: The path '/home/vsts/work/1/s/our-repository' does not exist

Which, funnily, was preceded by our (successful) checkout step.

Reason

Eventually, I figured this repository had a naming convention with both uppercase and lowercase letters, whereas most of our repositories were named in all lowercase. And mixed-case naming conventions are no bueno with unix-based build agents!

Weirdly, the project path appeared to be correct. But still – this was the repository where it caused issues.

The checkout step/task doesn’t actually tell you the name of the directory it creates. But it respects mixed-case letters. But dirname apparently lowercased everything.

I couldn’t quickly figure out how smart dirname actually is. Maybe it doesn’t care whether the directory actually exists. Maybe it messes up the casing. I don’t know, and to be fair, I didn’t really care to figure it out.

This pipeline was using bash, and knowing I had this same stuff working in a different pipeline with a PowerShell task, I decided to try with PowerShell instead.

And it worked.

So… Let’s think of this as a neat workaround and not worry too much about how nitpicky some of these tools can be about their path formatting :)

Solution

Okay, so let’s replace bash stuff with PowerShell. Since PowerShell is nowadays pretty much completely cross-platform and seems to work better (although probably not faster) than bash whenever you’re not completely unix-y-ligious

Time needed: 10 minutes

How to solve mixed-case issues in Azure DevOps by switching from bash to PowerShell?

  1. Replace your bash task with PowerShell

    I’m sure someone would pick bash over PowerShell, but I’m not that someone.

  2. Remove any file names from the path

    To successfully change the directory, instead of using bash-y dirname, we’ll use Split-Path, which we have in PowerShell:
    $directory = Split-Path -Path ${{ parameters.projPath }}
    (parameters.projPath is something like the location of your .csproj file, or something else that tells what to restore and where to build).

  3. Change your directory location

    Now that we’ve got the $directory figured out, you can set the location for the rest of your script. Forget about cd, we’ll use Set-Location instead:
    Set-Location $directory

  4. Run the rest of your script

    Now it’s safe to run whatever you were planning to run. In my case restore and build commands for our tooling – and like magic, everything now works.

Hopefully, it goes as well for you as it did for me. Let me know in the comments below!

Script sample

So the end result would look somewhat like this:

- task: PowerShell@2
  displayName: Build
  inputs:
    targetType: inline
    pwsh: true
    script: |
      $directory = Split-Path -Path ${{ parameters.projPath }}
      Set-Location $directory 

      our-build-tool restore
      our-build-tool build

References

mm
0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments