Azure DevOps Tips: Empty AWS S3 bucket using AWS CLI or PowerShell script

Not only Azure DevOps has many built-in tools and tasks to support CI and CD processes. But it also has a marketplace for extensions if built-in ones are not sufficient for your CI/CD pipelines.

One of such extensions is the official AWS Tools for Microsoft Visual Studio Team Services provided by Amazon. This extension contains tasks to interact with Amazon S3, Elastic Beanstalk, CodeDeploy, AWS Lambda and many more of Amazon’s services. And it even contains a task to run AWS CLI commands.

The goal — use AWS CLI task to empty S3 bucket

AWS CLI sounds great, doesn’t it? But there is one catch. Well, let’s try to set up a CLI task to empty an S3 bucket.

Error on run! It turns out that the AWS CLI is not installed automatically when the extension is installed.

Solution №1 — Add extra Command Line task

Install awscli python package using Command Line task and the following commands.

curl -O https://bootstrap.pypa.io/get-pip.py

python3 get-pip.py

pip install awscli — upgrade

Solution №2 — use AWS Tools for Windows PowerShell Script tasks

Instead of using Command Line and AWS CLI tasks we can use only AWS Tools for Windows PowerShell Script tasks and a PowerShell script.

Get-S3Object -BucketName “s3-test-bucket” | ForEach-Object {
    $key = $_.Key;
    Write-Output “Removing S3 Object: $key”
    Remove-S3Object -BucketName $BucketName -Key $key -Force:$true
}

Conclusion

These are two but not the only solutions to empty an S3 bucked from Azure DevOps pipeline. I can only suggest you try out Azure DevOps if you haven’t yet. And let me know if you have found another way to empty an S3 bucket.