Reduce the pain of expiring NuGet keys

So in June of this year the NuGet team wrote a blog post “NuGet API key expiration”, where they basically wrote all existing NuGet API keys used for pushing packages will expire. That post caused quite the storm in the .NET community, in August the updated blog post “Changes to Expiring API Keys” was posted explaining that based on the feedback that API keys created before that blog post and are used at least once each year will never expire. but all keys created after August 25, 2016 will have an expiry between 1 and 365days from it’s creation.

NuGet Generate API Key options

Personally I’m all for expiring keys, my only debacle with the way NuGet.org went about expiring keys was the lack of APIs for automating key generation and lack of guidance on how to manage keys, really wish that would've been present before they enabled this feature.
If you could automate the key generation, then even daily expiring keys wouldn’t be an issue as you could potentially just generate keys every x hours with margin.

I don’t like how multiple keys also aren’t supported, which means old key will expire as soon as generate a new one. Why is this bad? Well this means that any running build will fail to publish to NuGet.org when you generate a new key. This is why you at least want 2 keys, a primary and a secondary, which you generate with margin and overlap to go from guaranteed failure to small chance of failure.

If you only use you API key on one place the expiring keys won’t be that big of an issue, you just need every 364 days login to NuGet.org generate a new key and update that setting. So not that much to complaint about.

The issue comes when you login to your build server, in our case AppVeyor and the footer displays page 1 of 3, 10 projects per page…

AppVeyor project paging

…so that’s up to 30 projects that needs to be reconfigured each time a key expires, then it’s no fun and doing so would most certainly be error prone.

So what do you do in this scenario? Well you do as the great @DonovanBrown says, you rub some devops on it! :)
A.K.A. automate all the things FTW!

So what I set out to do was iterate thru all AppVeyor projects and then

• For each project compare a given set of settings
• Update any settings that diff and add any that are missing
• If any project settings changed, report back those to AppVeyor

Fortunately AppVeyor as an REST API which allows you to achieve just this using your language of choice via HTTP calls, you authenticate to the service thru using a token passed to the the Authorization HTTP header, you find your token under the account drop-down on the AppVeyor CI web.

API Token

This time I went with PowerShell, it has a nice command called Invoke-RestMethod which not only makes it easy to do HTTP calls, but it’ll also parses the XML or JSON returned from API into objects you can easily work with from within your script.
Getting and iterating thru all projects would look something like this:

# Declare HTTP headers
$headers = @{
 'Authorization' = "Bearer $($ENV:APPVEYOR_API_TOKEN)"
 'Content-type' = 'application/json'
}
# Fetch all projects
$getProjectsUrl = 'https://ci.appveyor.com/api/projects'
[object[]] $projects = Invoke-RestMethod -Uri $getProjectsUrl -Headers $headers -Method Get
# Iterate over each project, print account name & project slug
$projects | % {
 "{0}/{1}" -f $_.accountName, $_.slug
}

The AppVeyor API endpoints needed to iterate, fetch and update settings are

• Get projects
• Get project settings
• Update project
  (unfortunately there’s no method to update just the environment variable settings, so you’ll have to update the whole project)

The standard ConvertFrom-Json/ConvertTo-Json PowerShell commands are fine for just parsing and serializing objects, but for more advanced manipulations they quickly became cumbersome to work with. PowerShell has the ability to reference any .NET assembly and use it’s types in your script. The defacto standard for working with JSON on .NET is JSON.Net, so I went with that.
Loading and using JSON.Net from PowerShell would look something like this:

# Get absolute path to assembly
[string] $jsonNetPath = Resolve-Path './Packages/Newtonsoft.Json/lib/net45/Newtonsoft.Json.dll'
# Load assembly
[Reflection.Assembly]::LoadFile($jsonNetPath) | Out-Null
# Parse json into generic JObject
[Newtonsoft.Json.Linq.JObject] $json = [Newtonsoft.Json.Linq.JObject]::Parse('{"hello":  "world"}')
# Get value from parsed object
$json['hello'].Value

Once I’d identified all the moving parts and APIs needed, then I could quickly iterate, refine and tweak the script so it did exactly what I wanted and end result was:

• First parameter was path to JSON file which was the template for how the environment variables should be
• Second parameter was the AppVeyor token and if not specified it tried to fetch token from environment variables
• The script iterated thru all projects, fetching all settings, comparing and then updating/adding only those in supplied JSON file, leaving unknown environment variables untouched.

The input JSON file for changing “NUGET_API_KEY” would look something like below (using the same format as AppVeyor API uses for individual environment variables):

[
  {
    "name": "NUGET_API_KEY",
    "value": {
      "isEncrypted": true,
      "value": "ababababa-dbdbdb-1234-5678-ffdsdfsdf"
    }
  }
]

The script usage would be something like

.\AppVeyorSyncSettings.ps1 -SourceEnvVarsJsonPath settings.json

And the output from the script

Occured  Key                      Status
-------  ---                      ------
20:02:25 cakecontrib/cake-aliasql Validating environment variables...
20:02:32 NUGET_API_KEY            DIFF
20:02:32 NUGET_API_KEY            REMOVED
20:02:32 NUGET_API_KEY            ADDED
20:02:32 Has changes              True
20:02:32 cakecontrib/cake-aliasql Putting settings to AppVeyor.
20:02:37 cakecontrib/cake-aliasql Done.

Writing the script took around 15 minutes, time well spent IMHO, as any change to future API keys will just be matter of changing in one place, kicking of script, grab coffee and it’ll be done when you get back.
But as it can set one to many settings we can use it to keep any setting in sync like Slack / Gitter channel, MyGet credentials, etc. etc.
Also for a new project you just execute the script and it’ll add all your standard variables.

I’ve made my current script avail as a gist here, it’s probably in need of some refactoring and it’s only tested on PowerShell 5.1 (Windows 10), but I think it’s good enough state to be shared and help others. Feel free to modify it to your needs! And please to ping back any feedback to me.

Thanks for reading,
/Mattias