Archiving Bitbucket Server repositories to S3 or GCS

A quick script example using “git bundle” to help archive git repositories to AWS Glacier or GCS Coldline storage.

Bitbucket Server is organised into projects each of which contains multiple repositories. Now let’s take an example of an organisation migrating tens of thousands of repositories away from their hosted Bitbucket Server set up.

Not all of these repositories were being migrated. Many tens, if not hundreds of these projects are very old and some teams wanted to ‘cold archive’ them rather than migrate to their new Git hosting — with their choice being either S3 Glacier or GCP Coldline storage.

Note that this article is aimed at end-users rather than the administrators of a Bitbucket Server, who have access to other tools for full or partial backups.

A quick introduction to the relevant concepts of the Bitbucket Server API.

• Authorization requires a TOKEN which is passed to the API via the Authorization: Bearer TOKEN header.
• Given a PROJECT, then the API offers an endpoint rest/api/1.0/projects/PROJECT/repos to get its repositories.
• The API has paging with a limit=NNN to control how many records are returned.

The pseudo-logic we are after is;

1. Obtain the list of repositories for the project
2. Iterate each, clone…