Two-way-synchronization between VSTS and GitHub

There are several reasons why you might want to keep your GitHub repositories synchronized with VSTS, or vice versa.
For instance:
– You want to use the continuous integration features of VSTS for your projects hosted in GitHub;
– You want to work with your team in VSTS, but still publish your source code as open source;

To achieve that goal you need to set up two build definitions in VSTS, one of them being triggered by commits in GitHub, and the other one being triggered by commits in VSTS.

Creating the other repository

The first steps of the procedure depends on where your starting point is: you already have your GitHub repository, or your sources are in VSTS. From the moment you have two repositories, all the work can be done in VSTS.

I assume you know how to make a new GitHub or VSTS repository.
If you already have your GitHub repository, then you need to make a corresponding repository in VSTS, either by creating a new empty repository in an existing VSTS project, or by creating a new project. A new project will automatically have an empty repository with same name.
When you make a new VSTS repository then you are forced to add a README and/or a .gitignore file, otherwise the Initialize button will not be enabled. So add a README and initialize the repository. Then immediately delete the readme.md file, to start with an empty master branch.

If, on the contrary, you already have your source code in VSTS, then you need to make an empty repository in GitHub. When you create a new repository in GitHub, also choose to initialize it with a readme file. Immediately delete it, to create an empty master branch.

The creation and immediate removal of a readme file feels a bit needless, but it is the only way to initialize a master branch.

The repository names

The name of your VSTS repository can be found by going to the repository and click the Clone button:

Wherever you read VSTS-REPO in the instructions replace it by this name.

The name of your Git repo can be found in a similar way:

I will refer to this name with the placeholder GIT-REPO.

1. Sync GitHub to VSTS
Generate a VSTS token

The first thing we are going to do is generate a token that we need when we access VSTS from the command line.
Go to the security option in your Profile Menu and add a personal access token.

     

Give your token a description, set an expiration, and in the selected scopes, grant it access to your code. Then hit the create token button and also store this token to the side for later. This is important, because it will never be visible again after you close this screen!

 

Make the build definition

    

VSTS asks you to choose a template. We start with an empty process.  In the next screen give it a meaningful name, like Sync_Demo_GitHub2VSTS and choose an agent. Click on Get sources, select GitHub and click on Authorize.

Fill in your GitHub credentials, and then you can choose your GitHub repository you want to connect to.

Then click on the + in Phase 1, scroll down to the Command Line task, and click on Add four times. Four empty Run tasks appear in Phase 1.

The sources will be read from GitHub, so the first thing we do is executing a git pull from the VSTS repository to merge the latest changes in VSTS. Then we can push the merged sources back into VSTS. In my case I also pull and push the tags. If you do not need to sync the tags, then two run tasks are enough.

 

Before filling in the details of those tasks we need to add the token to the variables.
Go to the Variables tab, click the Add button, and add the variable token and take the token you just generated in the previous step. Finally, you can secure the token by clicking on the lock icon.

Then go back to the Tasks tab to add the details to the Command Line tasks.

For all four tasks the Tool is git. The arguments are:
1. pull https://$(token)@VSTS-REPO master –allow-unrelated-histories
2. pull –tags https://$(token)@VSTS-REPO
3. push https://$(token)@VSTS-REPO head:master
4. push –tags https://$(token)@VSTS-REPO

If you are not interested in synchronizing the tags, then you can skip 2 and 4.

Then Save the build definition and click on Save & queue to test it immediately. When it has run succesfully you will see your GitHub repository perfectly copied to your VSTS repository.

Finally change the Build you just created to run it automatically on each commit you do in GitHub. Go to the Triggers tab, and enable Continuous Integration.

2. Sync VSTS to GitHub

The build definition for synchronizing VSTS to GitHub is very similar.

    

VSTS asks you to choose a template. Start with an empty process.  In the next screen give it a meaningful name, like Sync_Demo_VSTS2GitHub and choose an agent. Click on Get sources to verify that the correct VSTS repository is already preselected for you.

Then go to the Variables tab to add the credentials for the GitHub repository. Add a variable gitcredentials and give the value username:password. It might be a good idea to click the lock icon to make the value invisible for your team members.

Then go back the Tasks tab, click on the + in Phase 1, scroll down to the Command Line task, and click on Add four times. Four empty Run tasks appear in Phase 1.

The sources will be read from VSTS, so the first thing we do is executing a git pull from the GitHub repository to merge the latest changes you made in GitHub. Then we can push the merged sources back into GitHub. In my case I also pull and push the tags. If you do not need to sync the tags, then two run tasks are enough.

 

Now add the details to the Command Line tasks.

For all four tasks the Tool is git. The arguments are:
1. pull https://$(gitcredentials)@GIT-REPO master –allow-unrelated-histories
2. pull –tags https://$(gitcredentials)@GIT-REPO
3. push https://$(gitcredentials)@GIT-REPO head:master
4. push –tags https://$(gitcredentials)@GIT-REPO

If you are not interested in synchronizing the tags, then you can skip 2 and 4.

Then Save the build definition and click on Save & queue to test it immediately. When it has run succesfully you will see your VSTS repository perfectly copied to your GitHub repository.

Finally change the Build you just created to run it automatically on each commit you do in VSTS. Go to the Triggers tab, and enable Continuous Integration.

To conclude

So by following these instructions we have created a situation where the VSTS and GitHub repositories are kept in sync, no matter on which side you do the commits. The VSTS continuous integration tooling will take care of that silently. Well, not really… your inbox will be spammed with status reports after each build. You might feel the need to switch those notifications off once you are confident that it does what you want it to do.

Also note that it only keeps your master branches in sync. But following the same procedure you can easily add the steps to keep other branches in sync as well.

Finally, I would recommend to try and test it first with a small test project like I did. When you are happy with the result, then implement it in your production environment.

Happy syncing!!

Leave a Reply

Your email address will not be published. Required fields are marked *