Team Foundation Service currently has no GUI to delete team projects. Fortunately, Visual Studio 2012 or Team Explorer 2012 provide us with a tfsdeleteproject.exe which will do just that for us.

By default, the tool is located Visual Studio’s installation folder:
C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE
Direct the command prompt to the path above and run the following command with the appropriate values:
tfsdeleteproject.exe /collection:https://{{ your account name }}.visualstudio.com/DefaultCollection "{{ Team Project To Be Deleted }}"
A warning message should show telling you that there’s no going back if you press Y.

Warning: Deleting a team project is an irrecoverable operation. All version control, 
work item tracking and Team Foundation b uild data will be destroyed from the system. 
The only way to recover this data is by restoring a stored backup of the databas es. 
Are you sure you want to delete the team project and all of its data (Y/N)?

Just wait for the various tasks to complete…

Deleting from Build ...
Done
Deleting from Version Control ... 
Done 
Deleting from Work Item Tracking ... 
Done 
Deleting from TestManagement ... 
Done 
Deleting from Git ... 
Done 
Deleting from ProcessManagement ... 
Done 
Deleting from LabManagement ... 
Done 
Deleting from ProjectServer ... 
Done 
Warning. Did not find Report Server service. 
Warning. Did not find SharePoint site service. 
Deleting from Team Foundation Core ... 
Done

That is it!