Killing off TFS 2005: Part 3

Posted 07 April 2011, 18:30 | by | | Perma-link

Previously on Killing off Team Foundation Server 2005

As you no doubt recall, I've now got the hang of simple migration projects with the Team Foundation Server Integration Tools. I'm therefore trying out the migration of the actual data (from a backup; remember the /confirmed switch we added to the Upgrade command in Part 1?).

The full series

Part 3: Dry run with upgraded project

The first step was to Download the Process Templates (in Visual Studio, go to Team | Team Project Collection Settings | Process Template Manager) that had been used from the MissouriImported Project Collection, and Upload them into the DefaultCollection Project Collection.

I then tried to create a new Project using the Process Template I'd imported, but it failed on the "Populate Reports" task. As is increasingly the case, the logs are your friend. The cause was that the Reporting Services Data Source names have changed since TFS 2005 - they were originally called TfsOlapReportDS and TFSReportDS - however in TFS 2010 they are now Tfs2010OlapReportDS and TFS2010ReportDS respectively.

This was fairly simple to fix - navigate to the Reports folder in the downloaded process template, open the ReportsTasks.xml file and do a Find and Replace on "/TfsOlapReportDS" and "/TfsReportDS" for the 2010 versions - note the leading slash in there - you want to be replacing the "name" attribute, while leaving the "dnsname" attribute as is - this will allow TFS to find the incorrect version, and update it with the correct one. Save the file, and re-upload the process template. To be safe, I then restarted Visual Studio.

With the updated Process Template in place, I successfully created a couple of new projects using it, and decided to try and migrate the larger of the two projects.

It all started so well. I set up the TFS Integration configurations as per my learnings from the Hands On Lab (HOL), saved it, and started the migration. The tool soon found 1,699 Work Item Change groups (work items, changes/updates and links - there are about 600 work items in that project, we weren't really using it properly), analysed them, and started migrating them - that's when the trouble started.

It turns out that someone added a new Work Item Type to the project without updating the Process Template - so there are just over 200 work items of type "Defect" that have no where to go, which is odd, as the (admittedly out of date) HOL had implied that "the default is to move all work item types" and this is only disabled if you define a mapping - I've not defined any mappings, but clearly it's not going to create it on it's own. It's all the more tedious as the preferred fix is to create a mapping rule, and resolve the issue that way - however it appears at the moment that I have to do this one conflict at a time, and there are currently over 400 and counting so it could take some time. That said, the whole purpose of this test is to iron out these sorts of issues, so what I'm going to do for next time is import the missing Work Item Type.

At the time of me writing this (18:13), the process has been running for about 1.75 hours, it's migrated 1,236 of 1,699 work item "change groups" (after discovering and analysing them) - which is about what I'd expected and discovered over 12,000 Version Control "change groups" - the change set ID's on that server go up to 53,000, however they aren't all in the projects I'm migrating - Sidekicks seems to think it's around 33,000 Confused

There now follows a short intermission.

Filed under: TFS

Killing off TFS 2005: Part 2

Posted 07 April 2011, 13:10 | by | | Perma-link

Following on from where we left off in Part 1.

Recap

We've upgraded a copy of the Team Foundation Server 2005 databases to TFS 2010, and connected to it using Visual Studio 2010 to confirm I was happy that the source code, history and work items had indeed been migrated successfully.

I've also installed a copy of VS2003, and the latest version of the MSSCCI Provider to confirm that I'm happy with it in that scenario, as well as confirmed that users that aren't project administrators can also see the history, etc (we'd had issues with this on the VSS - TFS upgrade).

The full series

Part 2: Merging Projects

Now comes the more complicated part: Merging the projects from the upgraded Team Project Collection (MissouriImported). This is made more complicated by the fact that I didn't come up with a sensible name for the new instance of a project in the original TFS 2010 instance (Newyork) - so I now have two projects with the same name in different collections that I'd like to have in the same collection, but with different names.

After I'd installed the TFS Integration Tools (note to self: When it asks for the database server, and defaults to a SQL Express instance, if you're connecting to the default SQL Server instance on a box, you don't want to add the instance name - so just TFS10 and not TFS10\MSSQLSERVER Confused) I then tried going through the Getting Started with a TFS to TFS Migration Hands On Lab (HOL) that comes as part of the guidance - I strongly recommend that you do as well to familiarise yourself with the process, however it's not all plain sailing.

Running through the Hands On Lab

Basic Scenario

This worked out quite nicely, but then it's designed to minimise and work around all potential conflicts - as I'm not using the "Visual Studio ALM Rangers HOL base image" mentioned in step 4 of the HOL, I obviosuly had to set up some projects - this I did in much the same way the HOL described. Note that the HOL is a little out of date compared to the tools:

  • When you configure the Version Control and Work Item Tracking sessions, the screen-shots imply that the name of each session is of the form "Project Name (ServerName)", but the tool appears to use "ServerName (Session Type)", so instead of getting "TP-A (tfs10)" I got "tfs10 (VC)" for both the Left and Right sources. You can (and probably should, especally if you're planning on doing a two-way sync) change these names to something more useful.
  • When the HOL asks you to Save and Close you should Save to Database.

As I'm planing on running a "One-way migration" I skipped step 7.3, 7.5 & 7.4 [sic].

Advanced Scenario

This is where the issues really started. As I'm going to be importing into a project with the same team process, I wasn't too fussed about changing the WIT types, however it's good to read through it, as this is the easiest way to move projects to a different process template - something I may be doing at a later point for some of our old projects.

I basically hit an issue with the Version Control (VC) migration, in that the HOL assumes that the VC conflict you're going to see with the BuildProcessTemplates folders will result in the error "A content conflict has been detected…" with an option to use the "Always take the source system's changes" resolution, however this wasn't available to me, as the error I received was "A namespace conflict has been detected between the source and the target system…" which meant I had to provide "merged changesets for the source and target system (requires a manual merge)". This is rather tedious as the files themselves are identical when compared beyond their check-in timestamps, and I couldn't work out what it really wanted - providing some rather arbitrary changeset ids resulted in no source code at all being migrated.

In the end I worked around it by effectively ignoring the BuildProcessTemplates folder:

In the Version Control Session definition, I added a new Paths rule, set it to "Cloaked" for the BuildProcessTemplates paths:

BuildProcessTemplate set to "Cloaked".

With this rule in place, the duplicate BuildProcessTemplates folder and XAML files were ignored by the migration, and it completed smoothly.

As an example, here's the work items list for a check-in on Project1:

Details for changeset 12

As you can hopefully see, this check-in has a comment, and is associated with a closed task work item (2).

Here's the equivalent for the migrated project:

Details for Changeset 39

This is associated with the migrated task (14), and the check-in comment has been carried across, and had the details of where it came from, along with the original check-in details (changeset ID and timestamp).

I'm therefore fairly optimistic now for the dry run of the main project migration, even though it's got over 3GB of "stuff" under source control (I use the word "Source" advisedly there, there's a lot of other things in there, from binaries to SDO packages and beyond).

I'll let you know how it goes.

Filed under: TFS

Killing off TFS 2005: Part 1

Posted 05 April 2011, 14:30 | by | | Perma-link

I'm in the process of upgrading an instance of TFS2005 to 2010, with the added complication of wanting to combine the projects into an existing TFS 2010 Project Collection.

The full series

What am I doing?

As I've had to hunt around on the internet for various steps in the process, I thought I'd try and pull it all together in one place, with the caveat of "it worked for me on my servers", your mileage may vary (considerably) and I take no responsibility for any loss of data, source code, etc. that you may experience (however, I do have to take full responsibility for any loss of data, source code, etc that my team experiences Wink).

To set the scene then, we have the following servers involved in this process, because I want to minimise the downtime that the teams experience while I'm doing this, and the upgrade can't be undone:

  • Missouri - Legacy codebase running on TFS 2005 (upgraded from VSS), using a local instance of SQL Server 2005.
  • Newyork - Current codebase running on TFS 2010 (upgraded from 2008), using a remote SQL Server 2008 instance.
  • TFS10 - Clean install of TFS 2010 using a local instance of SQL Server 2008 for me to play around on.

The three servers: Missouri, TFS10 and Newyork

I found the following resources quite useful as well:

Part 1: Migration Upgrade from TFS 2005 to TFS 2010

These are the steps I took to migrate our TFS 2005 instance (Missouri) to the clean TFS 2010 instance (TFS10):

  1. Copy the latest backup files from Missouri to TFS10 of the following databases:
    • TFSActivityLogging
    • TFSBuild
    • TFSIntegration
    • TFSVersionControl
    • TFSWarehouse
    • TFSWorkItemTracking
    • TFSWorkItemTrackingAttachments
  2. Stop the TFS services using the command line tool found in Tools folder of your TFS installation:
    TFSServiceControl quiesce
  3. Create new databases on TFS10 with the same names as the backups, and restore the backups into these - Note to self: when restoring a database over an existing one, make sure you change the file path on the options pane to the correct destination file path.
  4. Because my friendly SysAdmin had kindly installed TFS for me, opening the TFS Administration Console didn't give me the option of starting the TFS Configuration Tool that's mentioned in the upgrade steps, however, you can run the upgrade process from the command line (again, in the TFS Tools folder):
    tfsconfig import /sqlInstance:tfs10 /collectionName:MissouriImported /confirmed
    Obviously, your SQL Instance and Collection Name will be different, but you get the point.

    This will take a while to run - anecdotal reports are "between 30 and 180 hours", depending on the size of your repository, and the amount of processing resources available. Mine took around 15 minutes on a 10GB version control database and a 76MB work item tracking database.
  5. If all goes well, the command window should report that it's completed "Collection creation step 208 of 208", and that "The Upgrade operation on the ApplicationTier feature has completed. There were 0 errors and 0 warnings.". I can't really advise on what to do if either of those don't happen for you, other than to say "Take a look in the log file", which has quite a lot of detail in it.
  6. Restart the TFS services:
    TFSServiceControl unquiesce
  7. Connect an instance of Visual Studio at the new Project Collection and verify that things look ok. You might also want to try connecting VS2003 (yes, some of the code in there is indeed ASP.NET 1.1, which pretty much requires VS2003, some of it is older still, and is classic ASP/VBScript Sad)

More to follow shortly.

Filed under: TFS

Doodle Grouper 1.0.0.3 Released

Posted 05 April 2011, 11:25 | by | | Perma-link

Update: There's now a "How-To" guide to Doodle Grouper 28/11/2011


Doodle Grouper LogoA quick update to say that I've recently pushed out an updated version of my Seesmic Desktop plugin Doodle Grouper - note that the marketplace version is a bit old, but is the quickest way to install it, and it will auto-update to the latest version.

Key changes since the last update:

  1. Much smaller download - easy mistake to make, I'd left the references as "Copy Local", seeing as I'm either referencing core framework libraries, or the main Seesmic SDK libraries, there's no need for me to include a copy of them.
  2. Controllable memory footprint - one of the issues with this plugin is that it delivers updates to new timelines, which seems to have a fairly impressive impact on memory usage in the SLLauncher process. I've added a plug-in wide setting that will limit the app to only keeping a certain number of updates in each group.
  3. Controllable logging - as I've added more logging to the plug-in, which no-one else should care about, it makes sense to disable this by default, obviously now that Seesmic uses a database file for persistent storage, it means I can't hack around in an XML file to change this, so I've had to add a check-box to my settings page, and I guess I'll be in trouble if I break the settings template at the same time as something else.

Enjoy.

Filed under: Plug-ins, Seesmic, Silverlight

Integrating Tridion with Team Foundation Server

Posted 30 July 2010, 15:00 | by | | Perma-link

I thought I'd start a new series focusing on what I'm currently working on - Tridion from SDL.

The key thing I'm trying to solve at the moment is the deployment of work from one environment to another - initially one developer machine to another, then onto a shared build/deployment setup, then on through QA, User Acceptance Testing and onto live.

We've already done similar things with SharePoint, so we have a baseline of what we're trying to achieve, so it shouldn't be completely impossible.

Seeing as we're working on our C# code, CSS, JavaScript, etc in Visual Studio, tracking our tasks and bugs in TFS, and a build process built around this that takes these assets, compiles them, and generates some scripts to be run on the various environments to deploy them, we felt we'd like to try and utilise this for Tridion elements as well.

Development CMs to TFS to build and deploy servers

The first thing we tried, apparently the "recommended" way, was to use Content Porter. We wanted to prepare a "base" image that we could easily roll out to developers to start working on. We spent the better part of two days trying to break down the solution into small enough packages that we could reliably import them without it throwing an error and rolling back the last hours attempt.

We felt that not only was this frustrating, but also it wouldn't scale nicely - it relies on the developers remembering exactly what they've changed, manually exporting them out with CP - I'm sure it's possible but it does leave a large margin of error.

What we came up with was the following "ideal plan", following a standard "Good, Better Best" approach:

 

Good
(For deployment to QA)

Better

Best
(For deployment to live)

Good, Better, Best Tridion to TFS
Manual Steps

Existing content: Checkout in TFS, edit where appropriate.

New content: Add to TFS via WebDav.

Check in pending changes (including associating work items).

Check in pending changes (including associating work items). Check in pending changes (including associating work items).
Build Solution
For Automation
Automate the creation of TFS workspaces so processes can "Get Latest" straight from TFS into Tridion via WebDav.

Hook into Tridion's Pre-Save event to check-out TFS version, or add to TFS. Warn user of potential issues

Automate the creation of publications in Tridion.

We've started this, and hit our first stumbling block - some content appears to be coming back from webdav as UTF8, while some it coming back as UTF16 - which seems to be confusing TFS when doing a check in, throwing up an error such as:

Visual Studio

z:\300 Design\Building Blocks\System\btn_conference.gif.gif: TF203083: The checksum value of the file that you uploaded does not match the provided value.

Which generally went away if I tried to check the file in a second time.

This seems to be more of a problem with TFS than Tridion or WebDav - an old post from Buck Hodges "How TFS Version Control determines a file's encoding" seems to hint at difficulties with encoding, especially around:

Unfortunately, TFS does not support changing the encoding of a pending add.  If you need to do that, you will have to undo the pending add, and then re-add the file using the command line and specify the /type option.

We think we've found a solution - using a different mechanism to map the WebDav folder to a network drive seems to result in TFS seeing consistent encodings. so this feels like a good place to stop, and report back on progress later.

Filed under: TFS, Tridion

Further work on Seesmic Desktop

Posted 15 July 2010, 22:56 | by | | Perma-link

Update: There's now a "How-To" guide to Doodle Grouper 28/11/2011

Update: I've uploaded a newer version, for more details see: Doodle Grouper 1.0.0.3 Released 05/04/2011


Doodle Grouper LogoSo, with Seemic Desktop 2 hitting Beta this week, I thought I'd just let you all know that I've updated my plugin, Doodle Grouper to work with the current version.

Since the last version, talked about in "Playing with Seesmic Desktop", I've tidied up the interfaces a bit, so they are more in keeping with the main shell theme. This was actually a fairly simple process, as simple as binding the style of the ChildWindow declaration to the main theme:

<controls:ChildWindow x:Class="DoodleGrouper.View.TempModifyGroupWindow" [...]
                      Style="{StaticResource CustomMessageBox}">

And that's it, pretty much, nice, grey dialogs, with pretty buttons on them. The only other thing I had to do was ensure that the Foreground colours of my text blocks were white so that they showed up:

Add User To List Dialog

So to sum up the changes:

  • The plugin now inherits the theme from the parent shell.
  • Groups can be renamed.
  • You can now add and remove users from a group - simple click on the Settings icon for a group, select the users you want to remove from the group, and press "Update", no additional tweets will be shown in that group, and next time you open Seesmic Desktop all their updates will be gone.
  • You can add users to the group at any time, no need to wait for them to post - just add them in, and the next time they post they'll appear - ideal for those infrequent posters that you really want to flag up.

Next on my list to consider is a way to enable popup toast on a per-group basis.

This plugin has two main advantages over the "user lists" feature in Seemic:

  1. It's not tied to just Twitter accounts - you can add Facebook and Google Buzz users in there too, in fact any account that shows posts on the "home" timeline should appear in the group timeline.
  2. It will auto-update - Twitter lists don't (didn't? I've not tried them much recently) update automatically, because each one needs to be requested individually, and this will eat into your api usage quite quickly. As these are updated as posts arrive, they don't use up that precious resource.

If you want to get your hands dirty:

  • The source - you'll need to update the references to the Seesmic.Sdp.* libraries.
  • Doodle Grouper (a repeat of the link to the compiled plug-in from above)

As always, let me know how you get on with it.

Filed under: Plug-ins, Seesmic, Silverlight

Editing Reports in TFS 2010

Posted 29 June 2010, 16:14 | by | | Perma-link

More TFS fun and games:

Team Foundation Server 2010 will happily install and configure Reporting Server for you to power all your reporting needs, but I've come up against the following issue when attempting to modify the existing reports, or create new ones:

When you click on "Report Builder" from the Reporting Services web interface, it opens up the click once application "Report Builder 1.0".

This application cannot open the reports that come with TFS process templates, and that are installed on Reporting Services - the error it reports is:

Cannot open this item
This report cannot be opened in Report Builder.

And if you check the details you see:

System.IO.StreamReader: The Report element was not found.

Which is odd, seeing as if I open the .rdl file in a text editor, I can clearly see the root Report element in there.

If you get the latest (at the time of writing 3.0) version of Report Builder, it will not connect to Reporting Services - it keeps claiming that either the services aren't configured, or I don't have permissions.

However, if you install version 2 of Report Builder, almost all your woes go away, and you can open and edit reports to your hearts content.

The only outstanding woe I had was when I opened a report from the server, edited it, and saved it and then my report appeared to break, claiming that the TfsOlapReportDS was invalid. To resolve that, I had to switch to the Properties tab of the report in the browser, select DataSources and re-map the report to the correct shared data source - once I'd done that, all future saves worked fine.

Filed under: Reporting Server, TFS

Rebuilding TFS Portals

Posted 15 June 2010, 11:29 | by | | Perma-link

We're in the process of moving to Team Foundation Server 2010, and as part of the process, we're updating the process template we're using, creating branded versions of the default documents, etc. Also, with new projects kicking off we're able to really sort out which process template we're using.

However, while we're ironing these changes out, we obviously need to actually do some work - so to that end we'd tweaked a template, created our project using that, and in the background we can work on the template for other projects.

This has left us with two things:

  1. A number of projects in our test Project Collection with portals in the wrong place in SharePoint.
  2. A number of projects using out of date portals.

At the moment, we've not made any changes to the contents in SharePoint (beyond access rights), we're mostly just using the dashboards and reports, so the cost of recreating these templates isn't that high however I was having a lot of difficulty working out how to move existing portals, or how to rebuild a portal after creating the project - the New Team Project wizard includes the option "Do not configure a SharePoint site at this time", with no real guidance on how to create one later.

As this is currently looking at WSS, and these aren't publishing sites, I can't move them easily in SharePoint. We tried exporting them, and importing them into the new location, but that also didn't work - errors about a template missing from the _forms folder (I guess the default template for the excel reports document library).

I tried creating a new site in SharePoint using the TFS templates supplied, but that didn't pick up my changes to the template, nor set up/create the shared documents, or the project dashboard. I also tried creating a dummy project with a portal in the right place, disconnecting the portal in TFS, then pointing the right project at the portal and finally deleting the dummy project, which worked, but was going to be a lot of work for all the other projects.

Finally, I took a look at the actions included in the tfpt command line tool from the Team Foundation Server Power Tools and noticed:

addprojectportal Add or move portal for an existing team project

Pass this the path to TFS, the project and process template you want to move, and optionally the details for the SharePoint instance you want to use and away it goes, creating a new Project Portal for you with all the modifications you've made.

I would point out the following issue I've had with this tool, but it may be due to my usage of it: when I pointed this at a project that already had a portal, I expected the tool to move the portal for me, however (probably because the project was built using a different Process Template) I now have two portals for this project, the old one, and the new one, not a big deal, I can easily delete the old portal - TFS seems to be happily pointing at the new one.

Running it without the /verbose option seemed to fail - and the logs weren't very helpful. Running the same command with /verbose resulted in a successful run. I may have just been unlucky.

Morals of this post:

  • Always check the power tools first.
  • MS have a habit of not finishing things, and releasing the bits you actually need as power tools.
  • Mario Rodriguez rocks.

Filed under: SharePoint, TFS

Working with TypeKit

Posted 09 June 2010, 11:10 | by | | Perma-link

Something I meant to include in yesterday's post was around the steps I'd taken to get TypeKit working with the new GoogleApi WebFont Loader hosted version.

As I'm not using Google AJAX api  decided to just call the file directly, but I couldn't seem to get it to work using the docs on GoogleApi using the WebFontConfig object, however following the directions on the Git hub seemed to work better:

Call the webfont.js file:

<script type="text/javascript" 
        src="http://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js"></script>

Then call load on the WebFont object:

  WebFont.load({
    typekit: { id: 'gjb1flq' }
    });

Passing in your TypeKit Id, (not the Kit Id as is implied in the docs).

To reduce the "Flicker of Unstyled Text" I then added the following to my CSS:

.wf-loading h1 { visibility: hidden; }
.wf-loading h2 { visibility: hidden; }
.wf-loading h3 { visibility: hidden; }
.wf-loading #logo p { visibility: hidden; }
.wf-loading code { visibility: hidden; }
.wf-loading pre { visibility: hidden; }
.wf-active { visibility: visible; }

Basically, as the web fonts are loading, the WebFont.js applies the .wf-loading style to html element, then once they are all in, applies the .wf-active style.

Rather than following the guidance on the git hub page (which suggests having the base state of the h1 hidden, and only showing it when the fonts are active) I've gone down this route, so if the script fails to load, or is turned off, the titles, code, etc will still show, albeit in their default fonts.

If I really wanted to I guess I could implement font based activation as well, using the .wf-adelle1adelle2-n7-active event.

Filed under: AJAX, Site Updates

Upgrading to ASP.NET 4.0

Posted 08 June 2010, 22:00 | by | | Perma-link

Bit of a Link List this one, but it's got a few nuggets in there too.

Thanks to my excellent web hosts LiquidSix Hosting who upgraded our servers within about a week of ASP.NET 4.0 coming out, I've been able to focus on pulling this site over to the latest platform.

There were various reasons for wanting to do this, from cleaner web.config files, the new <%: %> syntax for HTML encoding output, and indeed answering some of my issues with Entity Framework, but obviously the key reason is it's new and shiney Grin

The key issue I had with it was that I forgot to update one small part of my web.config:

<system.web>
    <httpRuntime requestValidationMode="2.0"/>
</system.web>

As I'd forgotten to set this, I was unable to use any markup when editing my blogs. Thankfully the error message tells you exactly what to set, so the fix was easy.

Other tweaks that also went live this week:

  • TypeKit have recently opensourced their code, and added an events system, which means that in FireFox the Flicker of Unstyled Text should be somewhat less intrusive.
  • The launch of DoodleStatic to host my static files (JS, CSS, images) a small, cookieless domain to speed up download of the site infrastructure.
  • Changing my calls to jQuery and Typkit's core libraries to ajax.GoogleApis.com, so your browser should be able to find them in its cache where other sites have used them.
  • Thanks to Damien Gaurd for spending the time to point out how to use the System.ServiceModel.Syndication namespaces in the context of an ASP.NET MVC site, something I'd been meaning to do for some time.

And a few other minor tweaks to keep things tidy.

Filed under: ASP.NET, ASP.NET MVC, Site Updates