SSIS: Adding a derived column

Derived Columns are a great feature in adding new data or metadata to your transforms in SSIS. In this example, I will show you how I used the Slowly Changing Dimension transform against the Customers table in the Northwind database to build a Type II transform. There are several choices you have when marking your active records; my preference is to sue three columns: a start date, and end date and an active flag.

In the SSIS SCD Wizard, you will get this screen in choosing how to mark records that are in transition. I use the wizard to set my bit flag and will later show you how I use the derived columns to set the dates:

 Fig. 1

Here's a quick screen grab of the entire Data Flow Task, our first step is to modify the item highlighted:

 Fig. 2

Which we will set thusly; note the second entry for RecordStartDate, which is my manual input. The wizard handled adding the derived column information for the IsCurrent column:

 Fig. 3

And for the final step, we edit the OLE DB Destination object (called "Insert Destination" in transform):

Fig. 4

Next, we'll edit the secondary branch in the SCD transform -- the branch that deals with changed records. From the overall transform map in Fig. 2, this is the branch on the right side of the picture. The Derived Column object details will look the same as Fig. 3 except that you will name the column RecordEndDate. After editing the Derived Column object, you will next modify the OLE DB Command object, which in Fig. 2 is very blandly called OLE DB Command. There are two places that you will edit this, the first being on the Component Properties table, under the SQLCommand property. You will then set the SQL as in Fig. 5:

After editing the SQL, go to the Column Mappings tab. Here is what it will look like before you edit:

And here is the after picture:

The parameter mappings are done in order of appearance in the SQLCommand property, thus CustomerID moves to the last parameter and RecordEndDate takes over the second spot.

And that's it! You have not set your Slowly Changing Dimension transform to use three columns to indicate the state of a Type II record. I find that using the bit flag for indicating a record's active state works great for performance when querying the system to always get current attributes for a record set. The dates allow for flexibility in looking at point in time states for attribute records around your fact tables -- particularly good for processes you are reporting on that require auditing. Feel free to post any questions/comments.

Tags:
Categories: SQL Server | SSIS

October 12, 2007 20:47 by Sid
E-mail | Permalink | Comments (6) | Comment RSSRSS comment feed

Comments

November 18. 2007 05:44

Jason McLean

If only the Microsoft help was as informative and well written.  Maybe you need to write a book on how to fly a 747.?

Jason McLean

November 18. 2007 17:09

Jason,

Thanks for the compliment -- though I do wish to get my pilot's license some day, I'm a ways off from writing the book. Keep reading and let me know if there is anything else you would like to see detailed in my blog.

Thanks!
Sid

Sid

Add comment


(Will show your Gravatar icon)

  Country flag

biuquote
  • Comment
  • Preview
Loading



Sponsors

Author

Tags

Recent Posts