SQL Server 2016 Temporal Tables and Type II Data Warehouse Dimensions

SQL Server 2016 introduces temporal tables, an interesting and “data warehouse-y” addition to the relational database.

So how do temporal tables work? They add a couple columns to a table that are used to track the date a row is inserted or last updated, based on the primary key. Suppose you have a row that looks like this:

Temporal_0

If you begin tracking this data temporally, the row could appear similar to this:

Temporal_1

Note the dateTime2 data type used for the date-range fields. A ToDate value of 9999-12-31 23:59:59.9 indicates this row is “current.” The value in the FromDate field indicates when the row was inserted or last updated, whichever happened last.

How did we get here? Let’s build out a demo. But first, a shout-out to Damian Widera, who’s SQLBlog post titled “Temporal table behind the scenes – part 1” raised my interest in temporal tables.

Demo

You can begin by creating a database. I created TemporalTests using this following statement:

If Not Exists(Select name
          From sys.databases
          Where name = ‘TemporalTests’)
begin
       Create Database TemporalTests
end
go

Next, I create a table named TestTable in TemporalTests:

Use TemporalTests
go

If Not Exists(Select name
              From sys.tables
               Where name = ‘TestTable’)
begin
  Create Table dbo.TestTable
  (
       Id Int Identity(1,1)
        Constraint PK_TestTable Primary Key,
       Name VarChar(20),
       Value Int,
       FromDate DateTime2(1) Generated Always As Row Start Not NULL,
       ToDate DateTime2(1) Generated Always As Row End Not NULL,
       Period For System_Time (FromDate, ToDate)
  ) With (System_Versioning = ON)
end
go

I won’t go into all the details of the syntax, but I will point out three requirements:

  1. The table needs a primary key. This serves to identify the row and, I believe, should not be updated. This makes the primary key function a lot like a surrogate key in a data warehouse dimension.
  2. The table needs two DateTime2 fields configured similar to those shown (FromDate and ToDate) and a clause identifying these fields as the temporal value columns – Period For System_Time (FromDate, ToDate).
  3. The table needs the key phrase With (System_Versioning = ON) at the end of the Create Table statement (when creating a temporal table).

When created, the table in Object Browser appears as shown:

Temporal_2

Note you can supply a name for the temporal system table, but if you don’t supply a name SQL Server will create one for you.

Let’s load some data!

Insert Into dbo.TestTable
   (Name, Value)
Values
   (‘A’, 0)
, (‘B’, 1)
, (‘C’, 2)
, (‘D’, 3)
, (‘E’, 4)

If we query the table, we’ll see all the rows have the same FromDate and ToDate:

Temporal_3

Now, if we update a few rows:

Update dbo.TestTable
Set Value = Value * 10
Where Value > 2

The table now has some rows – the last two – with different FromDate values:

Temporal_4

We can now exercise a Type-II-like dimension query to examine the state of the rows at some time before the latest update:

Select *
From dbo.TestTable
For SYSTEM_TIME As Of ‘2015-09-18 22:13:00.0’

The results? The values as they were at the time specified in the For SYSTEM_TIME As Of ‘2015-09-18 22:13:00.0’ clause:

Temporal_5

Cool? I thought so too.

I still have unanswered questions about temporal tables for Type-II dimension tracking. The largest question regards performance at scale. We’ll see once we get out of the SQL Server 2016 CTPs…

:{>

Andy Leonard

andyleonard.blog

Christian, husband, dad, grandpa, Data Philosopher, Data Engineer, Azure Data Factory, SSIS guy, and farmer. I was cloud before cloud was cool. :{>

7 thoughts on “SQL Server 2016 Temporal Tables and Type II Data Warehouse Dimensions

  1. Hi Andy, I can’t see it working for type 2 SCD’s without app-temporal time rather than system temporal.
    Regards,
    Greg

  2. Hi Greg,
      I usually (not always) push application time zone management into the reporting solution. In the US, this solves issues with time zones and those pesky switches between Daylight Savings Time.
    :{>

  3. Hi Andy, this looks really nice and handles some temporal issues but we/I/you/the DW community would still need a way at looking at attribution based on the source systems effective dating of attribution.  Its a good start on MS part but they need a way to have a user defined effective dating column.  So you could write a statement that says something like….
    Select * from TestTable System_Time as of ‘9999-12-31’ and SOURCE_EFFECTIVE_TIME as of ‘2015-01-01’.  I don’t have a really good idea of how to load or how to represent the SOURCE_EFFECTIVE_TIME.

  4. Steve nailed it. This is a good concept but there are some bits of transactional metadata that should be transparent with a solution like this. Azure SQL Database with Mobile Services makes this totally transparent. I really, really wish Microsoft had taken that approach with system-versioned tables in the on-prem DB.

  5. You’ve showed the historical rows getting end dated after the update. Do the new current rows (type 2 inserts) get new surrogate keys? Since that would have to be inserted into the fact to link in the correct version of the row assuming you’re tracking the change in relationship to facts.

  6. Greg, Steve, Kevin, and Shaun,
      Those are excellent thoughts, and solid limitations to using temporal tables to manage Type II data warehouse dimensions.
    :{>

  7. There is a standard for how this should be done. That’s the distinction between sys-temporal (what we got) and app-temporal (what’s needed most of the time), and bi-temporal (that provides both).

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.