SQL Server 2016 introduces temporal tables, an interesting and “data warehouse-y” addition to the relational database.
So how do temporal tables work? They add a couple columns to a table that are used to track the date a row is inserted or last updated, based on the primary key. Suppose you have a row that looks like this:
If you begin tracking this data temporally, the row could appear similar to this:
Note the dateTime2 data type used for the date-range fields. A ToDate value of 9999-12-31 23:59:59.9 indicates this row is “current.” The value in the FromDate field indicates when the row was inserted or last updated, whichever happened last.
How did we get here? Let’s build out a demo. But first, a shout-out to Damian Widera, who’s SQLBlog post titled “Temporal table behind the scenes – part 1” raised my interest in temporal tables.
Demo
You can begin by creating a database. I created TemporalTests using this following statement:
If Not Exists(Select name
From sys.databases
Where name = ‘TemporalTests’)
begin
Create Database TemporalTests
end
go
Next, I create a table named TestTable in TemporalTests:
Use TemporalTests
go
If Not Exists(Select name
From sys.tables
Where name = ‘TestTable’)
begin
Create Table dbo.TestTable
(
Id Int Identity(1,1)
Constraint PK_TestTable Primary Key,
Name VarChar(20),
Value Int,
FromDate DateTime2(1) Generated Always As Row Start Not NULL,
ToDate DateTime2(1) Generated Always As Row End Not NULL,
Period For System_Time (FromDate, ToDate)
) With (System_Versioning = ON)
end
go
I won’t go into all the details of the syntax, but I will point out three requirements:
- The table needs a primary key. This serves to identify the row and, I believe, should not be updated. This makes the primary key function a lot like a surrogate key in a data warehouse dimension.
- The table needs two DateTime2 fields configured similar to those shown (FromDate and ToDate) and a clause identifying these fields as the temporal value columns – Period For System_Time (FromDate, ToDate).
- The table needs the key phrase With (System_Versioning = ON) at the end of the Create Table statement (when creating a temporal table).
When created, the table in Object Browser appears as shown:
Note you can supply a name for the temporal system table, but if you don’t supply a name SQL Server will create one for you.
Let’s load some data!
Insert Into dbo.TestTable
(Name, Value)
Values
(‘A’, 0)
, (‘B’, 1)
, (‘C’, 2)
, (‘D’, 3)
, (‘E’, 4)
If we query the table, we’ll see all the rows have the same FromDate and ToDate:
Now, if we update a few rows:
Update dbo.TestTable
Set Value = Value * 10
Where Value > 2
The table now has some rows – the last two – with different FromDate values:
We can now exercise a Type-II-like dimension query to examine the state of the rows at some time before the latest update:
Select *
From dbo.TestTable
For SYSTEM_TIME As Of ‘2015-09-18 22:13:00.0’
The results? The values as they were at the time specified in the For SYSTEM_TIME As Of ‘2015-09-18 22:13:00.0’ clause:
Cool? I thought so too.
I still have unanswered questions about temporal tables for Type-II dimension tracking. The largest question regards performance at scale. We’ll see once we get out of the SQL Server 2016 CTPs…
:{>
Hi Andy, I can’t see it working for type 2 SCD’s without app-temporal time rather than system temporal.
Regards,
Greg
Hi Greg,
I usually (not always) push application time zone management into the reporting solution. In the US, this solves issues with time zones and those pesky switches between Daylight Savings Time.
:{>
Hi Andy, this looks really nice and handles some temporal issues but we/I/you/the DW community would still need a way at looking at attribution based on the source systems effective dating of attribution. Its a good start on MS part but they need a way to have a user defined effective dating column. So you could write a statement that says something like….
Select * from TestTable System_Time as of ‘9999-12-31’ and SOURCE_EFFECTIVE_TIME as of ‘2015-01-01’. I don’t have a really good idea of how to load or how to represent the SOURCE_EFFECTIVE_TIME.
Steve nailed it. This is a good concept but there are some bits of transactional metadata that should be transparent with a solution like this. Azure SQL Database with Mobile Services makes this totally transparent. I really, really wish Microsoft had taken that approach with system-versioned tables in the on-prem DB.
You’ve showed the historical rows getting end dated after the update. Do the new current rows (type 2 inserts) get new surrogate keys? Since that would have to be inserted into the fact to link in the correct version of the row assuming you’re tracking the change in relationship to facts.
Greg, Steve, Kevin, and Shaun,
Those are excellent thoughts, and solid limitations to using temporal tables to manage Type II data warehouse dimensions.
:{>
There is a standard for how this should be done. That’s the distinction between sys-temporal (what we got) and app-temporal (what’s needed most of the time), and bi-temporal (that provides both).