Deploying SSIS Projects to a Restored SSIS Catalog (SSISDB)

This post was originally shared on SQLBlog.

You’re coding away with SSIS, happy as a clam, and it’s now time to deploy your hard work to an instance of the SSIS Catalog. If you’re in Visual Studio, you right-click the project name in Solution Explorer and click Deploy. If you’ve been given an ISPAC file, you double-click it in Windows Explorer. Either way, the Integration Services Deployment Wizard starts and you (happily) supply the SSIS Catalog host instance and choose (or create) the SSIS Catalog folder, and then click Next buttons until you click the Deploy button. It’ll be just a minute now, and you’ll see four green circles with checks…

Except you don’t.

You see three green circles with checks and one ugly red circle that contains an X:

 kaboom

What’s worse, there’s an ERROR message box – sporting another of those handy red-X circles – that states:

Please create a master key in the database or open the master key in the session before performing this operation. (Microsoft SQL Server, Error: 15581)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=13.00.4199&EvtSrc=MSSQLServer&EvtID=15581&LinkId=20476

Ok, not to worry. Copy the For-help-click URL and paste…

 This isn’t the site you are looking for…

What Now?

I’ve seen this happen in the wild. It’s not pretty and never convenient. Sometimes, someone has “saved all sorts of time and money” by backing up the SSISDB database – say, on a Production server – and restored it to a QA server using code similar to this:


That’s not entirely accurate

And that restore executes and succeeds – just like the one I’ve shown above. Everything’s good to go, right? Well, everything looks good to go…

With the restore complete, you’re ready to deploy the latest updates to QA for testing. That’s when this particular error rears its ugly head.

“How Do I Fix It, Andy?”

I’m glad you asked!

Fix #1

SSISDB is not your ordinary, run-of-the-mill database. There’s an application built around it. I can hear you thinking, “That’s pretty ordinary, Andy…” and you’re right. But most of the applications databases work with are not built into SQL Server Management Studio (SSMS), now are they? SSISDB’s application is found in the SSMS Object Explorer node named “Integration Services Catalogs.”

Microsoft has some very helpful instructions at a page named Backup, Restore, and Move the SSIS Catalog (for SSIS Catalogs in SQL Server 2016. See this link for 2012 and this link for 2014). To backup and restore the SSISDB database, you’ll need to know the password used for encryption when you initially created the SSIS Catalog. The script I created to restore my SQL Server 2016 SP1 Catalog test instance is appended to this post. Feel free to copy, paste, and edit. But again, you must have the password used to create the original SSIS Catalog.

Fix #2

You can use SSIS Catalog Compare to generate scripts and ISPAC files from your Production instance of the SSIS Catalog, execute the scripts and ISPAC files in the prescribed order on your QA server, and be on your merry way. If your updates reside in a folder (or folders), you can opt to script the contents of a single folder thus:

If you desire to migrate the contents of an entire SSIS Catalog instance to another instance, you can script the entire Catalog:

Either way, scripts and ISPAC files are generated inside the file system folder you select. A file system folder is created for each SSIS Catalog folder, and the contents of this file system folder are scripts and ISPAC files required to migrate your SSIS packages, projects, folder, environments, references, and parameter mappings to the SSIS Catalog of your choosing and you don’t need the original password used to create the original SSIS Catalog:

The scripts are named with numeric prefixes to ensure they are deployed according to SSIS Catalog precedence requirements.

You might want to learn more about SSIS Catalog Compare here.

You might also want to view the free recording my of my webinar SSIS Lifecycle Management (registration required).

Andy’s Script for restoring SSISDB to SQL Server 2016 SP1

As promised, here’s the script I use to restore SSISDB to a SQL Server 2016 SP1 instance:

/*

  My script for restoring SSISDB to a SQL Server 2016 SP1 instance of SQL Server.
I followed the instructions found at https://msdn.microsoft.com/en-us/library/hh213291(v=sql.130).aspx

Hope this helps,
Andy Leonard
*** Action is required where you see three asterisks “***”

*/

-- create the ##MS_SSISServerCleanupJobLogin## login if it does not already exist.
USE [master]
GO

print '##MS_SSISServerCleanupJobLogin## login'
If Not Exists(Select [name]
From sys.sql_logins
Where [name] = '##MS_SSISServerCleanupJobLogin##')
begin
print ' – Creating the ##MS_SSISServerCleanupJobLogin## login'
CREATE LOGIN [##MS_SSISServerCleanupJobLogin##] WITH PASSWORD=N'DWehrJfiRgMxEFaE=KxomUkF7fnV3poW/ZQPJ' -- *** change this, please – Andy
, DEFAULT_DATABASE=[master]
, DEFAULT_LANGUAGE=[us_english]
, CHECK_EXPIRATION=OFF
, CHECK_POLICY=OFF
print ' – ##MS_SSISServerCleanupJobLogin## login created'
end
Else
print ' – ##MS_SSISServerCleanupJobLogin## already exists.'
GO

print ''

print ' – Disabling the ##MS_SSISServerCleanupJobLogin## login'
ALTER LOGIN [##MS_SSISServerCleanupJobLogin##] DISABLE
print ' – ##MS_SSISServerCleanupJobLogin## login disabled'
GO

USE [master]
GO

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

print 'dbo.sp_ssis_startup stored procedure'
If Exists(Select s.name + '.' + p.name
From sys.procedures p
Join sys.schemas s
On s.[schema_id] = p.[schema_id]
Where s.[name] = 'dbo'
And p.name = 'sp_ssis_startup')
begin
print ' – Dropping dbo.sp_ssis_startup stored procedure'
Drop PROCEDURE [dbo].[sp_ssis_startup]
print ' – dbo.sp_ssis_startup stored procedure dropped'
end

print ' – Creating dbo.sp_ssis_startup stored procedure'
go

    CREATE PROCEDURE [dbo].[sp_ssis_startup]
AS
SET NOCOUNT ON
/* Currently, the IS Store name is 'SSISDB' */
IF DB_ID('SSISDB') IS NULL
RETURN

IF NOT EXISTS(SELECT name FROM [SSISDB].sys.procedures WHERE name=N'startup')
RETURN

/*Invoke the procedure in SSISDB  */
/* Use dynamic sql to handle AlwaysOn non-readable mode*/
DECLARE @script nvarchar(500)
SET @script = N'EXEC [SSISDB].[catalog].[startup]'
EXECUTE sp_executesql @script
GO
print ' – dbo.sp_ssis_startup stored procedure created'
print ''

use master
go
print 'Enabling SQLCLR'
exec sp_configure 'clr enabled', 1
reconfigure
print 'SQLCLR enabled'
print ''

print 'MS_SQLEnableSystemAssemblyLoadingKey asymetric key'
If Not Exists(Select [name]
From sys.asymmetric_keys
Where [name] = 'MS_SQLEnableSystemAssemblyLoadingKey')
begin
print ' – Creating MS_SQLEnableSystemAssemblyLoadingKey'
Create Asymmetric key MS_SQLEnableSystemAssemblyLoadingKey
From Executable File = 'E:\Program Files\Microsoft SQL Server\130\DTS\Binn\Microsoft.SqlServer.IntegrationServices.Server.dll'  -- *** check this, please – Andy
print ' – MS_SQLEnableSystemAssemblyLoadingKey created'
end
Else
print ' – MS_SQLEnableSystemAssemblyLoadingKey already exists.'
go
print ''

print 'MS_SQLEnableSystemAssemblyLoadingUser SQL Login'
If Not Exists(Select [name]
From sys.sql_logins
Where [name] = 'MS_SQLEnableSystemAssemblyLoadingUser')
begin
print ' – Attempting to create MS_SQLEnableSystemAssemblyLoadingUser Sql login'
begin try
Create Login MS_SQLEnableSystemAssemblyLoadingUser
From Asymmetric key MS_SQLEnableSystemAssemblyLoadingKey
print ' – MS_SQLEnableSystemAssemblyLoadingUser Sql login created'
print ' – Granting Unsafe Assembly permission to MS_SQLEnableSystemAssemblyLoadingUser'
Grant unsafe Assembly to MS_SQLEnableSystemAssemblyLoadingUser
print ' – MS_SQLEnableSystemAssemblyLoadingUser granted Unsafe Assembly permission'
end try
begin catch
print ' – Something went wrong while attempting to create the MS_SQLEnableSystemAssemblyLoadingUser Sql login, but it''s probably ok…'
-- nothing for now
end catch
end
Else
print ' – MS_SQLEnableSystemAssemblyLoadingUser Sql login already exists.'

go

print ''

print 'Restoring SSISDB'
USE [master]

begin try
ALTER DATABASE [SSISDB] SET SINGLE_USER WITH ROLLBACK IMMEDIATE
end try
begin catch
-- ignore the error (usually happens because the database doesn’t exist…)
end catch

RESTORE DATABASE [SSISDB]
FROM DISK = N'E:\Andy\backup\SSISDB_SP1.bak'  -- *** check this, please – Andy
WITH FILE = 1,
MOVE N'data' To N'E:\Program Files\Microsoft SQL Server\MSSQL13.TEST\MSSQL\DATA\SSISDB.mdf',   -- *** check this, please – Andy
MOVE N'log' TO N'E:\Program Files\Microsoft SQL Server\MSSQL13.TEST\MSSQL\DATA\SSISDB.ldf',   -- *** check this, please – Andy
NOUNLOAD
, REPLACE
, STATS = 5

ALTER DATABASE [SSISDB] SET MULTI_USER

GO
print ' – SSISDB restore complete'
print ''

print 'Set ProcOption to 1 for dbo.sp_ssis_startup stored procedure'
EXEC sp_procoption N'[dbo].[sp_ssis_startup]', 'startup', '1'
print 'ProcOption set to 1 for dbo.sp_ssis_startup stored procedure'

GO
print ''

Use SSISDB
go

print '##MS_SSISServerCleanupJobUser## user in SSISDB database'
If Not Exists(Select *
From sys.sysusers
Where [name] = '##MS_SSISServerCleanupJobUser##')
begin
print ' – Creating ##MS_SSISServerCleanupJobUser## user'
CREATE USER [##MS_SSISServerCleanupJobUser##] FOR LOGIN [##MS_SSISServerCleanupJobLogin##] WITH DEFAULT_SCHEMA=[dbo]
print ' – ##MS_SSISServerCleanupJobUser## user created'
end
Else
print ' – ##MS_SSISServerCleanupJobUser## already exists.'
GO
print ''

/*

-- One method for restoring the master key from the file.
-- NOTE: You must have the original SSISDB encryption password!

Restore master key from file = 'E:\Andy\backup\SSISDB_SP1_key'    -- *** check this, please – Andy
Decryption by password = 'SuperSecretPassword' -- 'Password used to encrypt the master key during SSISDB backup'    -- *** check this, please – Andy
Encryption by password = 'SuperSecretPassword' -- 'New Password'    -- *** check this, please – Andy
Force
go
*/

-- Another method for restoring the master key from the file.
-- NOTE: You must have the original SSISDB encryption password!
print 'Opening the master key'
Open master key decryption by password = 'SuperSecretPassword' --'Password used when creating SSISDB'   -- *** check this, please – Andy
Alter Master Key
Add encryption by Service Master Key
go
print 'Master key opened'

print ''

print 'Checking the SSIS Catalog Schema Version'
exec [catalog].check_schema_version @use32bitruntime = 0

My results appear similar to this image (click to enlarge):

I prefer writing idempotent scripts that inform me of what they’re doing.

If you’ve encountered this error, I hope this post helps you understand a couple options for responding.

:{>

You might like working with Enterprise Data & Analytics because we have experience with the SSIS Catalog.

Related Training:
SSIS Lifecycle Management (free recording, registration required)
IESSIS2: Immersion Event on Advanced Integration Services – Oct 2017, Chicago

Learn More:
Managing the SSIS Catalog
Announcing SSIS Catalog Compare v1.0
Stop an SSIS Package
Stairway to Integration Services
Stairway to Biml
SQL Server Central

Need help implementing an SSIS solution?
Contact Enterprise Data & Analytics today!

Andy Leonard

andyleonard.blog

Christian, husband, dad, grandpa, Data Philosopher, Data Engineer, Azure Data Factory, SSIS guy, and farmer. I was cloud before cloud was cool. :{>