Recently I have been looking into complex event processing, what the use cases for it are and the kind of insights you can get from it. As I am a .Net developer, I thought it best to start with the Microsoft stack, so I have been experimenting with StreamInsight. There are a few examples floating around, but the one that really inspired me was the Big Data Twitter Demo (link).

In this post I’m going to take a break from the actual development of custom components for SSIS and show how the RabbitMQ Source that I have created so far could be used in an ETL package to load a data warehouse. The Scenario The scenario for this post is a company that sells cool stuff over the internet, they have a website where people can signup and registered users can place orders.

A lot of the content in this article is similar to what I did in Part 2, so if you haven’t read it I suggest you go back and read it now :) The next part of this series will take a look at implementing a custom UI for configuration of the RabbitMQ Source component. The main reference for this entry can be found on MSDN. A shout out and my thanks goes to Tillmann Eitelberg (Web | Twitter) who pointed me in the right direction when I was trying to figure out how to get a list of all the connection managers in a package.

So far in this series I have shown how to create a custom connection manager and also how to create a user friendly interface to configure and test it. In this post I’m going to outline the steps I went through to create a custom source component for SSIS. By the end of this post I will show messages that are in a RabbitMQ queue being consumed by an SSIS package.

Following on from my introductory post, here I will be explaining how to develop a Custom Connection Manager for RabbitMQ. This connection manager will then be used in future posts by a custom Source and Destination component. What is a Connection Manager for? From the MSDN documentation: “Integration Services uses connection managers to encapsulate the information needed to connect to an external data source. […] If the connection managers and external data sources supported by Integration Services do not entirely meet your requirements, you can create a custom connection manager.

In my previous post I explained how to create a custom connection manager for SSIS to connect to RabbitMQ. In this post I’m going to extend this further by adding a custom UI where the connection can be configured and tested. Preparing the Project The first step is to add a few more references: Microsoft.SqlServer.Dts.Design Microsoft.SqlServer.DTSPipelineWrap Microsoft.SqlServer.DTSRuntimeWrap Microsoft.SqlServer.PipelineHost These assemblies can be found in C:\Program Files (x86)\Microsoft SQL Server\110\SDK\Assemblies The next step is to add two new files, the first is an empty class file which I’ve called RabbitMQConnectionManagerUI.