Service Broker implementation between two tables in one database

I have already provided small demonstration of how service broker works at “Small demonstration of Service broker for beginners” along with list of articles related to service broker from the concept of service broker to its implementation and definition of important part of Service Broker so I will not those concept here in this article, If you wish, you can have a look at those articles, here is the list:

• Importance of Service Broker in SQL Server (Click Here)
• Introduction of Service Broker in SQL Server (Click Here)
• Enable Service Broker in SQL Server database (Click Here)
• Small demonstration of Service broker for beginners (Click Here)
• Find Service Broker Object like Contract, Message Type, Service Name, Queue Name in SQL Server (Click Here)
• Find Service Broker Queue Count in SQL Server (Click Here)
• Get Transmission queue row count in Service Broker (Click Here)
• Purge target Transmission queue of Service Broker in SQL Server (Click Here)
• Deal with size of Transmission Queue table sysxmitqueue in MSDB database (Click Here)

Let us now start implementing Service Broker for two tables in one database.

Let me explain the business logic which will be used in this article.

Sales people in Company “Extreme Trading” (ET) goes to client site and accept the order from client. Sales people may use iPad, iPhone or any other smart phone, tablet, laptop or other portable device to insert order of client.

There is a rule in company that any order shouldn’t have same client id and project id. If same client give order for same project, it should be added in original order. Original order should be edited. It shouldn’t insert new order id for same client, project combination.

Well, we can define composite primary key or put some validation or check constraint etc. but I don’t want to debate on this rule.

If any duplicate order comes, it should be accepted from the sales people but later on it should insert original order id along with new order id in one table so back office team can handle it and sales person could be free immediately without even waiting for validation gets completed.

Demonstrating this exercise, we will need one database along with two sample table before we start implementing actual service broker.

--create database.
CREATE DATABASE ExtremeAdviceSBDemo
GO

--making broker enable for database.
ALTER DATABASE ExtremeAdviceSBDemo SET ENABLE_BROKER
ALTER DATABASE ExtremeAdviceSBDemo SET TRUSTWORTHY ON;
ALTER AUTHORIZATION ON DATABASE::ExtremeAdviceSBDemo TO [sa];
GO

USE ExtremeAdviceSBDemo
GO

--two sample table for demonstration
CREATE TABLE OrderDetail
(
OrderID INT IDENTITY(1,1),
ClientID VARCHAR(10),
ProjectID VARCHAR(50),
OrderDate DATETIME DEFAULT GETDATE()
)
GO

CREATE TABLE OrderHistory
(
ID INT IDENTITY(1,1),
OrderID INT,
DuplicateWith INT,
CheckDate DATETIME DEFAULT GETDATE()
)
GO

Now, let us start service broker code. I will not explain each block of code here, if you wish, you can refer my earlier articles given in first paragraph here.

--Create Message Type
CREATE MESSAGE TYPE [EXTREME://EX/Message/Order/Request] VALIDATION = WELL_FORMED_XML
CREATE MESSAGE TYPE [EXTREME://EX/Message/Order/Response] VALIDATION = WELL_FORMED_XML
GO

--Create Contract
CREATE CONTRACT [EXTREME://EX/Contract/Order]
(
[EXTREME://EX/Message/Order/Request] SENT BY INITIATOR,
[EXTREME://EX/Message/Order/Response] SENT BY TARGET
);
GO

--Create Initiator queue reader SP
CREATE PROC [OrderInitiatorQueueReader]
AS
SET NOCOUNT ON;
DECLARE              @XML XML,
@MessageBody VARBINARY(MAX),
@MessageTypeName SYSNAME,
@ConversationHandle UNIQUEIDENTIFIER,
@Response XML

BEGIN TRY

WHILE (1 = 1)
BEGIN
BEGIN TRANSACTION
WAITFOR (
RECEIVE TOP(1)
@MessageTypeName = message_type_name,
@MessageBody = message_body,
@ConversationHandle = [conversation_handle]
FROM dbo.OrderInitiatorQueue
), TIMEOUT 60000

IF (@@ROWCOUNT = 0 AND @@TRANCOUNT>0)
BEGIN
ROLLBACK TRANSACTION
BREAK
END

IF @MessageTypeName = 'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog' BEGIN
END CONVERSATION @ConversationHandle;
END
COMMIT TRANSACTION
END
END TRY
BEGIN CATCH

IF @@TRANCOUNT > 0
ROLLBACK TRANSACTION
DECLARE @msg VARCHAR(MAX)
SELECT @msg = ERROR_MESSAGE()
DECLARE @line INT, @proc VARCHAR(MAX)
SELECT @line = ERROR_LINE(), @proc = ERROR_PROCEDURE()
SELECT @msg = 'proc: ' + @proc + '; line: ' + CAST(@line as varchar) + '; msg: ' + @msg

END CATCH
GO

--Create Initiator Queue
CREATE QUEUE OrderInitiatorQueue
WITH STATUS = ON,
ACTIVATION(
PROCEDURE_NAME = OrderInitiatorQueueReader,
MAX_QUEUE_READERS = 1,
EXECUTE AS OWNER
);
GO

-- Create actual business logic SP
CREATE PROCEDURE [CheckOrderMessage]
(
@data XML
)
AS
BEGIN TRY

DECLARE @MessageType VARCHAR(50)
,@OrderID INT
,@ClientID VARCHAR(10)
,@ProjectID VARCHAR(50)

SELECT  @MessageType = @Data.value('(OrderMessage/Header/@MessageType)[1]','VARCHAR(50)'),
@OrderID = @data.value('(OrderMessage/Header/@OrderID)[1]','INT'),
@ClientID = @Data.value('(OrderMessage/Header/@ClientID)[1]','VARCHAR(10)'),
@ProjectID = @data.value('(OrderMessage/Header/@ProjectID)[1]','VARCHAR(50)')

SELECT @MessageType = ISNULL(@MessageType,'')

IF @MessageType = 'OrderHistory' AND @OrderID IS NOT NULL BEGIN
DECLARE @OrderIDOld INT
,@ClientIDOld VARCHAR(10)
,@ProjectIDOld VARCHAR(50)

SELECT @ClientIDOld=ClientID, @ProjectIDOld=ProjectID, @OrderIDOld=OrderID
FROM OrderDetail
WHERE ClientID=@ClientID AND ProjectID=@ProjectID AND OrderID<>@OrderID

IF(@OrderIDOld IS NOT NULL)
BEGIN
INSERT INTO OrderHistory (OrderID,DuplicateWith)
SELECT @OrderIDOld,@OrderID
END
RETURN
END
END TRY

BEGIN CATCH
DECLARE @msg VARCHAR(MAX)
SELECT @msg = ERROR_MESSAGE()
END CATCH
RETURN
GO

--Create Target queue reader SP
CREATE PROC [OrderTargetQueueReader]
AS
SET NOCOUNT ON;

DECLARE              @XML XML,
@MessageBody VARBINARY(MAX),
@MessageTypeName SYSNAME,
@ConversationHandle UNIQUEIDENTIFIER,
@Response XML

BEGIN TRY
WHILE (1 = 1)
BEGIN
WAITFOR (
RECEIVE TOP(1)
@MessageTypeName = message_type_name,
@MessageBody = message_body,
@ConversationHandle = [conversation_handle]
FROM dbo.OrderTargetQueue
), TIMEOUT 60000

IF (@@ROWCOUNT = 0)
BEGIN
BREAK
END
SET @XML = CAST(@MessageBody AS XML);
IF @MessageTypeName = 'EXTREME://EX/Message/Order/Request'
BEGIN
--PRINT 'executing CheckOrderMessage @XML'
EXECUTE CheckOrderMessage @XML

--PRINT 'sending response'
SELECT @xml = 'OK';
SEND ON CONVERSATION @ConversationHandle
MESSAGE TYPE [EXTREME://EX/Message/Order/Response](@xml);
--PRINT 'ending cnversation'
END CONVERSATION @ConversationHandle;
END
END
END TRY
BEGIN CATCH
DECLARE @msg VARCHAR(MAX)
SELECT @msg = ERROR_MESSAGE()
DECLARE @line INT, @proc VARCHAR(MAX)
SELECT @line = ERROR_LINE(), @proc = ERROR_PROCEDURE()
SELECT @msg = 'proc: ' + @proc + '; line: ' + CAST(@line as varchar) + '; msg: ' + @msg
SELECT @msg
END CATCH
GO

--Create Target Queue
CREATE QUEUE OrderTargetQueue
WITH STATUS = ON,
ACTIVATION(
PROCEDURE_NAME = OrderTargetQueueReader,
--You can change queue reader number based on your queue load
MAX_QUEUE_READERS = 1,
EXECUTE AS OWNER
);
GO

--------------Create SP which will be called from app
--or trigger or by any other way
--and activate service broker
CREATE PROCEDURE [SendOrderRequestMessage]
(
@data AS XML
)
AS
SET XACT_ABORT OFF;
BEGIN TRY
BEGIN TRANSACTION

DECLARE @DialogHandle UNIQUEIDENTIFIER;

BEGIN DIALOG CONVERSATION @DialogHandle
FROM SERVICE [EXTREME://EX/Service/Order/Initiator]
TO SERVICE N'EXTREME://EX/Service/Order/Target'
ON CONTRACT [EXTREME://EX/Contract/Order]
WITH ENCRYPTION = OFF;

SEND ON CONVERSATION @DialogHandle
MESSAGE TYPE [EXTREME://EX/Message/Order/Request](@data);
--END CONVERSATION @DialogHandle
COMMIT ;
END TRY
BEGIN CATCH
ROLLBACK TRAN;
DECLARE @msg VARCHAR(MAX);
SELECT @msg = ERROR_MESSAGE();
PRINT @msg
END CATCH;
GO

--Create Initiator Service
CREATE SERVICE [EXTREME://EX/Service/Order/Initiator]  ON QUEUE [OrderInitiatorQueue] ([EXTREME://EX/Contract/Order])
GO

--Create Target Service
CREATE SERVICE [EXTREME://EX/Service/Order/Target]  ON QUEUE [OrderTargetQueue] ([EXTREME://EX/Contract/Order])
GO

Now, we will define one trigger which will call SB to check whether order is duplicate or not every time we insert new record (not calling trigger for Update, you can do it if it is your business need)

-------service broker call from trigger.
CREATE TRIGGER trg_OrderDetail_Ins
ON  OrderDetail
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;

DECLARE @Data XML
SELECT @Data =
(
SELECT
'OrderHistory' AS 'Header/@MessageType',
OrderID AS 'Header/@OrderID',
ClientID AS 'Header/@ClientID',
ProjectID AS 'Header/@ProjectID'
FROM
(SELECT * FROM inserted) AS T
FOR XML PATH(''), ROOT('OrderMessage'), TYPE
)

EXEC [SendOrderRequestMessage] @Data
END
GO

Now, we are ready with service broker architecture and ready to test whether SB is working or not.

--Testing SB by inserting one record in OrderDetail Table
--which will execute Trigger and trigger will activate SB
INSERT INTO OrderDetail(ClientID,ProjectID)
SELECT 'TC01','NY Healthcare'
GO

--you will get one record which we have inserted above.
SELECT * FROM OrderDetail
GO

--you will get no record in OrderHistory table
--as we have not inserted duplicate record yet.
SELECT * FROM OrderHistory
GO

--inserting same record again so SB will detect it as duplicate order
--that will be inserted in ORderHistory table
INSERT INTO OrderDetail(ClientID,ProjectID)
SELECT 'TC01','NY Healthcare'
GO

--you will get one record in OrderHistory table
--as we have inserted duplicate record
SELECT * FROM OrderHistory
GO

I have not handled Error properly in CATCH block of Stored Procedure but just printed error message on screen because error handling is out of scope of this article, If you want to handle Error properly in Stored Procedure, have a look at my earlier article here.

If you like this article, do like “Extreme-Advice” page in Facebook.

Reference: Ritesh Shah

http://Extreme-Advice.com

http://www.sqlhub.com

Note: Microsoft Books online is a default reference of all articles.

Purge target Transmission queue of Service Broker in SQL Server

Last Sunday I have got very interesting question from one of my blog reader and before we discuss his question and solve it, let me give you some basic understanding regarding how message traverse between Initiator and Target Queue in very simple word by eliminating deep technicality.

When any message comes to service broker, message has to pass through various way including validation until it reaches to target queue and get processed.

Transmission Queue plays an important role in whole service broker architecture. Transmission Queue works as a “postman” between Initiator Queue and Target Queue.

Let me give you this understanding in simple words, without including deep technical explanation.

Suppose we have two server “S1” and “S2”. We have service broker implemented between these two servers. S1 is an initiator and S2 is a target. S1 sends message to process to S2 and S2 replies back to S1 via Transmission Queue.

Here is simple explanation in brief wording.

• S1 send message to process to S2 via Transmission Queue and starts conversation dialog.
• S2 receive message from S1’s Transmission Queue and acknowledge the same to S1 via S2’s Transmission Queue
• Now, S2 process the message and again acknowledge the same to S1 via S2’s Transmission Queue
• Finally message get processed and S2 send acknowledge of the same via Transmission Queue to S1 to finally close the conversation dialog
• When S1 closes the conversation, it acknowledge the same to S2 and message gets deleted from the transmission queue of S2.

Please note that, our task get completes in first three steps whereas fourth & fifth steps are formality to close open conversation in S1 & S2.

I have written articles to watching the message count currently stays inside different queues.

• Find Service Broker Queue Count in SQL Server (Click Here)
• Get Transmission queue row count in Service Broker (Click Here)

One of the blog reader has implemented the functions I have given in both of the above articles and found that his initiator and target queue always having few hundreds message in peak business hours and become even zero many times but his transaction queue in target server (S2 in our example) having millions of message unprocessed. He wanted me to help him in dealing with that huge queue.

Obviously I don’t have access to his server so I have to guess after getting certain information from him. I have asked him to execute following query in his S2 server ‘s database where he has setup service broker and transmission queue showing millions of records.

SELECT
'MessageNeedsToClose' AS ActionToTake,
COUNT(*) AS RowsCount
FROM
sys.transmission_queue
WHERE
message_body IS NULL
AND is_end_of_dialog = 1
AND enqueue_time < getdate() - 7
UNION ALL
SELECT
'MessageToKeep' AS ActionToTake,
COUNT(*) AS RowsCount
FROM
sys.transmission_queue
WHERE
message_body IS NOT NULL
GO

He sent me approx million+ records in “MessageNeedsToClose” and less then hundred count comes in “MessageToKeep”.

SELECT query before the UNION ALL clause checks where the row in transmission queue is having any message inside it or not. If Message_Body is NULL then there is no message at all. If “is_end_of_dialog” is 1 then that message is just an acknowledgment nothing else. So if there is no message to process in message_body and if message is just an acknowledgment, we can probably close it to fix the high number of queue one time, if message is already closed from initiator side.

Reader has already told me that transmission queue on initiator side (in server S1) always stays below 1000 and become zero also so there is no chance that the million message we have seen in S2 is still there open in S1.

Since I didn’t have access to his server to investigate further and the messages we have in target transmission queue is not a real business process, we can close that dialog. If I would have access to server, I would run profiler for broker in S1 as well as in S2 to see what is happening with that big transmission queue but the reader was already convinced and wanted to settled down by closing the conversation of dialog.

I have written one customized Stored procedure for him to create on Server S2 in the database where he has Service Broker setup.

CREATE PROCEDURE [CleanTransmissionQueueByEndingDialog]
AS
SET NOCOUNT ON
BEGIN
BEGIN TRY
DECLARE @ConvHandle UNIQUEIDENTIFIER

DECLARE conv CURSOR FOR

SELECT
S2TransQueue.conversation_handle
FROM
sys.conversation_endpoints AS S2ConvEnd WITH(NOLOCK)
INNER JOIN
sys.transmission_queue S2TransQueue WITH(NOLOCK)
ON
S2TransQueue.conversation_handle = S2ConvEnd.conversation_handle
AND S2TransQueue.message_body is null
AND S2TransQueue.is_end_of_dialog = 1
AND S2TransQueue.enqueue_time < getdate() - 7

--I HIGHLY recomment uncomment the following WHERE clause to check
--where same message is alive in initiator end or not
--if it is alive, we shouldn't close that message.

--WHERE NOT EXISTS(
-- SELECT 1
-- --S1 is a Linked Server, you have to replace it with your linked server of Initiator
-- --"DatabaseName" should be replace by your database name where you have setup
-- --Service Broker in Initiator Server.
-- FROM S1.DatabaseName.sys.conversation_endpoints S1ConvEnd with(nolock)
-- WHERE S1ConvEnd.conversation_id = S2ConvEnd.conversation_id
--)

OPEN conv
FETCH NEXT FROM conv INTO @ConvHandle

WHILE @@FETCH_STATUS = 0 Begin
END CONVERSATION @ConvHandle WITH cleanup
FETCH NEXT FROM conv into @ConvHandle
PRINT @ConvHandle
END

CLOSE conv
DEALLOCATE conv

END TRY

BEGIN CATCH
PRINT ERROR_MESSAGE()
END CATCH
END
GO

Please note that I have investigated this issue without accessing live server of blog reader. Whatever suggestions I have given here is going to affect your live production environment if you are executing this script in production environment so please give a second thought and do it at your own risk.

I have written some other articles related to service broker, if you are directly landed to this article and new to Service broker, have a look at following articles which covers Service Broker concept from scratch.

• Importance of Service Broker in SQL Server (Click Here)
• Introduction of Service Broker in SQL Server (Click Here)
• Enable Service Broker in SQL Server database (Click Here)
• Small demonstration of Service broker for beginners (Click Here)
• Find Service Broker Object like Contract, Message Type, Service Name, Queue Name in SQL Server (Click Here)
• Find Service Broker Queue Count in SQL Server (Click Here)
• Get Transmission queue row count in Service Broker (Click Here)

If you like this article, do like “Extreme-Advice” page in Facebook.

Reference: Ritesh Shah

http://Extreme-Advice.com

http://www.sqlhub.com

Note: Microsoft Books online is a default reference of all articles.