Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 Cursors are bad m'kay?

Author  Topic 

theboyholty
Posting Yak Master

226 Posts

Posted - 2012-03-22 : 07:24:54
I almost never hear anyone say anything good about cursors, rather people refer to 'set based' programming as a preferred alternative. Truth is, I don't really know what is meant by set-based operations (although I may find that it is what I have been using all this time).

I would personally use a WHILE loop to iterate through a dataset because I find the syntax easy to implement and performance seems to be ok generally speaking (although I'm using fairly small datasets), but would be interested to hear how other experienced SQL Developers feel on the subject.

Cheers.

---------------------------------------------------------------------------------
http://www.mannyroadend.co.uk A Bury FC supporters website and forum

GilaMonster
Master Smack Fu Yak Hacker

4507 Posts

Posted - 2012-03-22 : 08:49:10
A cursor is a while loop by another name, nothing more. Any form of looping is inefficient, whether it be cursor, while loop or other mechanism.

Set based programming is when you process a set of data in a single operation instead of looping over a set of rows and processing them one by one.

--
Gail Shaw
SQL Server MVP
Go to Top of Page

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2012-03-22 : 09:21:11
Read this Classic explanation by Rob

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=118835#466749


KH
[spoiler]Time is always against us[/spoiler]

Go to Top of Page

theboyholty
Posting Yak Master

226 Posts

Posted - 2012-03-23 : 05:46:47
That's a really useful explanation, but I still can't see a better way of doing what I need to do, through set based SQL.

I generate tables of up to a few thousand rows of data, and in each row we've built a URL for posting data to our clients' external systems. This is a stripped down version of the code.


DECLARE @c INT = 1, @PostURL VARCHAR(1000)
WHILE @c <= (SELECT COUNT(*) FROM FM_Magazine_WebSelection)
BEGIN
SET @PostURL = (SELECT PostURL FROM FM_Magazine_WebSelection WHERE RowNumber = @c)
EXEC usp_CLR_PostURL @PostURL
SET @c = @c + 1
END


As you can see, it loops through each row of the dataset, and individually fires the CLR proc for each URL string generated. I'd like to see if this can be done without the loop if possible.

---------------------------------------------------------------------------------
http://www.mannyroadend.co.uk A Bury FC supporters website and forum
Go to Top of Page

GilaMonster
Master Smack Fu Yak Hacker

4507 Posts

Posted - 2012-03-23 : 07:06:10
What does the CLR proc do? Turning that set-based will involve changing the procedure somehow, hence will need to see the code.

It's worth noting that CLR procs are always iterative, that's the nature of .Net, and there are some things (like sending email) that can't be done set-based. Those are exceptions and should not be the norm.

--
Gail Shaw
SQL Server MVP
Go to Top of Page

theboyholty
Posting Yak Master

226 Posts

Posted - 2012-03-26 : 11:16:50
Thanks for having a look at this. Here's the vb.net code which basically posts a URL to the web to send data to an associate's database via the web.

An example of the parameter that might be passed to it is as follows :
htp://leadintelrg.com/leads?cid=45843&siteid=340&title=Mr&first_name=John&last_name=Smith&email_address=js@test.com&telephone_number=01234567890
(by the way it does really begin with 'http://' but when I submitted my response it formatted it weirdly)

Imports System
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports Microsoft.SqlServer.Server
Imports System.Data.Sql
Imports System.IO
Imports System.Net
Imports System.Text
Imports System.Web
Imports System.Collections
Imports System.ComponentModel
Imports System.Text.RegularExpressions
Imports System.Diagnostics
Imports System.Web.Services


Partial Public Class StoredProcedures
<Microsoft.SqlServer.Server.SqlProcedure()> _
Public Shared Sub usp_CLR_PostURL(ByVal InStr As String)
Dim request As HttpWebRequest

Dim response As HttpWebResponse = Nothing
Dim reader As StreamReader
Dim URLString As String
Dim address As Uri
Dim returnstring As String

URLString = InStr
address = New Uri(URLString)
request = DirectCast(WebRequest.Create(address), HttpWebRequest)
request.Timeout = 1000000
response = DirectCast(request.GetResponse(), HttpWebResponse)
reader = New StreamReader(response.GetResponseStream())
returnstring = reader.ReadToEnd

Using connection As New SqlConnection("context connection=true")
connection.Open()

Dim command As New SqlCommand("SELECT '""" & Replace(returnstring, Chr(39), "") & """'", connection)
SqlContext.Pipe.ExecuteAndSend(command)

End Using

End Sub
End Class


Thanks.

---------------------------------------------------------------------------------
http://www.mannyroadend.co.uk A Bury FC supporters website and forum
Go to Top of Page

robvolk
Most Valuable Yak

15732 Posts

Posted - 2012-03-26 : 12:15:26
I'm not a .Net programmer, but it looks like you are passing a URL to the CLR procedure, requesting the the response from that URL (getting the web page), and then returning the contents of that page, is that correct?

One thing I can say for certain, there's no need to open a SQLContext.Pipe just to do a simple text replace, and it's definitely NOT a good way to do that.
Go to Top of Page

Vinnie881
Master Smack Fu Yak Hacker

1231 Posts

Posted - 2012-03-26 : 12:56:22
The proper way to accomplish the task in my opinion is the following.

1:vb.net
Initiate a Procedure call to return all URL's (ONE SINGLE CALL)

2:SQL write the procedure to return all information

Vb.net
3:Return ALL data to VB.NET and build a XML file (SOAP/WEBSERVICE/ETC)

4:POST ENTIRE XML file with one request/response and PARSE appropriately, if this is not a option due to you do nothave proper access to the host computer, then open 1 web request and loop through the posts, so you are not opening and closing web connections each time.




Success is 10% Intelligence, 70% Determination, and 22% Stupidity.
\_/ _/ _/\_/ _/\_/ _/ _/- 881
Go to Top of Page

yosiasz
Master Smack Fu Yak Hacker

1635 Posts

Posted - 2012-03-26 : 16:32:19
cursor loops have been good to me when used with set based processing
This is what I have used successfully


do you have a key column on your source data?

1. create temp db

create TABLE [dbo].migration_temp(KeyField int not null)
GO

2. Create a processing table that holds all rows that you want to move based on a key field

create table dbo.Process(keyFieldId int NOT NULL, isProcessed bit NOT NULL default((0)))


3. Dump into Process all unique keyField values

insert into dbo.Process(keyFieldID)
SELECT keyFieldID FROm SourceTableView


4.dump batch rows into temp table

WHILE (SELECT COUNT(*) FROM dbo.Process WHERE isProcessed = 0) > 0

INSERT INTO dbo.migration_temp(KeyField)
SELECT TOP 100 KeyFieldID
FROM dbo.Process with (nolock)
WHERE isProcessed = 0



6. Batch process your code

7. Update process table

UPDATE tgt
SET tgt.isProcessed = 1
FROM dbo.Process tgt with (nolock)
inner join dbo.migration_temp src
on src.KeyFieldID= tgt.KeyFieldID


<><><><><><><><><><><><><><><><><>
If you don't have the passion to help people, you have no passion
Go to Top of Page

Vinnie881
Master Smack Fu Yak Hacker

1231 Posts

Posted - 2012-03-26 : 21:34:47
yosiasz,

I am not sure if your approach will add any performance benefit to what he already has, also in just a note, in your while loop it may be best to only check the count one time and set it to a variable and increase a counter in the loop to avoid needing to run that (Select Count(*)... ) each time.


I would strongly suggest to use a method to return all data via one db call and post via a single webrequest. I have done numerous software apps prior using .net code where the data is derived from via databases, and for me it has provided drastic improvements over individually processing each record re-initiating all connections and everything else every time.

There are situations where this can not be accomplished if the receiving computer is something that you do not have control over to modify to handle the post, but even in that scenario you can query all the data you need in one call, open a web connection to the posting site and quickly post all data in a loop on the .net end.

It is just my opinion, but the current approach you have illustrated I would personally not use.


Success is 10% Intelligence, 70% Determination, and 22% Stupidity.
\_/ _/ _/\_/ _/\_/ _/ _/- 881
Go to Top of Page

yosiasz
Master Smack Fu Yak Hacker

1635 Posts

Posted - 2012-03-27 : 01:16:32
Vinnie I agree with you what I provided might not add any performance approach. But it depends what you are trying to accomplish. there is also the option you mention which is one call that can return xml/json data that can be processed in the .net side of things. that is by far the best approach for his situation

<><><><><><><><><><><><><><><><><>
If you don't have the passion to help people, you have no passion
Go to Top of Page
   

- Advertisement -