DTS Importing Data from Web page

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2000 Forums

Import/Export (DTS) and Replication (2000)

DTS Importing Data from Web page

Author

Topic

KTL_DK
Starting Member

15 Posts

Posted - 2003-09-10 : 06:27:56

I'm doing a DTs that should read Data from a Web page, for instance
a web page that have a table . This web page is online (www.site.com\table.asp) , and i pretend to connect to it once a week to extract the data in order to do some inserts in my Database .
Could someone tell me how i do this, i now it's possible because a long time ago i made something similar , but i don't remenber very well.

thanks

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2003-09-10 : 12:05:11

I still don't understand why you need to do this in DTS.

Tara

KTL_DK
Starting Member

15 Posts

Posted - 2003-09-10 : 12:12:07

Hello,

In case you didn't noted , this is a diferent issue from the other post that it's focused in directing info to a web page.
Now i need to do other thing , and in my opinion totaly diferent.
However if you can do the objective that i propouse , without dts's , go ahead and say how, ok?
The objective is to retrieve all the data from an online web page,and insert it on a Database. So asume that it's a normal web page, asp, that displays an html table.

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2003-09-10 : 12:13:38

Why can't your ASP just insert the data into a database?

Tara

KTL_DK
Starting Member

15 Posts

Posted - 2003-09-10 : 12:17:39

Because the website isn't mine !
It's an external website, an online website that i don't
control and i want to go there and retrieve the data.

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2003-09-10 : 12:51:40

What you are trying to do is called screen scraping. This can be illegal.

[url]http://devguru.411asp.net/home/tutorial/howto/networkf/httpfunc?cob=devguru[/url]

Tara

KTL_DK
Starting Member

15 Posts

Posted - 2003-09-10 : 13:21:56

ilegal????????????????????????????????

And if the site belongs to my company , they gave me access to the data, but they still whant that i access trogh the web, don't want to
grant access to the server or to the data from no other source than the one that i mentioned?

that assumptions that you make are very strange,so you shouldn't make them, so i repeat you should focus on the technical issue nothing more.

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2003-09-10 : 13:27:07

You said that you didn't own the page, so I was just mentioning that certain screen scraping is illegal. I wasn't saying that what you were specifically doing was illegal because I didn't even have enough information. The link that I provided gives you examples of how to do it.

Why doesn't your company create a web service to do this?

Tara

Stoad
Freaky Yak Linguist

1983 Posts

Posted - 2003-09-10 : 13:57:07

Oh... exactly the same way

dim h, s
set h=createobject("microsoft.xmlhttp")
h.open "get", "yourURL", 0
h.send
s=h.responseText
set h=nothing

msgbox s ' here you'll see the HTML Code of the page,
' the same code as you can see in IE >> View >> HTML Code
' after loading the page

How to parse it... tedious but easy task. You know what I
mean... table tags (<tr></tr> and so on). Pity I have no
ready-to-use parsing VB Script code.

oitsubob
Yak Posting Veteran

70 Posts

Posted - 2004-06-07 : 13:15:24

I'm trying to do something similar. See my thread in the developer forum: http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=35900

What I have is an ActiveX Script Task in a DTS package using the following code:

Function Main()
dim http, httpresult
set http=createobject("microsoft.xmlhttp")
http.open "get", "http://finance.yahoo.com/d/quotes.csv?s=^DJI&f=sl1d1t1c1ohgv&e=.csv", 0
http.send
set httpresult=http.responseText
set http=nothing
msgbox httpresult
Main = DTSTaskExecResult_Success
End Function

The good news is that based on the error, SQL is reaching out and pulling in csv of stock values. The bad news is, the step is failing. The error is as follows:

Error Code: 0
Error Source= Microsoft VBScript Runtime Error
Error Description: Object required: '[string: """^DJI",10333.38, "6/7"]'

I'm not exactly sure what this means. Any ideas?

Thanks,

Bob

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources