Use script to navigate web page?

There is a web site I get information from. The site displays pages
of items with individual checkboxes to select each item. There is
also a checkbox to select all the item checkboxes. And there is the
page numbers with Next and Previous.

What I would like to do is log in to the site and navigate to the page
I need, then automate (by script??) going through each page and
selecting the Select All Checkboxes On This Page box.

On the page, I can use Find to get to Next, tab once and hit Enter to
go to the next page. And I can use Find to get Select All, Shift+Tab
gets the checkbox before the words, and the spacebar selects the
checkbox. I can do this for a couple of pages - more than 10 and I
start wondering if I missed any! Plus I can't help but feel that a
script or other automatino can do this much faster than I can!

I do a bit of VBA macros in Word and Excel, and a few VB scripts (with
much help!). I'm wondering if a script would help here, or if another
way would be better - or if I should drop the idea of automating this
all together.

Ed

Jeremy
Wed May 09 16:38:19 CDT 2007

Have a look into AutoIt. This will be able to do what you want.
http://www.autoitscript.com/autoit3/

"Ed" <prof_ofwhat@yahoo.com> wrote in message
news:1178733093.797260.206820@y80g2000hsf.googlegroups.com...
> There is a web site I get information from. The site displays pages
> of items with individual checkboxes to select each item. There is
> also a checkbox to select all the item checkboxes. And there is the
> page numbers with Next and Previous.
>
> What I would like to do is log in to the site and navigate to the page
> I need, then automate (by script??) going through each page and
> selecting the Select All Checkboxes On This Page box.
>
> On the page, I can use Find to get to Next, tab once and hit Enter to
> go to the next page. And I can use Find to get Select All, Shift+Tab
> gets the checkbox before the words, and the spacebar selects the
> checkbox. I can do this for a couple of pages - more than 10 and I
> start wondering if I missed any! Plus I can't help but feel that a
> script or other automatino can do this much faster than I can!
>
> I do a bit of VBA macros in Word and Excel, and a few VB scripts (with
> much help!). I'm wondering if a script would help here, or if another
> way would be better - or if I should drop the idea of automating this
> all together.
>
> Ed
>

Ayush
Wed May 09 20:11:00 CDT 2007

[Ed]s message :
> There is a web site I get information from. The site displays pages
> of items with individual checkboxes to select each item. There is
> also a checkbox to select all the item checkboxes. And there is the
> page numbers with Next and Previous.
>
> What I would like to do is log in to the site and navigate to the page
> I need, then automate (by script??) going through each page and
> selecting the Select All Checkboxes On This Page box.
>
> On the page, I can use Find to get to Next, tab once and hit Enter to
> go to the next page. And I can use Find to get Select All, Shift+Tab
> gets the checkbox before the words, and the spacebar selects the
> checkbox. I can do this for a couple of pages - more than 10 and I
> start wondering if I missed any! Plus I can't help but feel that a
> script or other automatino can do this much faster than I can!
>
> I do a bit of VBA macros in Word and Excel, and a few VB scripts (with
> much help!). I'm wondering if a script would help here, or if another
> way would be better - or if I should drop the idea of automating this
> all together.
>
> Ed
>

Take a look at InternetExplorer.Application object. It can do what you want.


Good Luck, Ayush.
--
XP-Tips [Change the thumbnail picture of a folder] :
http://www.microsoft.com/windowsxp/using/setup/tips/folderpic.mspx

OfficeGuyGoesWild
Wed May 09 20:37:27 CDT 2007

On May 10, 5:51 am, Ed <prof_ofw...@yahoo.com> wrote:
> There is a web site I get information from. The site displays pages
> of items with individual checkboxes to select each item. There is
> also a checkbox to select all the item checkboxes. And there is the
> page numbers with Next and Previous.
>
> What I would like to do is log in to the site and navigate to the page
> I need, then automate (by script??) going through each page and
> selecting the Select All Checkboxes On This Page box.
>
> On the page, I can use Find to get to Next, tab once and hit Enter to
> go to the next page. And I can use Find to get Select All, Shift+Tab
> gets the checkbox before the words, and the spacebar selects the
> checkbox. I can do this for a couple of pages - more than 10 and I
> start wondering if I missed any! Plus I can't help but feel that a
> script or other automatino can do this much faster than I can!
>
> I do a bit of VBA macros in Word and Excel, and a few VB scripts (with
> much help!). I'm wondering if a script would help here, or if another
> way would be better - or if I should drop the idea of automating this
> all together.
>
> Ed


Hi,

A few components that may help are:

'Create instance of Internet explorer:
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True

'Navigate to web page
ie.Navigate("http://www.test.com/page.asp" )
Do While ie.Busy
Wscript.Sleep 500
Loop

'Get each line of the web page into an array
PageArray = Split(ie.Document.Body.InnerHTML,Chr(13))

'Do some logic to find the bits you are after
For x = 0 to (ubound(PageArray)-1)
If instr(PageArray(x),"thingyouarelookingfor") Then
'DoSomething
Exit For
End If
Next

Also.. using SendKeys may be useful:
ref:
http://www.thescriptlibrary.com/default.asp?Action=Display&Level=Category3&ScriptLanguage=VBScript&Category1=Scripting%20Techniques&Category2=Retrieving%20Input&Title=%20Method:%20%20WshShell.SendKeys


Regards,
Marty Lindsay
Editor
www.TheScriptLibrary.com

Brendan
Thu May 10 06:51:41 CDT 2007

The keystroke method you have suggested will work, but as Ayush above
has stated the best way to do this is to start an IE object in
VBScript. You can then navigate the page using the DOM i.e. Document
Object Model. All this means is that VBScript treats the webpage as an
object upon which you can operate. The Microsoft webpage for this is
here: http://msdn2.microsoft.com/en-us/library/ms533050.aspx

Using the DOM only works well on well-designed web pages that have
properly named elements. If there is a popup login window then you
have to use sendkeys. A useful tool is the IE DOM browser:
http://www.microsoft.com/downloads/details.aspx?familyid=E59C3964-672D-4511-BB3E-2D5E1DB91038&displaylang=en

If you comb the web a little you can find a lot of good examples. I
recently had to do this with only a little VBS and WSH experience and
although the learning curve is a bit steep I was happy with the
result.

If you want a quick fix then AutoIt or SendKeys is the way to go. I
don't have my code handy ATM but if you need to login to a popup
window I can dig it up. Keep us posted on your efforts.

Good Luck

Ed
Mon May 14 11:47:44 CDT 2007

Marty:

I used some of your sample code and some other stuff I found to piece
this together:

'Create instance of Internet explorer:
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
Stop

'Navigate to web page
ie.Navigate("http://www.google.com")
Do While ie.Busy
Wscript.Sleep 500
Loop

Stop

'Get each line of the web page into an array
PageArray = Split(ie.Document.Body.InnerHTML,Chr(13))

Const END_OF_STORY = 6
Const MOVE_SELECTION = 0

Set objWord = CreateObject("Word.Application")
objWord.Visible = True

Set objDoc = objWord.Documents.Open("C:\Documents and Settings
\edward.millis\Desktop\TestMe.doc")
Set objSelection = objWord.Selection
objSelection.EndKey END_OF_STORY, MOVE_SELECTION

objSelection.Range.Text= "Hi"
objSelection.TypeParagraph
objSelection.TypeParagraph

x=0
Do
objSelection.Range.Text= PageArray(x)
x= x+1
Loop


It works as far as it goes, and I guess I can play with it a bit more
to get some of the web page objects. But I really need to set the IE
object to the currently displayed web page, rather than use the
Navigate, because I don't always want to go to the same web page. I
will perform the same actions on different pages, but the pages
themselves will be different. Might you be able to help with that?

Ed

Ed
Mon May 14 11:51:59 CDT 2007

Marty:

I pieced together this from your code and some other sample stuff I
found:

'Create instance of Internet explorer:
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
Stop

'Navigate to web page
ie.Navigate("http://www.google.com")
Do While ie.Busy
Wscript.Sleep 500
Loop

Stop

'Get each line of the web page into an array
PageArray = Split(ie.Document.Body.InnerHTML,Chr(13))

Const END_OF_STORY = 6
Const MOVE_SELECTION = 0

Set objWord = CreateObject("Word.Application")
objWord.Visible = True

Set objDoc = objWord.Documents.Open("C:\Documents and Settings\UserName
\Desktop\TestMe.doc")
Set objSelection = objWord.Selection
objSelection.EndKey END_OF_STORY, MOVE_SELECTION

objSelection.Range.Text= "Hi"
objSelection.TypeParagraph
objSelection.TypeParagraph

x=0
Do
objSelection.Range.Text= PageArray(x)
x= x+1
Loop

It works. Thank you!!

But I really need to set an object to the currently displayed web
page, rather than navigate to a specific page. Can you help me do
this?

Ed

OfficeGuyGoesWild
Tue May 15 04:22:02 CDT 2007

On May 15, 4:51 am, Ed <prof_ofw...@yahoo.com> wrote:
> Marty:
>
> I pieced together this from your code and some other sample stuff I
> found:
>
> 'Create instance of Internet explorer:
> Set ie = CreateObject("InternetExplorer.Application")
> ie.Visible = True
> Stop
>
> 'Navigate to web page
> ie.Navigate("http://www.google.com")
> Do While ie.Busy
> Wscript.Sleep 500
> Loop
>
> Stop
>
> 'Get each line of the web page into an array
> PageArray = Split(ie.Document.Body.InnerHTML,Chr(13))
>
> Const END_OF_STORY = 6
> Const MOVE_SELECTION = 0
>
> Set objWord = CreateObject("Word.Application")
> objWord.Visible = True
>
> Set objDoc = objWord.Documents.Open("C:\Documents and Settings\UserName
> \Desktop\TestMe.doc")
> Set objSelection = objWord.Selection
> objSelection.EndKey END_OF_STORY, MOVE_SELECTION
>
> objSelection.Range.Text= "Hi"
> objSelection.TypeParagraph
> objSelection.TypeParagraph
>
> x=0
> Do
> objSelection.Range.Text= PageArray(x)
> x= x+1
> Loop
>
> It works. Thank you!!
>
> But I really need to set an object to the currently displayed web
> page, rather than navigate to a specific page. Can you help me do
> this?
>
> Ed

Hi,

If you create the instance of Internet Explorer [using Set ie =
CreateObject("InternetExplorer.Application")] then you can control
the instance. To get the value of the currently displayed page then
access it using this:

CurrentURL = ie.LocationURL


Regards.. Marty
www.TheScriptLibrary.com

Paul
Tue May 15 09:11:59 CDT 2007

----- Original Message -----
From: "Ed" <prof_ofwhat@yahoo.com>
Newsgroups: microsoft.public.scripting.vbscript
Sent: Monday, May 14, 2007 11:51 AM
Subject: Re: Use script to navigate web page?



>
> But I really need to set an object to the currently displayed web
> page, rather than navigate to a specific page. Can you help me do
> this?

Are you saying that you want to manually navigate to various web pages and
have the script extract info on demand? If so, then script two instances of
IE. In one, you do the navigating. In the other, you have a button, with
script to get and manipulate the other IE's document object when you click
the button.

With the shell object, you can attach to any IE window, but the script can't
get that IE window's current document object nearly as well as if your
script actually created that instance of IE. Perhaps a Guru can explain the
difference.

-Paul Randall

Ed
Tue May 15 16:08:42 CDT 2007

Hi, Marty.

> > But I really need to set an object to the currently displayed web
> > page, rather than navigate to a specific page. Can you help me do
> > this?
>
> > Ed
>
> Hi,
>
> If you create the instance of Internet Explorer [using Set ie =
> CreateObject("InternetExplorer.Application")] then you can control
> the instance. To get the value of the currently displayed page then
> access it using this:
>
> CurrentURL = ie.LocationURL


I apologize for being so slow on getting this. I got the following:

'Create instance of Internet explorer:
Set ie = CreateObject("InternetExplorer.Application")
'ie.Visible = True

CurrentURL = ie.LocationURL
MsgBox CurrentURL

It created a new instance of Explorer with no URL, so the MsgBox was
blank.

I tried changing to GetObject, and got a Null - Invalid Syntax error.

When I used this:

'Create instance of Internet explorer:
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
Stop

'Navigate to web page
ie.Navigate("http://www.google.com")
Do While ie.Busy
Wscript.Sleep 500
Loop

Stop

CurrentURL = ie.LocationURL
MsgBox CurrentURL

it found the page it had navigated to.

But I still can't run the script with a web page open and get the
currently displayed page. What am I missing?

Ed

OfficeGuyGoesWild
Tue May 15 17:26:22 CDT 2007

On May 16, 9:08 am, Ed <prof_ofw...@yahoo.com> wrote:
> Hi, Marty.
>
> > > But I really need to set an object to the currently displayed web
> > > page, rather than navigate to a specific page. Can you help me do
> > > this?
>
> > > Ed
>
> > Hi,
>
> > If you create the instance of Internet Explorer [using Set ie =
> > CreateObject("InternetExplorer.Application")] then you can control
> > the instance. To get the value of the currently displayed page then
> > access it using this:
>
> > CurrentURL = ie.LocationURL
>
> I apologize for being so slow on getting this. I got the following:
>
> 'Create instance of Internet explorer:
> Set ie = CreateObject("InternetExplorer.Application")
> 'ie.Visible = True
>
> CurrentURL = ie.LocationURL
> MsgBox CurrentURL
>
> It created a new instance of Explorer with no URL, so the MsgBox was
> blank.
>
> I tried changing to GetObject, and got a Null - Invalid Syntax error.
>
> When I used this:
>
> 'Create instance of Internet explorer:
> Set ie = CreateObject("InternetExplorer.Application")
> ie.Visible = True
> Stop
>
> 'Navigate to web page
> ie.Navigate("http://www.google.com")
> Do While ie.Busy
> Wscript.Sleep 500
> Loop
>
> Stop
>
> CurrentURL = ie.LocationURL
> MsgBox CurrentURL
>
> it found the page it had navigated to.
>
> But I still can't run the script with a web page open and get the
> currently displayed page. What am I missing?
>
> Ed

Hi,

What you ideally would do is:

start the script
script creates instance of Internet Explorer
Web browsing takes place [manually?]
script gets current url using ie.LocationURL
script does things with data from page
etc etc.

It is the fact that the script creates the Internet Explorer instance
that gives you most control of what subsequently goes on with that
instance. If you start Internet Explorer independently of the
script[ i.e by double clicking on the IE icon], then you have less or
no control of it.

clear as mud ? :)

.Marty
TheScriptLibrary.com

Ed
Tue May 15 17:39:44 CDT 2007

I think I see.

> What you ideally would do is:
>
> start the script
> script creates instance of Internet Explorer
> Web browsing takes place [manually?]
> script gets current url using ie.LocationURL
> script does things with data from page
> etc etc.
>
> It is the fact that the script creates the Internet Explorer instance
> that gives you most control of what subsequently goes on with that
> instance.

So I need two subs? One to create an IE instance which I manually
browse (which allows me to log in, find the correct page, and filter
things as needed), and another to do what I need to do? And because
I've manually browsed the script-created IE object, it can be worked
on by the script? Or will one sub do?

How do I either invoke the second sub with the IE object passed to
it? Or pause and restart the one sub?

Ed

Tim
Tue May 15 22:18:14 CDT 2007

Here's a VBA function which you can use

Eg

Dim o
set o = GetIE(http://www.google.com/)
'do stuff with o....




'*******************************************
'Find an IE window with matching location
' Assumes no frames.
Function GetIE(sAddress As String) As Object

Dim objShell As Object, objShellWindows As Object, o As Object
Dim retVal As Object, sURL As String


Set retVal = Nothing
Set objShell = CreateObject("Shell.Application")
Set objShellWindows = objShell.Windows

'see if IE is already open
For Each o In objShellWindows
sURL = ""
On Error Resume Next
sURL = o.document.Location
On Error GoTo 0
If sURL <> "" Then
If sURL Like sAddress & "*" Then
Set retVal = o
Exit For
End If
End If
Next o

Set GetIE = retVal
End Function
'*********************************

Tim


"Ed" <prof_ofwhat@yahoo.com> wrote in message
news:1179268784.212767.221250@l77g2000hsb.googlegroups.com...
>I think I see.
>
>> What you ideally would do is:
>>
>> start the script
>> script creates instance of Internet Explorer
>> Web browsing takes place [manually?]
>> script gets current url using ie.LocationURL
>> script does things with data from page
>> etc etc.
>>
>> It is the fact that the script creates the Internet Explorer instance
>> that gives you most control of what subsequently goes on with that
>> instance.
>
> So I need two subs? One to create an IE instance which I manually
> browse (which allows me to log in, find the correct page, and filter
> things as needed), and another to do what I need to do? And because
> I've manually browsed the script-created IE object, it can be worked
> on by the script? Or will one sub do?
>
> How do I either invoke the second sub with the IE object passed to
> it? Or pause and restart the one sub?
>
> Ed
>

Ed
Thu May 17 10:03:14 CDT 2007

With many thanks to all who have responded in both the Word VBA and
the Scripting newgroups (and with apologies for not having included
both groups from the beginning, because now I don't know how to join
the two threads!), I have a Word VBA macro that allows me to open an
instance of IE, browse to the page I need, and read information from
the page. What I can not do, though, is find and activate the two
controls I need. I want to find and check a checkbox, and then find
and activate the Next page link. I tried using SendKeys, but it
didn't work - I think perhaps because I was stepping through the macro
and the web page didn't have the focus. If anyone would like to point
me in the right direction, your further help is greatly appreciated.

Ed

Sub Try_IE()

Dim objIE
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True

'fmDoIE.Show

If MsgBox("Do you want to continue?", vbYesNo) = vbNo Then
objIE.Quit
Set objIE = Nothing
Exit Sub
End If

'MsgBox objIE.LocationURL

Dim objIE2
Set objIE2 = GetIE(objIE.LocationURL)
Dim docIE
Set docIE = objIE2.Document

'Get each line of the web page into an array
Dim PageArray
PageArray = Split(docIE.Body.InnerHTML, Chr(13))

Dim objDoc As Document
Dim docRng As Range
Dim parRng As Range
Set objDoc = Documents.Open("C:\Documents and Settings\edward.millis
\Desktop\TestMe.doc")
Set docRng = objDoc.Content

Dim x As Long, y As Long, z As Long
x = UBound(PageArray)

Stop

For z = 0 To x
y = docRng.Paragraphs.Count
Set parRng = objDoc.Paragraphs(y).Range
parRng.Text = PageArray(z) & Chr(13)
Next z

Set parRng = objDoc.Paragraphs(docRng.Paragraphs.Count).Range
parRng.Text = "^p" & "^p" & "^p"
Set parRng = objDoc.Paragraphs(docRng.Paragraphs.Count).Range

Stop

On Error GoTo EndLoop
Dim frmIE
For Each frmIE In docIE.Forms
parRng.Text = "Name:= " & frmIE.Name & "; " & "ID:= " & frmIE.ID &
vbCrLf
Set parRng = objDoc.Paragraphs(docRng.Paragraphs.Count).Range
Next frmIE

EndLoop:
On Error GoTo 0

'objIE2.Activate

SendKeys "^F", Wait
SendKeys "Clear checkboxes", Wait
SendKeys "{ENTER}", Wait
SendKeys "{ESC}", Wait
SendKeys "{TAB}", Wait
SendKeys " ", Wait


Stop

Set objIE = Nothing
Set objIE2 = Nothing
Set docIE = Nothing

'objDoc.Close wdDoNotSaveChanges

End Sub

'*******************************************
' huge thanks to Tim Williams for this function!

'Find an IE window with matching location
' Assumes no frames.
Function GetIE(sAddress As String) As Object


Dim objShell As Object, objShellWindows As Object, o As Object
Dim retVal As Object, sURL As String


Set retVal = Nothing
Set objShell = CreateObject("Shell.Application")
Set objShellWindows = objShell.Windows


'see if IE is already open
For Each o In objShellWindows
sURL = ""
On Error Resume Next
sURL = o.Document.Location
On Error GoTo 0
If sURL <> "" Then
If sURL Like sAddress & "*" Then
Set retVal = o
Exit For
End If
End If
Next o


Set GetIE = retVal
End Function