How-to: Automate user browsing from a text file containing URLs

In this blog I'll go through how I configure my lab environment so that computer/users will automatically connect to web servers via a text document on a schedule. The reason I set this up is to simulate constant 'real world' browsing from computers in the lab so that my web filtering reports are filled up when it comes time to demo to customers!

The main requirements to set this up include:

1. Download the wget program
2. Download or build your URL list
3. Create a script to run wget and read the URL list
4. Create a taskto run the script on a schedule.

Lucky for y'all I've already uploaded a file HERE that contains everything we need to get the ball rolling.

Download the zip file and extract the folder somewhere the user has run permissions to. For this example we'll use C:.

The folder contains the program (wget), the script (runme.bat) plus three URL text files that I've collected in the last few weeks. Please note that I haven't gone through and checked the safety of these URLs so please use at your own risk. Having said that the script will only connect to the websites, it won't download the pages.

The script contains the following command:


wget -i urls3.txt -t 1 -T 1 -w 1 --spider --read-timeout 1 --no-check-certificate

Change the 'urls3.txt' section bolded above to reflect the URL file you wish to use.

Let's run the script now to ensure that it's working correctly. Open up a command prompt run the script with the following commands:

cd /autobrowser
runme.bat

You'll now see the script begin to pull up the URLs and connect to them.

When you're satisfied that it's connecting correctly press Ctrl+C to cancel the script.

Now we need to create the scheduled task.

Click on Start and search for 'Task Scheduler' (highlighted). Run the program

Click on Action and select 'Create Basic Task'

Give the task a name and description.

Select how often we want the task to trigger. For my lab I want it to trigger once a day.

 Select the time of dayyou want the task to run.

 Select 'Start a program'

Enter the path+filename for the 'Program/script' field, and enter the path only for the 'Start in (optional) field.

Click finish to save.

You should now see your saved task, run time and author. You can always edit this to run as a different user or run when a user isn't logged in.

If you can't wait for the scheduled time to arrive, you can right click on the task and select 'Run' to run it immediately.

The web traffic should now be visable on the network. Here we can see two computers I've setup the script for (using different URL files), PPARKER and MJANE.