edu.duke

Class URLResource

java.lang.Object

edu.duke.URLResource

public class URLResource
extends java.lang.Object

The URLResource class opens a connection to a URL and allows access to the contents of the web page a line at a time, using the method lines, or a word at a time, using the method words. These strings can then be iterated over using a for loop.

Example usage:

 URLResource ur = new URLResource("http://www.dukelearntoprogram.com/");
 for (String s : ur.lines()) {
     // print or process s
 }
 

If each line of the web page represents separated data values, because its a CSV file, then the user can get a getCSVParser object to access that data more directly, using one of the getCSVParser methods.

Example CSV usage:

 URLResource ur = new URLResource("http://www.dukelearntoprogram.com/course2/java/food.csv");
 for (CSVRecord record : ur.getCSVParser()) {
     // print or process fields in record
     String name = record.get("Name");
     // other processing
 }
 

This software is licensed with an Apache 2 license, see http://www.apache.org/licenses/LICENSE-2.0 for details.

Constructor Summary

Constructors

Constructor and Description
URLResource(java.lang.String name) Create a URLResource object bound to the web page whose URL is given as the parameter.

Method Summary

Modifier and Type	Method and Description
java.lang.String	asString() Return entire open web page as one string.
java.lang.Iterable<java.lang.String>	getCSVHeaders(CSVParser parser) Allows access to the column names of the header row of a CSV file (the first line in the file) one at a time.
CSVParser	getCSVParser() Returns a CSVParser object to access the contents of an open web page.
CSVParser	getCSVParser(boolean withHeader) Returns a CSVParser object to access the contents of an open web page, possibly without a header row.
CSVParser	getCSVParser(boolean withHeader, java.lang.String delimiter) Returns a CSVParser object to access the contents of an open web page, possibly without a header row and a different data delimiter than a comma.
java.lang.Iterable<java.lang.String>	lines() Allow access to open web page one line at a time.
java.lang.Iterable<java.lang.String>	words() Allow access to this open web page one word at a time, where words are separated by white-space.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

URLResource

public URLResource(java.lang.String name)

Create a URLResource object bound to the web page whose URL is given as the parameter. Constructing the object opens a connection and reads the contents of the web page.

Parameters:

name - is the name of the URL, it must start with "http" or "https"

Throws:

exception - if the URL does not start with "http" or "https"

Method Detail

lines

public java.lang.Iterable<java.lang.String> lines()

Allow access to open web page one line at a time.

Returns:

an Iterable that allows access one line at a time

words

public java.lang.Iterable<java.lang.String> words()

Allow access to this open web page one word at a time, where words are separated by white-space. This means any form of spaces, like tabs or newlines, can delimit words.

Returns:

an Iterable that allows access one word at a time

asString

public java.lang.String asString()

Return entire open web page as one string.

Returns:

a String that is the contents of the open web page

getCSVParser

public CSVParser getCSVParser()

Returns a CSVParser object to access the contents of an open web page. Each line of the web page should be formatted as data separated by commas and with a header row to describe the column names.

Returns:

a CSVParser that can provide access to the records in the web page one at a time

Throws:

exception - if this web page does not represent a CSV formatted data

getCSVParser

public CSVParser getCSVParser(boolean withHeader)

Returns a CSVParser object to access the contents of an open web page, possibly without a header row. Each line of the web page should be formatted as data separated by commas and with/without a header row to describe the column names.

Parameters:

withHeader - uses first row of data as a header row only if true

Returns:

a CSVParser that can provide access to the records in the web page one at a time

Throws:

exception - if this web page does not represent a CSV formatted data

getCSVParser

public CSVParser getCSVParser(boolean withHeader,
                              java.lang.String delimiter)

Returns a CSVParser object to access the contents of an open web page, possibly without a header row and a different data delimiter than a comma. Each line of the web page should be formatted as data separated by the delimiter passed as a parameter and with/without a header row to describe the column names. This is useful if the data is separated by some character other than a comma.

Parameters:

withHeader - uses first row of data as a header row only if true

delimiter - a single character that separates one field of data from another

Returns:

a CSVParser that can provide access to the records in the web page one at a time

Throws:

exception - if this web page does not represent a CSV formatted data

exception - if delimiter.length() != 1

getCSVHeaders

public java.lang.Iterable<java.lang.String> getCSVHeaders(CSVParser parser)

Allows access to the column names of the header row of a CSV file (the first line in the file) one at a time. If the CSV file did not have a header row, then an empty Iterator is returned.

Parameters:

parser - the CSVParser that has been created for this web page

Returns:

an Iterable that allows access one header name at a time