The isp definitions are written in JavaScript Object Notation. Version 2 of isps.js is designed for greater flexibility in the procedure which data is obtained from the ISP website and offers more customisation of ISP-specific options. The API allows for:
Creating a definition involves declaring the simple steps for the browser to take, and any regular expression to parse into the vars array.
An identifier for the ISP. This should be unique.
The full name of the ISP
The unit for the numbers printed by the ISP's usage page. Allowable values are UNIT_B, UNIT_KB, UNIT_MB. This is for integer auto conversion (See below).
Allows the ISP to have custom preference values that will be shown on the net usage preference dialog. It is an associative array in the format of the following:
{ '<variable name>': ['<display name>', <type>], ... }
<variable name> is the name of variable in which the value will be written to.
'<display name>' is the label shown next to the input box on the preference dialog
<type> denotes the type of variable can be one of PREF_TYPE_INT PREF_TYPE_STRING PREF_TYPE_BOOL
Special variables USERNAME and PASSWORD allows the label next to the username and password boxes changed. Since they are always of type string, there's only one string parameter.
'USERNAME': '<display name>',
'PASSWORD': '<display name>',
An associative array of default variables to be inserted into this.vars array in the form of:
{ '<variable name>': '<value>', ... }
An array of the steps required to log into the ISPs usage page. Each step consists of 3 array elements.
At the very end of the whole process array is the post process function. The number of elements in the final process array should be (number of steps * 3) + 1.
function (vars) { }, // preprocess function
'', // The web access command
<extraction patterns>, // parse variables could be regexp, or an associative array of variable to xpaths
...
function (vars) { } // post process function
The pre/post-process functions take a vars parameter which is the central associative array containing all preference and variables extracted from the previous steps. If the function sets a errortext variable inside the vars array, then fetch process will yeild an error, with the string specified in the errortext varible.
Any desired numeric manipulation on variables can be performed in the preprocess and postprocess sections of this file
An example of a postprocess function to perform a subtraction is shown below
function (vars) {
if (vars['2'] == 0) { // passed through process step 2 - regexp branch 0
vars['free-download'] = parseInt(vars['2:2'], 10) - parseInt(vars['2:1'], 10);
}
}
The above code will take the first two variables from the regexp at step 2, subtracting the first variable from the second [2:2] - [2:1]. The result is stored in the free-download variable. Modifications made to a variable in a postprocess function will override any assignments in the variable mapping section.
The web access command is a string in the form of:
<Request Method> <URL> [<Post Data>]
Possible values for Request Method are:
Variable substitution also takes place for vars named in {}
Examples:
POST http://imaginaryisp.com.au/cgi-bin/login.pl username={USERNAME}&password={PASSWORD}
GET http://imaginaryisp.com.au/cgi-bin/usage.pl
GET http://imaginaryisp.com.au/cgi-bin/logout.pl
The <extraction patterns> can be a regexp or an associative array of variable to xpaths, or also an array of regexp and/or xpaths. Each regexp or xpath will be tested in turn until a matching pattern is found. For the xpath, a match is determined by the existing of xpath specified by the XPATHVERIFY key in the xpath associative array. If there are multiple regexp patterns or multiple sets of xpaths, then this steps has multiple branches. The first xpathset or regexp will be branch 0. The second will be branch 1 and so fourth.
Branching determines the next step number taken by net usage process should a certain parsing branch be successful. Setting the value to null indicates the process should stop when it reached that particular branch. This allows the process to abort if we have reached an ISP error page. Examples include invalid username and password.
{
'0,1': null,
'1,2': 2
}
In addition to a null value, assigning a value of 100 to a branch indicates a successful scrape of the usage page, and a more complicated process may have multiple branches to exit from. The example below illustrates process with a number of successful completion points.
{
'0,0': null,
'1,0': 2,
'2,0': 100,
'2,1': 100,
'1,1': 3,
'3,0': 100,
}
In this example a usage page is successfully retrieved and scraped if we get to Step 2 - Branch 0, or Step 2 - Branch 1, or Step 3 - Branch 0. Using this method a more complex ISP usage page can be managed by a single isp entry in the file.
A 2D associative array for copying variables of this.vars in the form of:
{
'<step>,<branch>': {
'<destination name>': '<source name>',
...
}
...
}
If the destination variable name already exists. No copying will occur. This allows variables to be overridden by preferences or process function that occurred previously.
While you can name your var any specific name, these are the vars you should use in relation to usage numbers. The names suffixed with po signifies off-peak. free suffix signifies usage belonging a free stream. Not every ISP will have those. Usage is the actual amount of MB charged, which is normally just download, but also in some rare cases, upload plus download (i.e. Telstra).
download download-quota upload upload-quota usage usage-quota downloadpo downloadpo-quota uploadpo uploadpo-quota usagepo usagepo-quota free-download free-download-quota free-upload free-upload-quota free-usage free-usage-quota free-downloadpo free-downloadpo-quota free-uploadpo free-uploadpo-quota free-usagepo free-usagepo-quota
If the ISP has irregular usage cycles, start date and end date of the cycle may need to be pulled from the usage page. Using the special vars (startdate, enddate, rolldate), the date will automatically be parsed. You can specify either of those three or both startdate and enddate. If none is specified, it will be assumed that startdate is the 1st of this month and enddate is the last day of the month.
rolldate specifies the day (e.g. 21) OR the full date (e.g. 21/12/2006) of when the next cycle will begin.
This is a 2D associative array of variables that will be displayed on the tooltip.
{
'<step>,<branch>': {
'<entry name>': '<entry value>',
...
}
...
}
<entry value> can include variables by enclosing the variable name in parenthesis. Eg
'Download': '{download} / {download-quota}',
will display: Download 1000 / 2000
Note that the / character above is not a numeric operator on the two variables. Numeric manupipulation on variables can be performed in the preprocess or postprocess areas, as described in the process section of this document.
^critter <lpgcritter at nasquan dot com>
benniee <benniee at gmail dot com>