WebFetch::General - download and save headlines from other WebFetch sites

NAME

WebFetch::General - download and save headlines from other WebFetch sites

SYNOPSIS

In perl scripts:

use WebFetch::General;

From the command line:

perl -w -MWebFetch::General -e "&fetch_main" -- --dir directory --file output-filename --url source-url [--format format]

DESCRIPTION

This module gets the current headlines from any WebFetch site that exports its news with the ``WebFetch Export'' format. You can do this with the --export command-line parameter. It works for any WebFetch module that defines the export() function, which includes all the modules that come packaged with WebFetch.

The webmaster of a remote site only needs to arrange for a cron job to update a WebFetch Export file, and let others know the URL to reach that file. (On the exporting site, it is most likely they'll use WebFetch::SiteNews to export their own news.) Then you can use the WebFetch::General module to read the remote file and generate and HTML summary of the news.

After WebFetch::General runs, the file specified in the --file parameter will be created or replaced. If there already was a file by that name, it will be moved to a filename with ``O'' (for old) prepended to the file name.

FORMAT STRINGS

WebFetch::General uses a format string to generate HTML from the incoming data. The default format for retrieved data is

<a href=``%url%''>%title%</a>

This means that fields named ``url'' and ``title'' must exist in the incoming WebFetch-exported data, and will be used to fill in the %url% and %title% strings, respectively. You may use the --format parameter to specify any format you wish. But the field names you choose in the format must match fields defined in the input stream. Otherwise they will fail to be expanded.

THE ``WebFetch Export'' FILE FORMAT

This is an example WebFetch Export file generated by WebFetch::SiteNews:

    [WebFetch export]
    Version: 0.09
    # This was generated by the Perl5 WebFetch 0.09 module.
    # WebFetch info can be found at http://www.webfetch.org/
    #
    # Exported from WebFetch::SiteNews
    # "date" is the date of the news
    # "title" is a one-liner title
    # "url" is url to the news source
    # "text" is the news text


    date: August 15, 1999
    title: WebFetch 0.09 released
    url: http://www.webfetch.org/news.shtml#19990815-000
    text:   <a href="dist/WebFetch-0.09.tar.Z">WebFetch 0.09 beta</a> adds\
            a new LinuxTelephony module, contributed by Greg Youngblood\
            as well as modules for Linux Dev.Net, 32BitsOnline\
            and <a href="dist/Changes">other updates</a>


    date: August 1, 1999
    title: WebFetch 0.08 released
    url: http://www.webfetch.org/news.shtml#19990801-000
    text:   <a href="dist/WebFetch-0.08.tar.Z">WebFetch 0.08 beta</a>\
            adds a new DebianNews module contributed by Chuck Ritter,\
            as well as <a href="dist/Changes">other updates</a>


    date: August 1, 1999
    title: Slashdot lists Webfetch first among headlines tools
    url: http://www.webfetch.org/news.shtml#19990801-001
    text:   Many thanks to <a href="http://slashdot.org/";>Slashdot</a>\
            for listing WebFetch <i>first</i> among the\
            <a href="http://slashdot.org/code.shtml";>tools that can\
            display headlines from their site</a>.

Each news item is separated by a blank line.

Within each news item, the fields use a ``name: value'' format, similar to RFC822 headers (i.e. as in e-mail and news.)

The names of the fields are chosen by the exporting module. Though for the convenience of the user, the author of an exporting module should keep in mind the default WebFetch::General format uses fields called ``url'' and ``title''. If you use fields by different names, warn your receiving users that they will need to make a format to use with WebFetch::General, though you may provide them with one in their setup instructions.

AUTHOR