[Python-modules-commits] r2168 - in
/packages/urlgrabber/trunk/debian: ./ README.Debian changelog
compat control copyright dirs docs postinst rules urlgrabber.1
urlgrabber.txt urlgrabber.xml watch
rustybear-guest at users.alioth.debian.org
rustybear-guest at users.alioth.debian.org
Tue Apr 24 15:44:22 UTC 2007
Author: rustybear-guest
Date: Tue Apr 24 15:44:22 2007
New Revision: 2168
URL: http://svn.debian.org/wsvn/python-modules/?sc=1&rev=2168
Log:
[svn-inject] Applying Debian modifications to trunk
Added:
packages/urlgrabber/trunk/debian/ (with props)
packages/urlgrabber/trunk/debian/README.Debian
packages/urlgrabber/trunk/debian/changelog
packages/urlgrabber/trunk/debian/compat
packages/urlgrabber/trunk/debian/control
packages/urlgrabber/trunk/debian/copyright
packages/urlgrabber/trunk/debian/dirs
packages/urlgrabber/trunk/debian/docs
packages/urlgrabber/trunk/debian/postinst
packages/urlgrabber/trunk/debian/rules (with props)
packages/urlgrabber/trunk/debian/urlgrabber.1
packages/urlgrabber/trunk/debian/urlgrabber.txt
packages/urlgrabber/trunk/debian/urlgrabber.xml
packages/urlgrabber/trunk/debian/watch
Propchange: packages/urlgrabber/trunk/debian/
------------------------------------------------------------------------------
mergeWithUpstream = 1
Added: packages/urlgrabber/trunk/debian/README.Debian
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/README.Debian?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/README.Debian (added)
+++ packages/urlgrabber/trunk/debian/README.Debian Tue Apr 24 15:44:22 2007
@@ -1,0 +1,7 @@
+urlgrabber for Debian
+---------------------
+
+The files keepalive.py and byterange.py are generic urllib2 extension modules
+and can be used to add keepalive and range support to any urllib2 application.
+
+ -- Kevin Coyner <kevin at rustybear.com> Fri, 6 Apr 2007 22:01:01 -0400
Added: packages/urlgrabber/trunk/debian/changelog
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/changelog?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/changelog (added)
+++ packages/urlgrabber/trunk/debian/changelog Tue Apr 24 15:44:22 2007
@@ -1,0 +1,58 @@
+urlgrabber (3.1.0-3) unstable; urgency=low
+
+ * debian/control: Added python modules packaging team to uploaders and added
+ VCS fields.
+
+ -- Kevin Coyner <kevin at rustybear.com> Mon, 09 Apr 2007 19:27:36 -0600
+
+urlgrabber (3.1.0-2) unstable; urgency=low
+
+ * debian/control: Changed "Architecture: any" to all.
+
+ -- Kevin Coyner <kevin at rustybear.com> Mon, 09 Apr 2007 15:20:02 -0600
+
+urlgrabber (3.1.0-1) unstable; urgency=low
+
+ * New upstream release.
+ * New maintainer. (Closes: #418095)
+ * Added man page.
+ * Cleaned up cruft in debian/rules.
+ * Rewrote debian/copyright.
+ * Cleaned up debian/control and added homepage.
+ * Added debian/README.Debian.
+ * Added debian/postinst to clean up unneeded docs that were inappropriately
+ added in previous versions.
+ * Removed unneeded debian/pycompat file.
+
+ -- Kevin Coyner <kevin at rustybear.com> Fri, 06 Apr 2007 22:27:03 -0400
+
+urlgrabber (2.9.9-1) unstable; urgency=low
+
+ * New upstream release
+ * Apply Ana Beatriz Guerrero Lopez's patch to
+ * Update to new Python policy (Closes: #373402)
+ * Switch to standards version 3.7.2
+ * Update to debhelper compat level 5
+ * Thanks Ana!
+
+ -- Anand Kumria <wildfire at progsoc.org> Thu, 6 Jul 2006 09:16:37 +1000
+
+urlgrabber (2.9.7-2) unstable; urgency=low
+
+ * When I imported urlgrabber into bzr, I somehow lost a Build-Dep: on
+ python. Re-adding it so I can (Closes: #335340)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sat, 31 Dec 2005 15:34:22 +1100
+
+urlgrabber (2.9.7-1) unstable; urgency=low
+
+ * New upstream release (Closes: #344934)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sat, 31 Dec 2005 15:34:22 +1100
+
+urlgrabber (2.9.6-1) unstable; urgency=low
+
+ * Initial release (Closes: #312698)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sun, 9 Oct 2005 13:06:55 +1000
+
Added: packages/urlgrabber/trunk/debian/compat
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/compat?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/compat (added)
+++ packages/urlgrabber/trunk/debian/compat Tue Apr 24 15:44:22 2007
@@ -1,0 +1,1 @@
+5
Added: packages/urlgrabber/trunk/debian/control
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/control?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/control (added)
+++ packages/urlgrabber/trunk/debian/control Tue Apr 24 15:44:22 2007
@@ -1,0 +1,27 @@
+Source: urlgrabber
+Section: python
+Priority: optional
+Maintainer: Kevin Coyner <kevin at rustybear.com>
+Uploaders: Debian Python Modules Team <python-modules-team at lists.alioth.debian.org>
+Build-Depends: debhelper (>= 5), python-all-dev (>= 2.3.5-11), python-support (>= 0.6)
+Standards-Version: 3.7.2
+XS-Vcs-Svn: svn://svn.debian.org/python-modules/packages/urlgrabber/trunk/
+XS-Vcs-Browser: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/?op=log
+
+Package: python-urlgrabber
+Architecture: all
+Depends: ${shlibs:Depends}, ${misc:Depends}, ${python:Depends}
+Provides: ${python:Provides}
+Description: A high-level cross-protocol url-grabber
+ urlgrabber dramatically simplifies the fetching of files. It is designed to
+ be used in programs that need common (but not necessarily simple)
+ url-fetching features. This package provides both a binary and a module, both
+ of the name urlgrabber.
+ .
+ It supports identical behavior for http://, ftp:// and file:/// URIs. It
+ provides HTTP keepalive, byte ranges, regets, progress meters, throttling,
+ retries, access to authenticated http/ftp servers, and proxies. Additionally
+ it has the ability to treat a list of mirrors as a single source and to
+ automatically switch mirrors if there is a failure.
+ .
+ Homepage: http://linux.duke.edu/projects/urlgrabber/
Added: packages/urlgrabber/trunk/debian/copyright
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/copyright?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/copyright (added)
+++ packages/urlgrabber/trunk/debian/copyright Tue Apr 24 15:44:22 2007
@@ -1,0 +1,32 @@
+This package was debianized by Anand Kumria <wildfire at progsoc.org> on
+Sun, 9 Oct 2005 13:06:55 +1000.
+
+It was downloaded from <http://linux.duke.edu/projects/urlgrabber/download/>
+
+Upstream Authors: Michael D. Stenner <mstenner at linux.duke.edu>
+ Ryan Tomayko <rtomayko at naeblis.cx>
+
+Copyright: (c) 2002-2006 Michael D. Stenner
+ Ryan Tomayko
+
+License:
+
+ This package is free software; you can redistribute it and/or modify it
+ under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or (at your
+ option) any later version.
+
+ This package is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
+ License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with this package; if not, write to the Free Software Foundation,
+ Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+
+On Debian systems, the complete text of the GNU Lesser General Public License
+can be found in `/usr/share/common-licenses/LGPL'.
+
+The Debian packaging is (C) 2007, Kevin Coyner <kevin at rustybear.com> and is
+licensed under the GPL, see `/usr/share/common-licenses/GPL'.
Added: packages/urlgrabber/trunk/debian/dirs
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/dirs?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/dirs (added)
+++ packages/urlgrabber/trunk/debian/dirs Tue Apr 24 15:44:22 2007
@@ -1,0 +1,1 @@
+usr/bin
Added: packages/urlgrabber/trunk/debian/docs
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/docs?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/docs (added)
+++ packages/urlgrabber/trunk/debian/docs Tue Apr 24 15:44:22 2007
@@ -1,0 +1,1 @@
+TODO
Added: packages/urlgrabber/trunk/debian/postinst
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/postinst?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/postinst (added)
+++ packages/urlgrabber/trunk/debian/postinst Tue Apr 24 15:44:22 2007
@@ -1,0 +1,16 @@
+#!/bin/sh
+set -e
+
+if [ -d /usr/share/doc/urlgrabber-2.9.9 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.9
+fi
+
+if [ -d /usr/share/doc/urlgrabber-2.9.8 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.8
+fi
+
+if [ -d /usr/share/doc/urlgrabber-2.9.7 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.7
+fi
+
+#DEBHELPER#
Added: packages/urlgrabber/trunk/debian/rules
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/rules?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/rules (added)
+++ packages/urlgrabber/trunk/debian/rules Tue Apr 24 15:44:22 2007
@@ -1,0 +1,82 @@
+#!/usr/bin/make -f
+# -*- makefile -*-
+# Sample debian/rules that uses debhelper.
+# This file was originally written by Joey Hess and Craig Small.
+# As a special exception, when this file is copied by dh-make into a
+# dh-make output file, you may use that output file without restriction.
+# This special exception was added by Craig Small in version 0.37 of dh-make.
+
+# Uncomment this to turn on verbose mode.
+#export DH_VERBOSE=1
+
+PYVERS = $(shell pyversions -r)
+
+CFLAGS = -Wall -g
+
+ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
+ CFLAGS += -O0
+else
+ CFLAGS += -O2
+endif
+
+configure: configure-stamp
+configure-stamp:
+ dh_testdir
+ touch configure-stamp
+
+build: build-stamp
+
+build-stamp: configure-stamp
+ dh_testdir
+ for py in $(PYVERS); do \
+ $$py setup.py build; \
+ $$py test/runtests.py; \
+ done
+ touch build-stamp
+
+clean:
+ dh_testdir
+ dh_testroot
+ -rm -f build-stamp configure-stamp
+ for py in $(PYVERS); do \
+ $$py setup.py clean; \
+ done
+ python setup.py clean
+ find $(CURDIR) -name "*.pyc" -exec rm -f '{}' \;
+ dh_clean
+
+install: build
+ dh_testdir
+ dh_testroot
+ dh_clean -k
+ dh_installdirs
+ # Add here commands to install the package into debian/urlgrabber.
+ for py in $(PYVERS); do \
+ $$py setup.py install --root=$(CURDIR)/debian/python-urlgrabber; \
+ done
+ # remove unneeded documents installed by setup.py
+ -rm -rf $(CURDIR)/debian/python-urlgrabber/usr/share/doc/urlgrabber-3*
+
+# Build architecture-independent files here.
+binary-indep: build install
+ dh_testdir
+ dh_testroot
+ dh_installchangelogs ChangeLog
+ dh_installdocs
+ dh_installman debian/urlgrabber.1
+ dh_link
+ dh_strip
+ dh_compress
+ dh_fixperms
+ dh_pysupport
+ dh_installdeb
+ dh_shlibdeps
+ dh_gencontrol
+ dh_md5sums
+ dh_builddeb
+
+# Build architecture-dependent files here.
+binary-arch: build install
+
+binary: binary-indep binary-arch
+.PHONY: build clean binary-indep binary-arch binary install configure
Propchange: packages/urlgrabber/trunk/debian/rules
------------------------------------------------------------------------------
svn:executable = *
Added: packages/urlgrabber/trunk/debian/urlgrabber.1
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/urlgrabber.1?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/urlgrabber.1 (added)
+++ packages/urlgrabber/trunk/debian/urlgrabber.1 Tue Apr 24 15:44:22 2007
@@ -1,0 +1,133 @@
+.\" Title: urlgrabber
+.\" Author:
+.\" Generator: DocBook XSL Stylesheets v1.72.0 <http://docbook.sf.net/>
+.\" Date: 04/09/2007
+.\" Manual:
+.\" Source:
+.\"
+.TH "URLGRABBER" "1" "04/09/2007" "" ""
+.\" disable hyphenation
+.nh
+.\" disable justification (adjust text to left margin only)
+.ad l
+.SH "NAME"
+urlgrabber \- a high\-level cross\-protocol url\-grabber.
+.SH "SYNOPSIS"
+\fIurlgrabber\fR [OPTIONS] URL [FILE]
+.sp
+.SH "DESCRIPTION"
+urlgrabber is a binary program and python module for fetching files. It is designed to be used in programs that need common (but not necessarily simple) url\-fetching features.
+.sp
+.SH "OPTIONS"
+.PP
+\-\-help, \-h
+.RS 4
+help page specifying available options to the binary program.
+.RE
+.PP
+\-\-copy\-local
+.RS 4
+ignored except for file:// urls, in which case it specifies whether urlgrab should still make a copy of the file, or simply point to the existing copy.
+.RE
+.PP
+\-\-throttle=NUMBER
+.RS 4
+if it's an int, it's the bytes/second throttle limit. If it's a float, it is first multiplied by bandwidth. If throttle == 0, throttling is disabled. If None, the module\-level default (which can be set with set_throttle) is used.
+.RE
+.PP
+\-\-bandwidth=NUMBER
+.RS 4
+the nominal max bandwidth in bytes/second. If throttle is a float and bandwidth == 0, throttling is disabled. If None, the module\-level default (which can be set with set_bandwidth) is used.
+.RE
+.PP
+\-\-range=RANGE
+.RS 4
+a tuple of the form first_byte,last_byte describing a byte range to retrieve. Either or both of the values may be specified. If first_byte is None, byte offset 0 is assumed. If last_byte is None, the last byte available is assumed. Note that both first and last_byte values are inclusive so a range of (10,11) would return the 10th and 11th bytes of the resource.
+.RE
+.PP
+\-\-user\-agent=STR
+.RS 4
+the user\-agent string provide if the url is HTTP.
+.RE
+.PP
+\-\-retry=NUMBER
+.RS 4
+the number of times to retry the grab before bailing. If this is zero, it will retry forever. This was intentional\&... really, it was :). If this value is not supplied or is supplied but is None retrying does not occur.
+.RE
+.PP
+\-\-retrycodes
+.RS 4
+a sequence of errorcodes (values of e.errno) for which it should retry. See the doc on URLGrabError for more details on this. retrycodes defaults to \-1,2,4,5,6,7 if not specified explicitly.
+.RE
+.SH "MODULE USE EXAMPLES"
+In its simplest form, urlgrabber can be a replacement for urllib2's open, or even python's file if you're just reading:
+.sp
+.RS 4
+.nf
+ from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()
+.fi
+.sp
+.RE
+Here, the url can be http, https, ftp, or file. It's also pretty smart so if you just give it something like /tmp/foo, it will figure it out. For even more fun, you can also do:
+.sp
+.RS 4
+.nf
+ from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string
+.fi
+.sp
+.RE
+Now, like urllib2, what's really happening here is that you're using a module\-level object (called a grabber) that kind of serves as a default. That's just fine, but you might want to get your own private version for a couple of reasons:
+.sp
+.RS 4
+.nf
+* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior
+.fi
+.sp
+.RE
+Therefore, you're probably better off making your own. This also gives you lots of flexibility for later, as you'll see:
+.sp
+.RS 4
+.nf
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)
+.fi
+.sp
+.RE
+This is nice because you can specify options when you create the grabber. For example, let's turn on simple reget mode so that if we have part of a file, we only need to fetch the rest:
+.sp
+.RS 4
+.nf
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)
+.fi
+.sp
+.RE
+The available options are listed in the module documentation, and can usually be specified as a default at the grabber\-level or as options to the method:
+.sp
+.RS 4
+.nf
+from urlgrabber.grabber import URLGrabber
+g = URLGrabber(reget='simple')
+local_filename = g.urlgrab(url, filename=None, reget=None)
+.fi
+.sp
+.RE
+.SH "AUTHORS"
+Written by: Michael D. Stenner <mstenner at linux.duke.edu> Ryan Tomayko <rtomayko at naeblis.cx>
+.sp
+This manual page was written by Kevin Coyner <kevin at rustybear.com> for the Debian system (but may be used by others). It borrows heavily on the documentation included in the urlgrabber module. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version published by the Free Software Foundation.
+.sp
+.SH "RESOURCES"
+Main web site: \fIhttp://linux.duke.edu/projects/urlgrabber/\fR
+.sp
Added: packages/urlgrabber/trunk/debian/urlgrabber.txt
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/urlgrabber.txt?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/urlgrabber.txt (added)
+++ packages/urlgrabber/trunk/debian/urlgrabber.txt Tue Apr 24 15:44:22 2007
@@ -1,0 +1,150 @@
+URLGRABBER(1)
+=============
+
+NAME
+----
+urlgrabber - a high-level cross-protocol url-grabber.
+
+SYNOPSIS
+--------
+'urlgrabber' [OPTIONS] URL [FILE]
+
+DESCRIPTION
+-----------
+urlgrabber is a binary program and python module for fetching files. It is
+designed to be used in programs that need common (but not necessarily simple)
+url-fetching features.
+
+OPTIONS
+-------
+--help, -h::
+ help page specifying available options to the binary program.
+
+--copy-local::
+ ignored except for file:// urls, in which case
+ it specifies whether urlgrab should still make
+ a copy of the file, or simply point to the
+ existing copy.
+
+--throttle=NUMBER::
+ if it's an int, it's the bytes/second throttle
+ limit. If it's a float, it is first multiplied
+ by bandwidth. If throttle == 0, throttling is
+ disabled. If None, the module-level default
+ (which can be set with set_throttle) is used.
+
+--bandwidth=NUMBER::
+ the nominal max bandwidth in bytes/second. If
+ throttle is a float and bandwidth == 0,
+ throttling is disabled. If None, the
+ module-level default (which can be set with
+ set_bandwidth) is used.
+
+--range=RANGE::
+ a tuple of the form first_byte,last_byte
+ describing a byte range to retrieve. Either or
+ both of the values may be specified. If
+ first_byte is None, byte offset 0 is assumed.
+ If last_byte is None, the last byte available
+ is assumed. Note that both first and last_byte
+ values are inclusive so a range of (10,11)
+ would return the 10th and 11th bytes of the
+ resource.
+
+--user-agent=STR::
+ the user-agent string provide if the url is HTTP.
+
+--retry=NUMBER::
+ the number of times to retry the grab before
+ bailing. If this is zero, it will retry
+ forever. This was intentional... really, it was
+ :). If this value is not supplied or is supplied
+ but is None retrying does not occur.
+
+--retrycodes::
+ a sequence of errorcodes (values of e.errno) for
+ which it should retry. See the doc on
+ URLGrabError for more details on this. retrycodes
+ defaults to -1,2,4,5,6,7 if not specified
+ explicitly.
+
+
+MODULE USE EXAMPLES
+-------------------
+In its simplest form, urlgrabber can be a replacement for urllib2's
+open, or even python's file if you're just reading:
+..................................
+ from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()
+..................................
+
+Here, the url can be http, https, ftp, or file. It's also pretty smart
+so if you just give it something like /tmp/foo, it will
+figure it out. For even more fun, you can also do:
+
+..................................
+ from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string
+..................................
+
+Now, like urllib2, what's really happening here is that you're using a
+module-level object (called a grabber) that kind of serves as a
+default. That's just fine, but you might want to get your own private
+version for a couple of reasons:
+
+..................................
+* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior
+..................................
+
+Therefore, you're probably better off making your own. This also gives
+you lots of flexibility for later, as you'll see:
+
+..................................
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)
+..................................
+
+This is nice because you can specify options when you create the
+grabber. For example, let's turn on simple reget mode so that if we
+have part of a file, we only need to fetch the rest:
+
+..................................
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)
+..................................
+
+The available options are listed in the module documentation, and can
+usually be specified as a default at the grabber-level or as options
+to the method:
+
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url, filename=None, reget=None)
+
+
+AUTHORS
+-------
+Written by:
+Michael D. Stenner <mstenner at linux.duke.edu>
+Ryan Tomayko <rtomayko at naeblis.cx>
+
+This manual page was written by Kevin Coyner <kevin at rustybear.com> for
+the Debian system (but may be used by others). It borrows heavily on
+the documentation included in the urlgrabber module. Permission is granted
+to copy, distribute and/or modify this document under the terms of
+the GNU General Public License, Version 2 any later version published
+by the Free Software Foundation.
+
+
+RESOURCES
+---------
+Main web site: http://linux.duke.edu/projects/urlgrabber/[]
Added: packages/urlgrabber/trunk/debian/urlgrabber.xml
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/urlgrabber.xml?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/urlgrabber.xml (added)
+++ packages/urlgrabber/trunk/debian/urlgrabber.xml Tue Apr 24 15:44:22 2007
@@ -1,0 +1,201 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+<refentry>
+<refmeta>
+<refentrytitle>urlgrabber</refentrytitle>
+<manvolnum>1</manvolnum>
+</refmeta>
+<refnamediv>
+ <refname>urlgrabber</refname>
+ <refpurpose>a high-level cross-protocol url-grabber.</refpurpose>
+</refnamediv>
+<refsynopsisdiv>
+<simpara><emphasis>urlgrabber</emphasis> [OPTIONS] URL [FILE]</simpara>
+
+</refsynopsisdiv>
+<refsect1>
+<title>DESCRIPTION</title>
+<simpara>urlgrabber is a binary program and python module for fetching files. It is
+designed to be used in programs that need common (but not necessarily simple)
+url-fetching features.</simpara>
+
+</refsect1>
+<refsect1>
+<title>OPTIONS</title>
+<variablelist>
+<varlistentry>
+<term>
+--help, -h
+</term>
+<listitem>
+<simpara>
+ help page specifying available options to the binary program.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--copy-local
+</term>
+<listitem>
+<simpara>
+ ignored except for file:// urls, in which case
+ it specifies whether urlgrab should still make
+ a copy of the file, or simply point to the
+ existing copy.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--throttle=NUMBER
+</term>
+<listitem>
+<simpara>
+ if it's an int, it's the bytes/second throttle
+ limit. If it's a float, it is first multiplied
+ by bandwidth. If throttle == 0, throttling is
+ disabled. If None, the module-level default
+ (which can be set with set_throttle) is used.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--bandwidth=NUMBER
+</term>
+<listitem>
+<simpara>
+ the nominal max bandwidth in bytes/second. If
+ throttle is a float and bandwidth == 0,
+ throttling is disabled. If None, the
+ module-level default (which can be set with
+ set_bandwidth) is used.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--range=RANGE
+</term>
+<listitem>
+<simpara>
+ a tuple of the form first_byte,last_byte
+ describing a byte range to retrieve. Either or
+ both of the values may be specified. If
+ first_byte is None, byte offset 0 is assumed.
+ If last_byte is None, the last byte available
+ is assumed. Note that both first and last_byte
+ values are inclusive so a range of (10,11)
+ would return the 10th and 11th bytes of the
+ resource.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--user-agent=STR
+</term>
+<listitem>
+<simpara>
+ the user-agent string provide if the url is HTTP.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--retry=NUMBER
+</term>
+<listitem>
+<simpara>
+ the number of times to retry the grab before
+ bailing. If this is zero, it will retry
+ forever. This was intentional… really, it was
+ :). If this value is not supplied or is supplied
+ but is None retrying does not occur.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--retrycodes
+</term>
+<listitem>
+<simpara>
+ a sequence of errorcodes (values of e.errno) for
+ which it should retry. See the doc on
+ URLGrabError for more details on this. retrycodes
+ defaults to -1,2,4,5,6,7 if not specified
+ explicitly.
+</simpara>
+</listitem>
+</varlistentry>
+</variablelist>
+</refsect1>
+<refsect1>
+<title>MODULE USE EXAMPLES</title>
+<simpara>In its simplest form, urlgrabber can be a replacement for urllib2's
+open, or even python's file if you're just reading:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()</literallayout>
+<simpara>Here, the url can be http, https, ftp, or file. It's also pretty smart
+so if you just give it something like /tmp/foo, it will
+figure it out. For even more fun, you can also do:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string</literallayout>
+<simpara>Now, like urllib2, what's really happening here is that you're using a
+module-level object (called a grabber) that kind of serves as a
+default. That's just fine, but you might want to get your own private
+version for a couple of reasons:</simpara>
+
+<literallayout class="monospaced">* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior</literallayout>
+<simpara>Therefore, you're probably better off making your own. This also gives
+you lots of flexibility for later, as you'll see:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)</literallayout>
+<simpara>This is nice because you can specify options when you create the
+grabber. For example, let's turn on simple reget mode so that if we
+have part of a file, we only need to fetch the rest:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)</literallayout>
+<simpara>The available options are listed in the module documentation, and can
+usually be specified as a default at the grabber-level or as options
+to the method:</simpara>
+
+<literallayout class="monospaced">from urlgrabber.grabber import URLGrabber
+g = URLGrabber(reget='simple')
+local_filename = g.urlgrab(url, filename=None, reget=None)</literallayout>
+</refsect1>
+<refsect1>
+<title>AUTHORS</title>
+<simpara>Written by:
+Michael D. Stenner <mstenner at linux.duke.edu>
+Ryan Tomayko <rtomayko at naeblis.cx></simpara>
+
+<simpara>This manual page was written by Kevin Coyner <kevin at rustybear.com> for
+the Debian system (but may be used by others). It borrows heavily on
+the documentation included in the urlgrabber module. Permission is granted
+to copy, distribute and/or modify this document under the terms of
+the GNU General Public License, Version 2 any later version published
+by the Free Software Foundation.</simpara>
+
+</refsect1>
+<refsect1>
+<title>RESOURCES</title>
+<simpara>Main web site: <ulink url="http://linux.duke.edu/projects/urlgrabber/">http://linux.duke.edu/projects/urlgrabber/</ulink></simpara>
+
+</refsect1>
+</refentry>
Added: packages/urlgrabber/trunk/debian/watch
URL: http://svn.debian.org/wsvn/python-modules/packages/urlgrabber/trunk/debian/watch?rev=2168&op=file
==============================================================================
--- packages/urlgrabber/trunk/debian/watch (added)
+++ packages/urlgrabber/trunk/debian/watch Tue Apr 24 15:44:22 2007
@@ -1,0 +1,2 @@
+version=3
+http://linux.duke.edu/projects/urlgrabber/download/urlgrabber-(.*)\.tar\.gz
More information about the Python-modules-commits
mailing list