[Python-modules-commits] [urlgrabber] 02/06: Import urlgrabber_3.10.1-2.debian.tar.xz
Michal Cihar
nijel at moszumanska.debian.org
Wed Nov 8 12:19:11 UTC 2017
This is an automated email from the git hooks/post-receive script.
nijel pushed a commit to annotated tag archive/debian/3.10.2-1
in repository urlgrabber.
commit 82b435d7bf79ef69cd4d19d409f46ba2cebc8228
Author: Michal Čihař <nijel at debian.org>
Date: Mon Oct 31 09:09:20 2016 +0100
Import urlgrabber_3.10.1-2.debian.tar.xz
[dgit import tarball urlgrabber 3.10.1-2 urlgrabber_3.10.1-2.debian.tar.xz]
---
.git-dpm | 11 +++
changelog | 192 +++++++++++++++++++++++++++++++++++++++++++
compat | 1 +
control | 27 ++++++
copyright | 37 +++++++++
docs | 1 +
gbp.conf | 5 ++
patches/libexec.diff | 37 +++++++++
patches/series | 1 +
postinst | 16 ++++
python-urlgrabber.manpages | 1 +
rules | 12 +++
source/format | 1 +
urlgrabber.1 | 133 ++++++++++++++++++++++++++++++
urlgrabber.txt | 150 +++++++++++++++++++++++++++++++++
urlgrabber.xml | 201 +++++++++++++++++++++++++++++++++++++++++++++
watch | 2 +
17 files changed, 828 insertions(+)
diff --git a/.git-dpm b/.git-dpm
new file mode 100644
index 0000000..3e7bf1e
--- /dev/null
+++ b/.git-dpm
@@ -0,0 +1,11 @@
+# see git-dpm(1) from git-dpm package
+9f0a228533a0d392f115c7b5547c7f7fbc92a555
+9f0a228533a0d392f115c7b5547c7f7fbc92a555
+b287268cb34fa36ab3acd9b19f41953bbd418498
+b287268cb34fa36ab3acd9b19f41953bbd418498
+urlgrabber_3.10.1.orig.tar.gz
+75206abe4c2498d4ff01498e4a35192a65c92f3e
+83735
+debianTag="debian/%e%v"
+patchedTag="patched/%e%v"
+upstreamTag="upstream/%e%u"
diff --git a/changelog b/changelog
new file mode 100644
index 0000000..3b00c65
--- /dev/null
+++ b/changelog
@@ -0,0 +1,192 @@
+urlgrabber (3.10.1-2) unstable; urgency=medium
+
+ * Team upload.
+ * Switch to debhelper 9.
+ - fixes lintian warning about deprecated level
+ - avoids running testsuite through dh_auto_test (Closes: #842248)
+
+ -- Michal Čihař <nijel at debian.org> Mon, 31 Oct 2016 09:09:20 +0100
+
+urlgrabber (3.10.1-1) unstable; urgency=low
+
+ * Team upload.
+
+ [ Jakub Wilk ]
+ * Use canonical URIs for Vcs-* fields.
+
+ [ Tristan Hill ]
+ * New upstream release (Closes: #747202).
+ * Drop all patches now applied upstream
+ * Add libexec patch that moves urlgrabber-ext-down script to /usr/share
+ * debian/control
+ + Switch to dh_python2
+ + Bumped standards version to 3.9.5. No changes needed.
+ + Update package description per lintian warning
+
+ [ Ondřej Nový ]
+ * Fixed VCS URL (https)
+
+ [ Michal Čihař ]
+ * Bumped standards version to 3.9.8. No changes needed.
+ * Do not run testsuite which relies on internet access
+ (Closes: #830113, #586684).
+ * Remove outdated README.Debian (Closes: #592619).
+
+ -- Michal Čihař <nijel at debian.org> Sun, 24 Jul 2016 10:19:54 +0200
+
+urlgrabber (3.9.1-4.2) unstable; urgency=medium
+
+ * Non-maintainer upload.
+ * Build using dh-python instead of python-support. Closes: #637262
+ * Update Vcs-* fields after DPMT migration to git.
+
+ -- Mattia Rizzolo <mattia at debian.org> Tue, 15 Dec 2015 02:14:39 +0000
+
+urlgrabber (3.9.1-4.1) unstable; urgency=medium
+
+ * Non-maintainer upload.
+ * Add debian/patches/https-verifyhost-fix.diff taken from upstream
+ to fix https connection issues (Closes: #715416, #722191).
+
+ -- John Paul Adrian Glaubitz <glaubitz at physik.fu-berlin.de> Tue, 30 Dec 2014 13:13:08 +0100
+
+urlgrabber (3.9.1-4) unstable; urgency=low
+
+ * Add two patches created from upstream development version. Closes: #587575.
+ * Changed to 3.0 quilt format:
+ + Add quilt to build-depends.
+ + Add quilt command to debian/rules.
+
+ -- Kevin Coyner <kcoyner at debian.org> Thu, 08 Jul 2010 17:40:08 +0000
+
+urlgrabber (3.9.1-3) unstable; urgency=low
+
+ * Add Depends on python-pycurl. Closes: #587000.
+
+ -- Kevin Coyner <kcoyner at debian.org> Fri, 25 Jun 2010 02:04:19 +0000
+
+urlgrabber (3.9.1-2) unstable; urgency=low
+
+ * Install with only the default python version to avoid unnecessary depends
+ on python2.5. Changes made to debian/rules. Closes: #587006. Thanks to
+ Stefano Rivera.
+ * Debian files uploaded to svn repository on alioth for python modules.
+ Closes: #587004.
+
+ -- Kevin Coyner <kcoyner at debian.org> Fri, 25 Jun 2010 01:25:25 +0000
+
+urlgrabber (3.9.1-1) unstable; urgency=low
+
+ * New upstream release. The main backend was changed from from urllib2 to
+ pycurl. The API is identical. Callers do not need to change anything.
+ Closes: #518436, #517993, #493251, #586400, #529752.
+ * debian/control:
+ + Bumped standards version to 3.8.4. No changes needed.
+ + Bumped debhelper version to 7.4~.
+ + Removed build dependency on dpatch.
+ + Added build dependency on python-pycurl.
+ + Updated homepage.
+ + Set XS-Python-Version: >= 2.5
+ * Removed keepalive.py patch.
+ * Updated debian/watch for new homepage.
+ * Update homepage reference in debian/copyright. Closes: #586399. Updated
+ copyright information for additional new author Seth Vidal.
+ * Add debian/source/format file set to 3.0 (quilt).
+ * debian/rules:
+ + Run setup.py with current python version only.
+ + Respect nocheck in DEB_BULID_OPTIONS (although failed tests don't abort
+ + Changed dh_clean -k to dh_prep to conform with debhelper version 7.
+ build, as several tests fail)
+ * Bumped debian/compat from 5 to 7.
+
+ -- Kevin Coyner <kcoyner at debian.org> Mon, 21 Jun 2010 20:36:19 +0000
+
+urlgrabber (3.1.0-5) unstable; urgency=low
+
+ [ Piotr Ożarowski ]
+ * Homepage field added
+ * Rename XS-Vcs-* fields to Vcs-* (dpkg supports them now)
+
+ [ Sandro Tosi ]
+ * debian/control
+ - switch Vcs-Browser field to viewsvn
+
+ [ Jakub Wilk ]
+ * Build-depend on python-all rather than python-all-dev.
+ * Remove superfluous references to CFLAGS from debian/rules.
+ * Prepare for Python 2.6 transition (closes: #556161).
+ * Typographical fixes in debian/copyright.
+ * Bump standards version to 3.8.3 (no additional changes needed).
+ * Point to the versioned LGPL-2.1 in debian/copyright.
+ * Add README.source.
+
+ -- Debian Python Modules Team <python-modules-team at lists.alioth.debian.org> Sat, 14 Nov 2009 09:37:03 +0100
+
+urlgrabber (3.1.0-4) unstable; urgency=low
+
+ * Patch to have urlgrabber.keepalive.HTTPHandler use Request.get_method() to
+ determine the appropriate HTTP method. Thanks to Jakub Wilk.
+ Closes: #433724
+ * Changed maintainer e-mail to reflect new Debian account.
+ * Added dpatch as Build-Depends to debian/control.
+
+ -- Kevin Coyner <kcoyner at debian.org> Sat, 04 Aug 2007 21:52:14 -0400
+
+urlgrabber (3.1.0-3) unstable; urgency=low
+
+ * debian/control: Added python modules packaging team to uploaders and added
+ VCS fields.
+
+ -- Kevin Coyner <kevin at rustybear.com> Mon, 09 Apr 2007 19:27:36 -0600
+
+urlgrabber (3.1.0-2) unstable; urgency=low
+
+ * debian/control: Changed "Architecture: any" to all.
+
+ -- Kevin Coyner <kevin at rustybear.com> Mon, 09 Apr 2007 15:20:02 -0600
+
+urlgrabber (3.1.0-1) unstable; urgency=low
+
+ * New upstream release.
+ * New maintainer. (Closes: #418095)
+ * Added man page.
+ * Cleaned up cruft in debian/rules.
+ * Rewrote debian/copyright.
+ * Cleaned up debian/control and added homepage.
+ * Added debian/README.Debian.
+ * Added debian/postinst to clean up unneeded docs that were inappropriately
+ added in previous versions.
+ * Removed unneeded debian/pycompat file.
+
+ -- Kevin Coyner <kevin at rustybear.com> Fri, 06 Apr 2007 22:27:03 -0400
+
+urlgrabber (2.9.9-1) unstable; urgency=low
+
+ * New upstream release
+ * Apply Ana Beatriz Guerrero Lopez's patch to
+ * Update to new Python policy (Closes: #373402)
+ * Switch to standards version 3.7.2
+ * Update to debhelper compat level 5
+ * Thanks Ana!
+
+ -- Anand Kumria <wildfire at progsoc.org> Thu, 6 Jul 2006 09:16:37 +1000
+
+urlgrabber (2.9.7-2) unstable; urgency=low
+
+ * When I imported urlgrabber into bzr, I somehow lost a Build-Dep: on
+ python. Re-adding it so I can (Closes: #335340)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sat, 31 Dec 2005 15:34:22 +1100
+
+urlgrabber (2.9.7-1) unstable; urgency=low
+
+ * New upstream release (Closes: #344934)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sat, 31 Dec 2005 15:34:22 +1100
+
+urlgrabber (2.9.6-1) unstable; urgency=low
+
+ * Initial release (Closes: #312698)
+
+ -- Anand Kumria <wildfire at progsoc.org> Sun, 9 Oct 2005 13:06:55 +1000
+
diff --git a/compat b/compat
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/compat
@@ -0,0 +1 @@
+9
diff --git a/control b/control
new file mode 100644
index 0000000..56c834e
--- /dev/null
+++ b/control
@@ -0,0 +1,27 @@
+Source: urlgrabber
+Section: python
+Priority: optional
+Maintainer: Kevin Coyner <kcoyner at debian.org>
+Uploaders: Debian Python Modules Team <python-modules-team at lists.alioth.debian.org>
+Build-Depends: debhelper (>= 9), dh-python, python-all (>= 2.6.6-3~), python-pycurl
+Standards-Version: 3.9.8
+Homepage: http://urlgrabber.baseurl.org/
+Vcs-Git: https://anonscm.debian.org/git/python-modules/packages/urlgrabber.git
+Vcs-Browser: https://anonscm.debian.org/cgit/python-modules/packages/urlgrabber.git
+XS-Python-Version: >= 2.5
+
+Package: python-urlgrabber
+Architecture: all
+Depends: ${misc:Depends}, ${python:Depends}, python-pycurl
+Provides: ${python:Provides}
+Description: high-level URL transfer library
+ urlgrabber dramatically simplifies the fetching of files. It is designed to
+ be used in programs that need common (but not necessarily simple)
+ url-fetching features. This package provides both a binary and a module, both
+ of the name urlgrabber.
+ .
+ It supports identical behavior for http://, ftp:// and file:/// URIs. It
+ provides HTTP keepalive, byte ranges, regets, progress meters, throttling,
+ retries, access to authenticated http/ftp servers, and proxies. Additionally
+ it has the ability to treat a list of mirrors as a single source and to
+ automatically switch mirrors if there is a failure.
diff --git a/copyright b/copyright
new file mode 100644
index 0000000..527eb6a
--- /dev/null
+++ b/copyright
@@ -0,0 +1,37 @@
+This package was debianized by Anand Kumria <wildfire at progsoc.org> on
+Sun, 9 Oct 2005 13:06:55 +1000.
+
+It was originally downloaded from <http://linux.duke.edu/projects/urlgrabber/download/>
+
+It can now be downloaded from <http://urlgrabber.baseurl.org/>
+
+Upstream Authors: Michael D. Stenner <mstenner at linux.duke.edu>
+ Ryan Tomayko <rtomayko at naeblis.cx>
+ Seth Vidal <skvidal at fedoraproject.org>
+
+Copyright: © 2002-2006 Michael D. Stenner
+ Ryan Tomayko
+
+Copyright: © 2009 Red Hat Inc, pycurl code written by Seth Vidal
+
+License:
+
+ This package is free software; you can redistribute it and/or modify it
+ under the terms of the GNU Lesser General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or (at your
+ option) any later version.
+
+ This package is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
+ License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with this package; if not, write to the Free Software Foundation,
+ Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+
+On Debian systems, the complete text of the GNU Lesser General Public License
+can be found in `/usr/share/common-licenses/LGPL-2.1'.
+
+The Debian packaging is © 2007, Kevin Coyner <kcoyner at debian.org> and is
+licensed under the GPL, see `/usr/share/common-licenses/GPL'.
diff --git a/docs b/docs
new file mode 100644
index 0000000..1333ed7
--- /dev/null
+++ b/docs
@@ -0,0 +1 @@
+TODO
diff --git a/gbp.conf b/gbp.conf
new file mode 100644
index 0000000..41f4f8c
--- /dev/null
+++ b/gbp.conf
@@ -0,0 +1,5 @@
+# Configuration file for git-buildpackage and friends
+
+[DEFAULT]
+sign-tags = True
+pristine-tar = True
diff --git a/patches/libexec.diff b/patches/libexec.diff
new file mode 100644
index 0000000..97f9774
--- /dev/null
+++ b/patches/libexec.diff
@@ -0,0 +1,37 @@
+From 9f0a228533a0d392f115c7b5547c7f7fbc92a555 Mon Sep 17 00:00:00 2001
+From: Stefano Rivera <stefanor at debian.org>
+Date: Tue, 13 Oct 2015 00:10:42 +0200
+Subject: libexec.diff
+
+Patch-Name: libexec.diff
+---
+ setup.py | 2 +-
+ urlgrabber/grabber.py | 2 +-
+ 2 files changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/setup.py b/setup.py
+index bfa4a18..ec96986 100644
+--- a/setup.py
++++ b/setup.py
+@@ -17,7 +17,7 @@ package_dir = {'urlgrabber':'urlgrabber'}
+ scripts = ['scripts/urlgrabber']
+ data_files = [
+ ('share/doc/' + name + '-' + version, ['README','LICENSE', 'TODO', 'ChangeLog']),
+- ('libexec', ['scripts/urlgrabber-ext-down']),
++ ('share/python-urlgrabber', ['scripts/urlgrabber-ext-down']),
+ ]
+ options = { 'clean' : { 'all' : 1 } }
+ classifiers = [
+diff --git a/urlgrabber/grabber.py b/urlgrabber/grabber.py
+index ef18d6a..d256ced 100644
+--- a/urlgrabber/grabber.py
++++ b/urlgrabber/grabber.py
+@@ -2042,7 +2042,7 @@ import subprocess
+ class _ExternalDownloader:
+ def __init__(self):
+ self.popen = subprocess.Popen(
+- '/usr/libexec/urlgrabber-ext-down',
++ '/usr/share/python-urlgrabber/urlgrabber-ext-down',
+ stdin = subprocess.PIPE,
+ stdout = subprocess.PIPE,
+ )
diff --git a/patches/series b/patches/series
new file mode 100644
index 0000000..1db2d70
--- /dev/null
+++ b/patches/series
@@ -0,0 +1 @@
+libexec.diff
diff --git a/postinst b/postinst
new file mode 100644
index 0000000..8a33e2a
--- /dev/null
+++ b/postinst
@@ -0,0 +1,16 @@
+#!/bin/sh
+set -e
+
+if [ -d /usr/share/doc/urlgrabber-2.9.9 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.9
+fi
+
+if [ -d /usr/share/doc/urlgrabber-2.9.8 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.8
+fi
+
+if [ -d /usr/share/doc/urlgrabber-2.9.7 ]; then
+ rm -rf /usr/share/doc/urlgrabber-2.9.7
+fi
+
+#DEBHELPER#
diff --git a/python-urlgrabber.manpages b/python-urlgrabber.manpages
new file mode 100644
index 0000000..085b824
--- /dev/null
+++ b/python-urlgrabber.manpages
@@ -0,0 +1 @@
+debian/urlgrabber.1
diff --git a/rules b/rules
new file mode 100755
index 0000000..7590fbf
--- /dev/null
+++ b/rules
@@ -0,0 +1,12 @@
+#!/usr/bin/make -f
+%:
+ dh $@ --with python2 --buildsystem=pybuild
+
+override_dh_auto_install:
+ dh_auto_install
+ # remove unneeded documents installed by setup.py
+ -rm -rf $(CURDIR)/debian/python-urlgrabber/usr/share/doc/urlgrabber-3*
+
+override_dh_installchangelogs:
+ dh_installchangelogs ChangeLog
+
diff --git a/source/format b/source/format
new file mode 100644
index 0000000..163aaf8
--- /dev/null
+++ b/source/format
@@ -0,0 +1 @@
+3.0 (quilt)
diff --git a/urlgrabber.1 b/urlgrabber.1
new file mode 100644
index 0000000..267e94c
--- /dev/null
+++ b/urlgrabber.1
@@ -0,0 +1,133 @@
+.\" Title: urlgrabber
+.\" Author:
+.\" Generator: DocBook XSL Stylesheets v1.72.0 <http://docbook.sf.net/>
+.\" Date: 04/09/2007
+.\" Manual:
+.\" Source:
+.\"
+.TH "URLGRABBER" "1" "04/09/2007" "" ""
+.\" disable hyphenation
+.nh
+.\" disable justification (adjust text to left margin only)
+.ad l
+.SH "NAME"
+urlgrabber \- a high\-level cross\-protocol url\-grabber.
+.SH "SYNOPSIS"
+\fIurlgrabber\fR [OPTIONS] URL [FILE]
+.sp
+.SH "DESCRIPTION"
+urlgrabber is a binary program and python module for fetching files. It is designed to be used in programs that need common (but not necessarily simple) url\-fetching features.
+.sp
+.SH "OPTIONS"
+.PP
+\-\-help, \-h
+.RS 4
+help page specifying available options to the binary program.
+.RE
+.PP
+\-\-copy\-local
+.RS 4
+ignored except for file:// urls, in which case it specifies whether urlgrab should still make a copy of the file, or simply point to the existing copy.
+.RE
+.PP
+\-\-throttle=NUMBER
+.RS 4
+if it's an int, it's the bytes/second throttle limit. If it's a float, it is first multiplied by bandwidth. If throttle == 0, throttling is disabled. If None, the module\-level default (which can be set with set_throttle) is used.
+.RE
+.PP
+\-\-bandwidth=NUMBER
+.RS 4
+the nominal max bandwidth in bytes/second. If throttle is a float and bandwidth == 0, throttling is disabled. If None, the module\-level default (which can be set with set_bandwidth) is used.
+.RE
+.PP
+\-\-range=RANGE
+.RS 4
+a tuple of the form first_byte,last_byte describing a byte range to retrieve. Either or both of the values may be specified. If first_byte is None, byte offset 0 is assumed. If last_byte is None, the last byte available is assumed. Note that both first and last_byte values are inclusive so a range of (10,11) would return the 10th and 11th bytes of the resource.
+.RE
+.PP
+\-\-user\-agent=STR
+.RS 4
+the user\-agent string provide if the url is HTTP.
+.RE
+.PP
+\-\-retry=NUMBER
+.RS 4
+the number of times to retry the grab before bailing. If this is zero, it will retry forever. This was intentional\&... really, it was :). If this value is not supplied or is supplied but is None retrying does not occur.
+.RE
+.PP
+\-\-retrycodes
+.RS 4
+a sequence of errorcodes (values of e.errno) for which it should retry. See the doc on URLGrabError for more details on this. retrycodes defaults to \-1,2,4,5,6,7 if not specified explicitly.
+.RE
+.SH "MODULE USE EXAMPLES"
+In its simplest form, urlgrabber can be a replacement for urllib2's open, or even python's file if you're just reading:
+.sp
+.RS 4
+.nf
+ from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()
+.fi
+.sp
+.RE
+Here, the url can be http, https, ftp, or file. It's also pretty smart so if you just give it something like /tmp/foo, it will figure it out. For even more fun, you can also do:
+.sp
+.RS 4
+.nf
+ from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string
+.fi
+.sp
+.RE
+Now, like urllib2, what's really happening here is that you're using a module\-level object (called a grabber) that kind of serves as a default. That's just fine, but you might want to get your own private version for a couple of reasons:
+.sp
+.RS 4
+.nf
+* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior
+.fi
+.sp
+.RE
+Therefore, you're probably better off making your own. This also gives you lots of flexibility for later, as you'll see:
+.sp
+.RS 4
+.nf
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)
+.fi
+.sp
+.RE
+This is nice because you can specify options when you create the grabber. For example, let's turn on simple reget mode so that if we have part of a file, we only need to fetch the rest:
+.sp
+.RS 4
+.nf
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)
+.fi
+.sp
+.RE
+The available options are listed in the module documentation, and can usually be specified as a default at the grabber\-level or as options to the method:
+.sp
+.RS 4
+.nf
+from urlgrabber.grabber import URLGrabber
+g = URLGrabber(reget='simple')
+local_filename = g.urlgrab(url, filename=None, reget=None)
+.fi
+.sp
+.RE
+.SH "AUTHORS"
+Written by: Michael D. Stenner <mstenner at linux.duke.edu> Ryan Tomayko <rtomayko at naeblis.cx>
+.sp
+This manual page was written by Kevin Coyner <kevin at rustybear.com> for the Debian system (but may be used by others). It borrows heavily on the documentation included in the urlgrabber module. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version published by the Free Software Foundation.
+.sp
+.SH "RESOURCES"
+Main web site: \fIhttp://linux.duke.edu/projects/urlgrabber/\fR
+.sp
diff --git a/urlgrabber.txt b/urlgrabber.txt
new file mode 100644
index 0000000..ac7eac7
--- /dev/null
+++ b/urlgrabber.txt
@@ -0,0 +1,150 @@
+URLGRABBER(1)
+=============
+
+NAME
+----
+urlgrabber - a high-level cross-protocol url-grabber.
+
+SYNOPSIS
+--------
+'urlgrabber' [OPTIONS] URL [FILE]
+
+DESCRIPTION
+-----------
+urlgrabber is a binary program and python module for fetching files. It is
+designed to be used in programs that need common (but not necessarily simple)
+url-fetching features.
+
+OPTIONS
+-------
+--help, -h::
+ help page specifying available options to the binary program.
+
+--copy-local::
+ ignored except for file:// urls, in which case
+ it specifies whether urlgrab should still make
+ a copy of the file, or simply point to the
+ existing copy.
+
+--throttle=NUMBER::
+ if it's an int, it's the bytes/second throttle
+ limit. If it's a float, it is first multiplied
+ by bandwidth. If throttle == 0, throttling is
+ disabled. If None, the module-level default
+ (which can be set with set_throttle) is used.
+
+--bandwidth=NUMBER::
+ the nominal max bandwidth in bytes/second. If
+ throttle is a float and bandwidth == 0,
+ throttling is disabled. If None, the
+ module-level default (which can be set with
+ set_bandwidth) is used.
+
+--range=RANGE::
+ a tuple of the form first_byte,last_byte
+ describing a byte range to retrieve. Either or
+ both of the values may be specified. If
+ first_byte is None, byte offset 0 is assumed.
+ If last_byte is None, the last byte available
+ is assumed. Note that both first and last_byte
+ values are inclusive so a range of (10,11)
+ would return the 10th and 11th bytes of the
+ resource.
+
+--user-agent=STR::
+ the user-agent string provide if the url is HTTP.
+
+--retry=NUMBER::
+ the number of times to retry the grab before
+ bailing. If this is zero, it will retry
+ forever. This was intentional... really, it was
+ :). If this value is not supplied or is supplied
+ but is None retrying does not occur.
+
+--retrycodes::
+ a sequence of errorcodes (values of e.errno) for
+ which it should retry. See the doc on
+ URLGrabError for more details on this. retrycodes
+ defaults to -1,2,4,5,6,7 if not specified
+ explicitly.
+
+
+MODULE USE EXAMPLES
+-------------------
+In its simplest form, urlgrabber can be a replacement for urllib2's
+open, or even python's file if you're just reading:
+..................................
+ from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()
+..................................
+
+Here, the url can be http, https, ftp, or file. It's also pretty smart
+so if you just give it something like /tmp/foo, it will
+figure it out. For even more fun, you can also do:
+
+..................................
+ from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string
+..................................
+
+Now, like urllib2, what's really happening here is that you're using a
+module-level object (called a grabber) that kind of serves as a
+default. That's just fine, but you might want to get your own private
+version for a couple of reasons:
+
+..................................
+* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior
+..................................
+
+Therefore, you're probably better off making your own. This also gives
+you lots of flexibility for later, as you'll see:
+
+..................................
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)
+..................................
+
+This is nice because you can specify options when you create the
+grabber. For example, let's turn on simple reget mode so that if we
+have part of a file, we only need to fetch the rest:
+
+..................................
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)
+..................................
+
+The available options are listed in the module documentation, and can
+usually be specified as a default at the grabber-level or as options
+to the method:
+
+ from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url, filename=None, reget=None)
+
+
+AUTHORS
+-------
+Written by:
+Michael D. Stenner <mstenner at linux.duke.edu>
+Ryan Tomayko <rtomayko at naeblis.cx>
+
+This manual page was written by Kevin Coyner <kevin at rustybear.com> for
+the Debian system (but may be used by others). It borrows heavily on
+the documentation included in the urlgrabber module. Permission is granted
+to copy, distribute and/or modify this document under the terms of
+the GNU General Public License, Version 2 any later version published
+by the Free Software Foundation.
+
+
+RESOURCES
+---------
+Main web site: http://linux.duke.edu/projects/urlgrabber/[]
diff --git a/urlgrabber.xml b/urlgrabber.xml
new file mode 100644
index 0000000..fe67a3a
--- /dev/null
+++ b/urlgrabber.xml
@@ -0,0 +1,201 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+<refentry>
+<refmeta>
+<refentrytitle>urlgrabber</refentrytitle>
+<manvolnum>1</manvolnum>
+</refmeta>
+<refnamediv>
+ <refname>urlgrabber</refname>
+ <refpurpose>a high-level cross-protocol url-grabber.</refpurpose>
+</refnamediv>
+<refsynopsisdiv>
+<simpara><emphasis>urlgrabber</emphasis> [OPTIONS] URL [FILE]</simpara>
+
+</refsynopsisdiv>
+<refsect1>
+<title>DESCRIPTION</title>
+<simpara>urlgrabber is a binary program and python module for fetching files. It is
+designed to be used in programs that need common (but not necessarily simple)
+url-fetching features.</simpara>
+
+</refsect1>
+<refsect1>
+<title>OPTIONS</title>
+<variablelist>
+<varlistentry>
+<term>
+--help, -h
+</term>
+<listitem>
+<simpara>
+ help page specifying available options to the binary program.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--copy-local
+</term>
+<listitem>
+<simpara>
+ ignored except for file:// urls, in which case
+ it specifies whether urlgrab should still make
+ a copy of the file, or simply point to the
+ existing copy.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--throttle=NUMBER
+</term>
+<listitem>
+<simpara>
+ if it's an int, it's the bytes/second throttle
+ limit. If it's a float, it is first multiplied
+ by bandwidth. If throttle == 0, throttling is
+ disabled. If None, the module-level default
+ (which can be set with set_throttle) is used.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--bandwidth=NUMBER
+</term>
+<listitem>
+<simpara>
+ the nominal max bandwidth in bytes/second. If
+ throttle is a float and bandwidth == 0,
+ throttling is disabled. If None, the
+ module-level default (which can be set with
+ set_bandwidth) is used.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--range=RANGE
+</term>
+<listitem>
+<simpara>
+ a tuple of the form first_byte,last_byte
+ describing a byte range to retrieve. Either or
+ both of the values may be specified. If
+ first_byte is None, byte offset 0 is assumed.
+ If last_byte is None, the last byte available
+ is assumed. Note that both first and last_byte
+ values are inclusive so a range of (10,11)
+ would return the 10th and 11th bytes of the
+ resource.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--user-agent=STR
+</term>
+<listitem>
+<simpara>
+ the user-agent string provide if the url is HTTP.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--retry=NUMBER
+</term>
+<listitem>
+<simpara>
+ the number of times to retry the grab before
+ bailing. If this is zero, it will retry
+ forever. This was intentional… really, it was
+ :). If this value is not supplied or is supplied
+ but is None retrying does not occur.
+</simpara>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>
+--retrycodes
+</term>
+<listitem>
+<simpara>
+ a sequence of errorcodes (values of e.errno) for
+ which it should retry. See the doc on
+ URLGrabError for more details on this. retrycodes
+ defaults to -1,2,4,5,6,7 if not specified
+ explicitly.
+</simpara>
+</listitem>
+</varlistentry>
+</variablelist>
+</refsect1>
+<refsect1>
+<title>MODULE USE EXAMPLES</title>
+<simpara>In its simplest form, urlgrabber can be a replacement for urllib2's
+open, or even python's file if you're just reading:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber import urlopen
+ fo = urlopen(url)
+ data = fo.read()
+ fo.close()</literallayout>
+<simpara>Here, the url can be http, https, ftp, or file. It's also pretty smart
+so if you just give it something like /tmp/foo, it will
+figure it out. For even more fun, you can also do:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber import urlopen
+ local_filename = urlgrab(url) # grab a local copy of the file
+ data = urlread(url) # just read the data into a string</literallayout>
+<simpara>Now, like urllib2, what's really happening here is that you're using a
+module-level object (called a grabber) that kind of serves as a
+default. That's just fine, but you might want to get your own private
+version for a couple of reasons:</simpara>
+
+<literallayout class="monospaced">* it's a little ugly to modify the default grabber because you have to
+ reach into the module to do it
+* you could run into conflicts if different parts of the code
+ modify the default grabber and therefore expect different
+ behavior</literallayout>
+<simpara>Therefore, you're probably better off making your own. This also gives
+you lots of flexibility for later, as you'll see:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber.grabber import URLGrabber
+ g = URLGrabber()
+ data = g.urlread(url)</literallayout>
+<simpara>This is nice because you can specify options when you create the
+grabber. For example, let's turn on simple reget mode so that if we
+have part of a file, we only need to fetch the rest:</simpara>
+
+<literallayout class="monospaced"> from urlgrabber.grabber import URLGrabber
+ g = URLGrabber(reget='simple')
+ local_filename = g.urlgrab(url)</literallayout>
+<simpara>The available options are listed in the module documentation, and can
+usually be specified as a default at the grabber-level or as options
+to the method:</simpara>
+
+<literallayout class="monospaced">from urlgrabber.grabber import URLGrabber
+g = URLGrabber(reget='simple')
+local_filename = g.urlgrab(url, filename=None, reget=None)</literallayout>
+</refsect1>
+<refsect1>
+<title>AUTHORS</title>
+<simpara>Written by:
+Michael D. Stenner <mstenner at linux.duke.edu>
+Ryan Tomayko <rtomayko at naeblis.cx></simpara>
+
+<simpara>This manual page was written by Kevin Coyner <kevin at rustybear.com> for
+the Debian system (but may be used by others). It borrows heavily on
+the documentation included in the urlgrabber module. Permission is granted
+to copy, distribute and/or modify this document under the terms of
+the GNU General Public License, Version 2 any later version published
+by the Free Software Foundation.</simpara>
+
+</refsect1>
+<refsect1>
+<title>RESOURCES</title>
+<simpara>Main web site: <ulink url="http://linux.duke.edu/projects/urlgrabber/">http://linux.duke.edu/projects/urlgrabber/</ulink></simpara>
+
+</refsect1>
+</refentry>
diff --git a/watch b/watch
new file mode 100644
index 0000000..4010fbc
--- /dev/null
+++ b/watch
@@ -0,0 +1,2 @@
+version=3
+http://urlgrabber.baseurl.org/download/urlgrabber-(.*)\.tar\.gz
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/python-modules/packages/urlgrabber.git
More information about the Python-modules-commits
mailing list