[Pkg-nginx-maintainers] Bug#869499: nginx: [PATCH] Automate modules watch & upgrade process
Mpampis Kostas
charkost.rb at gmail.com
Sun Jul 23 18:42:47 UTC 2017
Package: nginx
Severity: wishlist
Tags: patch
This patch introduces the debian/modules/uscan script and some helper
files which can be used to automate the modules watch & upgrade process.
The main script is executed as '$ debian/modules/uscan' without any arguments from
the root of the debianized nginx source tree where you see the debian/ directory.
It runs uscan for each nginx module listed in the newly deb822 formatted
debian/modules/README.Modules-version using the watchfiles in debian/modules/watch.
If a new version is available, it will ask you if you want to upgrade.
If you agree, it will download the tarball, place it in the parent directory
and upgrade the module's source.
After the upgrade, it will remove the files listed in the
Files-Excluded module field.
Example output:
$ debian/modules/uscan
Uscanning headers-more-nginx-module...up-to-date.
Uscanning nginx-development-kit...up-to-date.
Uscanning nginx-auth-pam...up-to-date.
Uscanning nginx-echo...up-to-date.
Uscanning nginx-lua...uscan: Newest version of nginx-lua on remote site is 0.10.8, local version is v0.10.7
(mangled local version is 0.10.7)
uscan: => Newer package available from
https://github.com/openresty/lua-nginx-module/archive/v0.10.8.tar.gz
Do you want to upgrade nginx-lua? [y/N]:
Uscanning nginx-upstream-fair...no watchfile available.
Uscanning nchan...uscan: Newest version of nchan on remote site is 1.1.7, local version is v1.0.8
(mangled local version is 1.0.8)
uscan: => Newer package available from
https://github.com/slact/nchan/archive/v1.1.7.tar.gz
Do you want to upgrade nchan? [y/N]:
Uscanning nginx-upload-progress...up-to-date.
Uscanning nginx-cache-purge...up-to-date.
Uscanning nginx-dav-ext-module...uscan: Newest version of nginx-dav-ext-module on remote site is 0.1.0, local version is v0.0.3
(mangled local version is 0.0.3)
uscan: => Newer package available from
https://github.com/arut/nginx-dav-ext-module/archive/v0.1.0.tar.gz
Do you want to upgrade nginx-dav-ext-module? [y/N]:
Uscanning ngx-fancyindex...up-to-date.
Uscanning ngx_http_substitutions_filter_module...up-to-date.
Uscanning nginx-rtmp...uscan: Newest version of nginx-rtmp on remote site is 1.2.0, local version is v1.1.11
(mangled local version is 1.1.11)
uscan: => Newer package available from
https://github.com/arut/nginx-rtmp-module/archive/v1.2.0.tar.gz
Do you want to upgrade nginx-rtmp? [y/N]: y
Removing Files-Excluded: debian/modules/nginx-rtmp/test
-------------- next part --------------
>From ddb17d64333ed2f64dce3906f7cdb308577ab000 Mon Sep 17 00:00:00 2001
From: Mpampis Kostas <charkost.rb at gmail.com>
Date: Sat, 22 Jul 2017 16:26:21 +0300
Subject: [PATCH] Automate modules watch & upgrade process
This patch introduces the debian/modules/uscan script and some helper
files which can be used to automate the modules watch & upgrade process.
The main script is executed as '$ debian/modules/uscan' without any arguments from
the root of the debianized nginx source tree where you see the debian/ directory.
It runs uscan for each nginx module listed in the newly deb822 formatted
debian/modules/README.Modules-version using the watchfiles in debian/modules/watch.
If a new version is available, it will ask you if you want to upgrade.
If you agree, it will download the tarball, place it in the parent directory
and upgrade the module's source.
After the upgrade, it will remove the files listed in the
Files-Excluded module field.
---
debian/modules/README.Modules-versions | 110 ++++++++++-----------
debian/modules/uscan | 78 +++++++++++++++
debian/modules/uupdate | 9 ++
debian/modules/watch/headers-more-nginx-module | 4 +
debian/modules/watch/nchan | 4 +
debian/modules/watch/nginx-auth-pam | 4 +
debian/modules/watch/nginx-cache-purge | 4 +
debian/modules/watch/nginx-dav-ext-module | 4 +
debian/modules/watch/nginx-development-kit | 4 +
debian/modules/watch/nginx-echo | 4 +
debian/modules/watch/nginx-lua | 4 +
debian/modules/watch/nginx-rtmp | 4 +
debian/modules/watch/nginx-upload-progress | 4 +
debian/modules/watch/ngx-fancyindex | 4 +
.../watch/ngx_http_substitutions_filter_module | 4 +
15 files changed, 190 insertions(+), 55 deletions(-)
create mode 100755 debian/modules/uscan
create mode 100755 debian/modules/uupdate
create mode 100644 debian/modules/watch/headers-more-nginx-module
create mode 100644 debian/modules/watch/nchan
create mode 100644 debian/modules/watch/nginx-auth-pam
create mode 100644 debian/modules/watch/nginx-cache-purge
create mode 100644 debian/modules/watch/nginx-dav-ext-module
create mode 100644 debian/modules/watch/nginx-development-kit
create mode 100644 debian/modules/watch/nginx-echo
create mode 100644 debian/modules/watch/nginx-lua
create mode 100644 debian/modules/watch/nginx-rtmp
create mode 100644 debian/modules/watch/nginx-upload-progress
create mode 100644 debian/modules/watch/ngx-fancyindex
create mode 100644 debian/modules/watch/ngx_http_substitutions_filter_module
diff --git a/debian/modules/README.Modules-versions b/debian/modules/README.Modules-versions
index c336064..4ef902e 100644
--- a/debian/modules/README.Modules-versions
+++ b/debian/modules/README.Modules-versions
@@ -1,69 +1,69 @@
-README for Modules versions
----------------------------
+# README for Modules versions
+# ---------------------------
- This file lists third party modules built with nginx in Debian, homepage and
- version.
+# This file lists third party modules built with nginx in Debian, homepage and
+# version.
- headers-more-nginx-module
- Homepage: https://github.com/agentzh/headers-more-nginx-module
- Version: v0.32
+Module: headers-more-nginx-module
+Homepage: https://github.com/agentzh/headers-more-nginx-module
+Version: v0.32
- nginx-development-kit
- Homepage: https://github.com/simpl/ngx_devel_kit/
- Version: v0.3.0
+Module: nginx-development-kit
+Homepage: https://github.com/simpl/ngx_devel_kit/
+Version: v0.3.0
- nginx-auth-pam
- Homepage: https://github.com/stogh/ngx_http_auth_pam_module
- Version: 1.5.1
+Module: nginx-auth-pam
+Homepage: https://github.com/stogh/ngx_http_auth_pam_module
+Version: v1.5.1
- nginx-echo
- Homepage: https://github.com/agentzh/echo-nginx-module
- Version: v0.60
- Patch: build-nginx-1.11.11.patch
+Module: nginx-echo
+Homepage: https://github.com/agentzh/echo-nginx-module
+Version: v0.60
+Patch: build-nginx-1.11.11.patch
- nginx-lua
- Homepage: https://github.com/openresty/lua-nginx-module
- Version: v0.10.7
- Patch: openssl-1.1.0.patch
- Patch: build-nginx-1.11.11.patch
+Module: nginx-lua
+Homepage: https://github.com/openresty/lua-nginx-module
+Version: v0.10.7
+Patch: openssl-1.1.0.patch
+Patch: build-nginx-1.11.11.patch
- nginx-upstream-fair
- Homepage: https://github.com/gnosek/nginx-upstream-fair
- Version: a18b409
- Patch: dynamic-module.patch
- Patch: openssl-1.1.0.patch
- Patch: drop-default-port.patch
+Module: nginx-upstream-fair
+Homepage: https://github.com/gnosek/nginx-upstream-fair
+Version: a18b409
+Patch: dynamic-module.patch
+Patch: openssl-1.1.0.patch
+Patch: drop-default-port.patch
- nchan
- Homepage: https://github.com/slact/nchan
- Version: 1.0.8
+Module: nchan
+Homepage: https://github.com/slact/nchan
+Version: v1.0.8
- nginx-upload-progress
- Homepage: https://github.com/masterzen/nginx-upload-progress-module
- rm -r debian/nginx-upload-progress/test
- Version: v0.9.2
+Module: nginx-upload-progress
+Homepage: https://github.com/masterzen/nginx-upload-progress-module
+Files-Excluded: debian/modules/nginx-upload-progress/test
+Version: v0.9.2
- nginx-cache-purge
- Homepage: https://github.com/FRiCKLE/ngx_cache_purge/
- Version: 2.3
- Patch: dynamic-module.patch
- Patch: segfault-1.11.6.patch
+Module: nginx-cache-purge
+Homepage: https://github.com/FRiCKLE/ngx_cache_purge/
+Version: 2.3
+Patch: dynamic-module.patch
+Patch: segfault-1.11.6.patch
- nginx-dav-ext-module
- Homepage: https://github.com/arut/nginx-dav-ext-module
- Version: v0.0.3
- Patch: dynamic-module.patch
+Module: nginx-dav-ext-module
+Homepage: https://github.com/arut/nginx-dav-ext-module
+Version: v0.0.3
+Patch: dynamic-module.patch
- ngx-fancyindex
- Homepage: https://github.com/aperezdc/ngx-fancyindex
- Version: v0.4.1
+Module: ngx-fancyindex
+Homepage: https://github.com/aperezdc/ngx-fancyindex
+Version: v0.4.1
- ngx_http_substitutions_filter_module
- Homepage: https://github.com/yaoweibin/ngx_http_substitutions_filter_module
- Version: v0.6.4
- Patch: dynamic-module.patch
+Module: ngx_http_substitutions_filter_module
+Homepage: https://github.com/yaoweibin/ngx_http_substitutions_filter_module
+Version: v0.6.4
+Patch: dynamic-module.patch
- nginx-rtmp
- Homepage: https://github.com/arut/nginx-rtmp-module
- rm -r debian/modules/nginx-rtmp-module/test
- Version: v1.1.11
+Module: nginx-rtmp
+Homepage: https://github.com/arut/nginx-rtmp-module
+Files-Excluded: debian/modules/nginx-rtmp/test
+Version: v1.1.11
diff --git a/debian/modules/uscan b/debian/modules/uscan
new file mode 100755
index 0000000..b432257
--- /dev/null
+++ b/debian/modules/uscan
@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+
+# debian/modules/uscan - scan/watch upstream nginx module sources for new releases
+#
+# This script is executed as '$ debian/modules/uscan' without any arguments from
+# the root of the debianized source tree where you see the debian/ directory.
+#
+# It runs uscan for each nginx module listed in debian/modules/README.Modules-version.
+#
+# If a new version is available, it will ask you if want to upgrade. If you agree
+# it will download the tarball, place it in the parent directory and upgrade
+# the module's source.
+#
+# After the upgrade, it will remove the files listed in the Files-Excluded field.
+
+import deb822
+import os
+import sys
+from subprocess import call
+
+MODULES_CTRL = file('debian/modules/README.Modules-versions')
+
+def prompt(query):
+ sys.stdout.write('%s [y/N]: ' % query)
+ choice = raw_input().lower()
+
+ return choice == 'y'
+
+
+def upgrade_module(watchfile, m):
+ uscan_upgrade_cmd = [
+ 'uscan', '--upstream-version', m['Version'],
+ '--watchfile', watchfile, '--no-symlink'
+ ]
+ upgrade_ret = call(uscan_upgrade_cmd, stdout=open(os.devnull, 'w'))
+
+ if 'Files-Excluded' in m and upgrade_ret == 0:
+ print 'Removing Files-Excluded: %s' % m['Files-Excluded']
+
+ files = m['Files-Excluded'].split("\n")
+ files = [f.strip() for f in files]
+ files = filter(None, files)
+
+ rm_cmd = ['rm', '-r']
+ rm_cmd.extend(files)
+
+ call(rm_cmd)
+
+
+def main():
+ paragraphs = deb822.Deb822.iter_paragraphs(MODULES_CTRL)
+
+ for p in paragraphs:
+ sys.stdout.write('Uscanning %s...' % p['Module'])
+ sys.stdout.flush()
+
+ watchfile = 'debian/modules/watch/%s' % p['Module']
+
+ if not os.path.isfile(watchfile):
+ print 'no watchfile available.'
+ continue
+
+ uscan_check_cmd = [
+ 'uscan', '--upstream-version', p['Version'],
+ '--watchfile', watchfile, '--package',
+ p['Module']
+ ]
+ check_ret = call(uscan_check_cmd)
+
+ if check_ret == 0:
+ if prompt('Do you want to upgrade %s?' % p['Module']):
+ upgrade_module(watchfile, p)
+ else:
+ print 'up-to-date.'
+
+
+if __name__ == '__main__':
+ main()
diff --git a/debian/modules/uupdate b/debian/modules/uupdate
new file mode 100755
index 0000000..50df7e7
--- /dev/null
+++ b/debian/modules/uupdate
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+# debian/modules/uupdate - simple tar wrapper used by uscan to upgrade modules source
+
+MOD_TAR_NAME=$2
+MOD_TAR_VER=$4
+DESTDIR=debian/modules/$1
+
+tar --strip-components=1 -xf ../$MOD_TAR_NAME-$MOD_TAR_VER.tar.gz -C $DESTDIR
diff --git a/debian/modules/watch/headers-more-nginx-module b/debian/modules/watch/headers-more-nginx-module
new file mode 100644
index 0000000..68861a8
--- /dev/null
+++ b/debian/modules/watch/headers-more-nginx-module
@@ -0,0 +1,4 @@
+version=4
+opts="uversionmangle=s/0.261/0.26.1/,dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-headers-more-$1.tar.gz%" \
+ https://github.com/agentzh/headers-more-nginx-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate headers-more-nginx-module ngx-mod-headers-more
diff --git a/debian/modules/watch/nchan b/debian/modules/watch/nchan
new file mode 100644
index 0000000..6b95884
--- /dev/null
+++ b/debian/modules/watch/nchan
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-nchan-$1.tar.gz%" \
+ https://github.com/slact/nchan/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nchan ngx-mod-nchan
diff --git a/debian/modules/watch/nginx-auth-pam b/debian/modules/watch/nginx-auth-pam
new file mode 100644
index 0000000..1f30a96
--- /dev/null
+++ b/debian/modules/watch/nginx-auth-pam
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-http-auth-pam-$1.tar.gz%" \
+ https://github.com/stogh/ngx_http_auth_pam_module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-auth-pam ngx-mod-http-auth-pam
diff --git a/debian/modules/watch/nginx-cache-purge b/debian/modules/watch/nginx-cache-purge
new file mode 100644
index 0000000..24bdf4c
--- /dev/null
+++ b/debian/modules/watch/nginx-cache-purge
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-cache-purge-$1.tar.gz%" \
+ https://github.com/FRiCKLE/ngx_cache_purge/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-cache-purge ngx-mod-cache-purge
diff --git a/debian/modules/watch/nginx-dav-ext-module b/debian/modules/watch/nginx-dav-ext-module
new file mode 100644
index 0000000..32424a8
--- /dev/null
+++ b/debian/modules/watch/nginx-dav-ext-module
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-dav-ext-$1.tar.gz%" \
+ https://github.com/arut/nginx-dav-ext-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-dav-ext-module ngx-mod-dav-ext
diff --git a/debian/modules/watch/nginx-development-kit b/debian/modules/watch/nginx-development-kit
new file mode 100644
index 0000000..72899a6
--- /dev/null
+++ b/debian/modules/watch/nginx-development-kit
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-devel-kit-$1.tar.gz%" \
+ https://github.com/simpl/ngx_devel_kit/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-development-kit ngx-mod-devel-kit
diff --git a/debian/modules/watch/nginx-echo b/debian/modules/watch/nginx-echo
new file mode 100644
index 0000000..51d58a2
--- /dev/null
+++ b/debian/modules/watch/nginx-echo
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-echo-$1.tar.gz%" \
+ https://github.com/agentzh/echo-nginx-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-echo ngx-mod-echo
diff --git a/debian/modules/watch/nginx-lua b/debian/modules/watch/nginx-lua
new file mode 100644
index 0000000..642add5
--- /dev/null
+++ b/debian/modules/watch/nginx-lua
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%lua-nginx-module-$1.tar.gz%" \
+ https://github.com/openresty/lua-nginx-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-lua lua-nginx-module
diff --git a/debian/modules/watch/nginx-rtmp b/debian/modules/watch/nginx-rtmp
new file mode 100644
index 0000000..c2ba8a3
--- /dev/null
+++ b/debian/modules/watch/nginx-rtmp
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-rtmp-$1.tar.gz%" \
+ https://github.com/arut/nginx-rtmp-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-rtmp ngx-mod-rtmp
diff --git a/debian/modules/watch/nginx-upload-progress b/debian/modules/watch/nginx-upload-progress
new file mode 100644
index 0000000..08038f5
--- /dev/null
+++ b/debian/modules/watch/nginx-upload-progress
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-upload-progress-$1.tar.gz%" \
+ https://github.com/masterzen/nginx-upload-progress-module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate nginx-upload-progress ngx-mod-upload-progress
diff --git a/debian/modules/watch/ngx-fancyindex b/debian/modules/watch/ngx-fancyindex
new file mode 100644
index 0000000..8326c8b
--- /dev/null
+++ b/debian/modules/watch/ngx-fancyindex
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-fancyindex-$1.tar.gz%" \
+ https://github.com/aperezdc/ngx-fancyindex/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate ngx-fancyindex ngx-mod-fancyindex
diff --git a/debian/modules/watch/ngx_http_substitutions_filter_module b/debian/modules/watch/ngx_http_substitutions_filter_module
new file mode 100644
index 0000000..4696aa1
--- /dev/null
+++ b/debian/modules/watch/ngx_http_substitutions_filter_module
@@ -0,0 +1,4 @@
+version=4
+opts="dversionmangle=s/v//,filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%ngx-mod-http-substitutions-filter-$1.tar.gz%" \
+ https://github.com/yaoweibin/ngx_http_substitutions_filter_module/tags \
+ (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian debian/modules/uupdate ngx_http_substitutions_filter_module ngx-mod-http-substitutions-filter
--
2.11.0
More information about the Pkg-nginx-maintainers
mailing list