[Python-modules-commits] r7800 - in packages/parallelpython/trunk/debian (4 files)

morph at users.alioth.debian.org morph at users.alioth.debian.org
Thu Mar 5 23:04:38 UTC 2009


    Date: Thursday, March 5, 2009 @ 23:04:37
  Author: morph
Revision: 7800

* New upstream release
* debian/rules
  - updated manpage location
  - updated html doc location, adding example.config
* debian/copyright
  - extended upstream copyright years
* debian/patches/10_remove_default_password.dpatch
  - adapted to new upstream code

Modified:
  packages/parallelpython/trunk/debian/changelog
  packages/parallelpython/trunk/debian/copyright
  packages/parallelpython/trunk/debian/patches/10_remove_default_password.dpatch
  packages/parallelpython/trunk/debian/rules

Modified: packages/parallelpython/trunk/debian/changelog
===================================================================
--- packages/parallelpython/trunk/debian/changelog	2009-03-05 23:03:27 UTC (rev 7799)
+++ packages/parallelpython/trunk/debian/changelog	2009-03-05 23:04:37 UTC (rev 7800)
@@ -1,3 +1,16 @@
+parallelpython (1.5.7-1) UNRELEASED; urgency=low
+
+  * New upstream release
+  * debian/rules
+    - updated manpage location
+    - updated html doc location, adding example.config
+  * debian/copyright
+    - extended upstream copyright years
+  * debian/patches/10_remove_default_password.dpatch
+    - adapted to new upstream code
+
+ -- Sandro Tosi <morph at debian.org>  Tue, 03 Mar 2009 22:18:33 +0100
+
 parallelpython (1.5.6-2) unstable; urgency=low
 
   * Uploading to unstable, since Lenny has been released

Modified: packages/parallelpython/trunk/debian/copyright
===================================================================
--- packages/parallelpython/trunk/debian/copyright	2009-03-05 23:03:27 UTC (rev 7799)
+++ packages/parallelpython/trunk/debian/copyright	2009-03-05 23:04:37 UTC (rev 7800)
@@ -9,7 +9,7 @@
 
 Copyright: 
 
-    Copyright (c) 2005-2008 Vitalii Vanovschi
+    Copyright (c) 2005-2009 Vitalii Vanovschi
 
 License:
 

Modified: packages/parallelpython/trunk/debian/patches/10_remove_default_password.dpatch
===================================================================
--- packages/parallelpython/trunk/debian/patches/10_remove_default_password.dpatch	2009-03-05 23:03:27 UTC (rev 7799)
+++ packages/parallelpython/trunk/debian/patches/10_remove_default_password.dpatch	2009-03-05 23:04:37 UTC (rev 7800)
@@ -1,22 +1,35 @@
 #! /bin/sh /usr/share/dpatch/dpatch-run
-## 10_remove_default_password.dpatch by Sandro Tosi <matrixhasu at gmail.com>
+## 10_remove_default_password.dpatch by Sandro Tosi <morph at debian.org>
 ##
 ## All lines beginning with `## DP:' are a description of the patch.
-## DP: This patch removes the default password from pp code, requiring users to explicitly write one; it updates the documentation along the source code
+## DP: This patch removes the default password from pp code, requiring users to
+## DP: explicitly write one; it updates the documentation along the source code
 
 @DPATCH@
+diff -urNad parallelpython~/doc/ppdoc.html parallelpython/doc/ppdoc.html
+--- parallelpython~/doc/ppdoc.html	2009-02-23 10:00:09.000000000 +0100
++++ parallelpython/doc/ppdoc.html	2009-03-05 23:47:57.362380774 +0100
+@@ -42,7 +42,7 @@
+ -s secret          : secret for authentication
+ -t seconds         : timeout to exit if no connections with clients exist
+ </pre>
+-<hr /><h1 id="COMMANDLINE">&nbsp; Security and secret key<a name="SECURITY" title="SECURITY"></a></h1><p>&nbsp;Due to the security concerns it is highly recommended to run ppserver.py with an non-trivial secret key (-s command line argument) which should be paired with the matching <em>secret</em> keyword of PP Server class constructor. Since PP 1.5.3 it is possible to set secret key by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong> which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in .pythonrc.py could be overridden by command line argument (for ppserver.py) and <em>secret</em> keyword (for PP Server class constructor). </p>
++<hr /><h1 id="COMMANDLINE">&nbsp; Security and secret key<a name="SECURITY" title="SECURITY"></a></h1><p>Due to the security concerns, in Debian we decided to disable default password authentication, and require to run <strong>ppserver</strong> with a non-trivial secret key (<trong>-s</strong> command line argument) which should be paired with the matching <em>secret</em> keyword of PP Server class constructor. An alternative way to set a secret key is by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong> which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in <strong>.pythonrc.py</strong> could be overridden by command line argument (for <strong>ppserver</strong>) and <em>secret</em> keyword (for PP Server class constructor). Note that passing the password on the command line allows every user to see it (e.g. using <strong>ps(1)</strong>), and that running it on an open/untrusted network can be a security problem as traffic, including the password, is not encrypted.</p>
+ 
+ 			</td>
+ 		</tr>
 diff -urNad parallelpython~/pp.py parallelpython/pp.py
---- parallelpython~/pp.py	2008-08-12 18:27:10.000000000 +0200
-+++ parallelpython/pp.py	2008-08-16 23:10:52.851082230 +0200
-@@ -237,7 +237,6 @@
+--- parallelpython~/pp.py	2009-02-24 07:34:00.000000000 +0100
++++ parallelpython/pp.py	2009-03-05 23:40:09.438381229 +0100
+@@ -274,7 +274,6 @@
      """
  
      default_port = 60000
 -    default_secret = "epo20pdosl;dksldkmm"
  
      def __init__(self, ncpus="autodetect", ppservers=(), secret=None,
-             loglevel=logging.WARNING, logstream=sys.stderr):
-@@ -248,9 +247,8 @@
+             loglevel=logging.WARNING, logstream=sys.stderr,
+@@ -286,9 +285,8 @@
                     the number of processors in the system
             ppservers - list of active parallel python execution servers
                     to connect with
@@ -27,8 +40,8 @@
 +                    command-line or configuration file
             loglevel - logging level
             logstream - log stream destination
- 
-@@ -321,7 +319,7 @@
+            restart - wheather to restart worker process after each task completion
+@@ -365,7 +363,7 @@
                  raise TypeError("secret must be of a string type")
              self.secret = str(secret)
          else:
@@ -37,52 +50,21 @@
          self.__connect()
          self.__creation_time = time.time()
          logging.info("pp local server started with %d workers"
-diff -urNad parallelpython~/ppdoc.html parallelpython/ppdoc.html
---- parallelpython~/ppdoc.html	2008-08-12 18:40:52.000000000 +0200
-+++ parallelpython/ppdoc.html	2008-08-16 23:08:47.331929304 +0200
-@@ -22,7 +22,7 @@
- <p> <table border="0" cellspacing="0" cellpadding="2" width="100%" summary="section"> <tbody><tr bgcolor="#ffc8d8"> <td colspan="3" valign="bottom">&nbsp;<br /> <font face="helvetica, arial" color="#000000"><a name="Template" title="Template"></a>class <strong>Template</strong></font></td></tr>      <tr bgcolor="#ffc8d8"><td rowspan="2">&nbsp;&nbsp;&nbsp;</td> <td colspan="2"><a href="#Template">Template</a>&nbsp;class<br />&nbsp;</td></tr>  <tr><td>&nbsp;</td> <td width="100%">Methods defined here:<br /> <dl><dt><a name="Template-__init__" title="Template-__init__"></a><strong>__init__</strong>(self, job_server, func, depfuncs<font color="#909090">=()</font>, modules<font color="#909090">=()</font>, callback<font color="#909090">=None</font>, callbackargs<font color="#909090">=()</font>, group<font color="#909090">=&#39;default&#39;</font>, globals<font color="#909090">=None</font>)</dt><dd>Creates&nbsp;<a href="#Template">Template</a>&nbsp;instance<br />  &nbsp;<br /> jobs_server&nbsp;-&nbsp;pp&nbsp;server&nbsp;for&nbsp;submitting&nbsp;jobs<br /> func&nbsp;-&nbsp;function&nbsp;to&nbsp;be&nbsp;executed<br /> depfuncs&nbsp;-&nbsp;tuple&nbsp;with&nbsp;functions&nbsp;which&nbsp;might&nbsp;be&nbsp;called&nbsp;from&nbsp;&#39;func&#39;<br />  modules&nbsp;-&nbsp;tuple&nbsp;with&nbsp;module&nbsp;names&nbsp;to&nbsp;import<br /> callback&nbsp;-&nbsp;callback&nbsp;function&nbsp;which&nbsp;will&nbsp;be&nbsp;called&nbsp;with&nbsp;argument&nbsp;<br />  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;list&nbsp;equal&nbsp;to&nbsp;callbackargs+(result,)&nbsp;<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;as&nbsp;soon&nbsp;as&nbsp;calculation&nbsp;is&nbsp;done<br /> callbackargs&nbsp;-&nbsp;additional&nbsp;arguments&nbsp;for&nbsp;callback&nbsp;function<br />  group&nbsp;-&nbsp;job&nbsp;group,&nbsp;is&nbsp;used&nbsp;when&nbsp;wait(group)&nbsp;is&nbsp;called&nbsp;to&nbsp;wait&nbsp;for<br /> jobs&nbsp;in&nbsp;a&nbsp;given&nbsp;group&nbsp;to&nbsp;finish<br />  globals&nbsp;-&nbsp;dictionary&nbsp;from&nbsp;which&nbsp;all&nbsp;modules,&nbsp;functions&nbsp;and&nbsp;classes<br /> will&nbsp;be&nbsp;imported,&nbsp;for&nbsp;instance:&nbsp;globals=globals()</dd></dl>  <dl><dt><a name="Template-submit" title="Template-submit"></a><strong>submit</strong>(self, *args)</dt><dd>Submits&nbsp;function&nbsp;with&nbsp;*arg&nbsp;arguments&nbsp;to&nbsp;the&nbsp;execution&nbsp;queue</dd></dl>   </td></tr></tbody></table></p>
- 
- 
--<p> <table border="0" cellspacing="0" cellpadding="2" width="100%" summary="section">  <tbody><tr bgcolor="#55aa55"> <td colspan="3" valign="bottom">&nbsp;<br /> <font face="helvetica, arial" color="#ffffff"><strong>Data</strong></font></td></tr>      <tr><td bgcolor="#55aa55">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td><td>&nbsp;</td> <td width="100%"><strong>copyright</strong> = &#39;Copyright (c) 2005-2008 Vitalii Vanovschi. All rights reserved&#39;<br /> <strong>version</strong> = &#39;1.5.4&#39;</td></tr></tbody></table>      </p><hr /><h1 id="QUICKSMP">&nbsp; Quick start guide, SMP<br /></h1> <p>1) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>2) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system </p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server()&nbsp;</strong></p><p>3) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>4) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p> <hr /><h1 id="QUICKCLUSTERS">&nbsp; Quick start guide, clusters&nbsp; </h1><p><em><strong>On the nodes</strong></em> <br /></p><p>1) Start parallel python execution server on all your remote computational nodes:</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py </strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py</strong></p><p><em><strong>On the client</strong></em> <br /></p><p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Create a list of all the nodes in your cluster (computers where you&#39;ve run ppserver.py) </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;node-1&quot;, &quot;node-2&quot;, &quot;node-3&quot;)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system and list of ppservers to connect with :</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a></p> <hr /><h1 id="QUICKCLUSTERSAUTO">&nbsp; Quick start guide, clusters with autodiscovery<br /> </h1><p><em><strong>On the nodes</strong></em>&nbsp;</p><p>1) Start parallel python execution server on all your remote computational nodes:</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py -a<br /> </strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py -a</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py -a<br /></strong></p><p><em><strong>On the client</strong></em></p><p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Set ppservers list to auto-discovery: </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;*&quot;,)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system and list of ppservers to connect with :</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a>&nbsp; </p><hr /><h1 id="ADVANCEDCLUSTERS">&nbsp;&nbsp;&nbsp; Advanced guide, clusters&nbsp; </h1> <p><em><strong>On the nodes</strong></em> &nbsp;</p><p>1) Start parallel python execution server on all your remote computational nodes (listen to a given port 35000,<br /> and local network interface only, accept only connections which know correct secret):</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py -p 35000 -i 192.168.0.101 -s &quot;mysecret&quot;<br /></strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py -p 35000 -i 192.168.0.102</strong><strong> -s &quot;mysecret&quot;</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py -p 35000 -i 192.168.0.103</strong><strong> -s &quot;mysecret&quot;</strong></p><p><em><strong>On the client</strong></em> <br /></p> <p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Create a list of all the nodes in your cluster (computers where you&#39;ve run ppserver.py) </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;node-1:35000&quot;, &quot;node-2:</strong><strong>35000</strong><strong>&quot;, &quot;node-3:</strong><strong>35000</strong><strong>&quot;)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system, <br />list of ppservers to connect with and secret key to authorize the connection:</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>, secret=&quot;</strong><strong>mysecret</strong><strong>&quot;)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;7) Print the execution statistics:<br /></p><p><strong>&nbsp;&nbsp;&nbsp; job_server.print_stats()</strong></p><p>To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p><hr /><h1 id="COMMANDLINE">&nbsp; Command line options, ppserver.py </h1> <pre>Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]<br /> Options:<br /> -h                 : this help message<br /> -d                 : debug<br /> -a                 : enable auto-discovery service<br /> -i interface       : interface to listen<br /> -b broadcast       : broadcast address for auto-discovery service<br /> -p port            : port to listen<br /> -w nworkers        : number of workers to start<br /> -s secret          : secret for authentication<br /> -t seconds         : timeout to exit if no connections with clients exist<br />  </pre><hr /><h1 id="COMMANDLINE">&nbsp; Security and secret key<a name="SECURITY" title="SECURITY"></a></h1><p>&nbsp;Due to the security concerns it is highly recommended to run ppserver.py with an non-trivial secret key (-s command line argument) which should be paired with the matching <em>secret</em> keyword of PP Server class constructor. Since PP 1.5.3 it is possible to set secret key by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong> which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in .pythonrc.py could be overridden by command line argument (for ppserver.py) and <em>secret</em> keyword (for PP Server class constructor). </p>
-+<p> <table border="0" cellspacing="0" cellpadding="2" width="100%" summary="section">  <tbody><tr bgcolor="#55aa55"> <td colspan="3" valign="bottom">&nbsp;<br /> <font face="helvetica, arial" color="#ffffff"><strong>Data</strong></font></td></tr>      <tr><td bgcolor="#55aa55">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</td><td>&nbsp;</td> <td width="100%"><strong>copyright</strong> = &#39;Copyright (c) 2005-2008 Vitalii Vanovschi. All rights reserved&#39;<br /> <strong>version</strong> = &#39;1.5.4&#39;</td></tr></tbody></table>      </p><hr /><h1 id="QUICKSMP">&nbsp; Quick start guide, SMP<br /></h1> <p>1) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>2) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system </p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server()&nbsp;</strong></p><p>3) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>4) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p> <hr /><h1 id="QUICKCLUSTERS">&nbsp; Quick start guide, clusters&nbsp; </h1><p><em><strong>On the nodes</strong></em> <br /></p><p>1) Start parallel python execution server on all your remote computational nodes:</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py </strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py</strong></p><p><em><strong>On the client</strong></em> <br /></p><p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Create a list of all the nodes in your cluster (computers where you&#39;ve run ppserver.py) </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;node-1&quot;, &quot;node-2&quot;, &quot;node-3&quot;)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system and list of ppservers to connect with :</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a></p> <hr /><h1 id="QUICKCLUSTERSAUTO">&nbsp; Quick start guide, clusters with autodiscovery<br /> </h1><p><em><strong>On the nodes</strong></em>&nbsp;</p><p>1) Start parallel python execution server on all your remote computational nodes:</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py -a<br /> </strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py -a</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py -a<br /></strong></p><p><em><strong>On the client</strong></em></p><p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Set ppservers list to auto-discovery: </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;*&quot;,)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system and list of ppservers to connect with :</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a>&nbsp; </p><hr /><h1 id="ADVANCEDCLUSTERS">&nbsp;&nbsp;&nbsp; Advanced guide, clusters&nbsp; </h1> <p><em><strong>On the nodes</strong></em> &nbsp;</p><p>1) Start parallel python execution server on all your remote computational nodes (listen to a given port 35000,<br /> and local network interface only, accept only connections which know correct secret):</p><p><strong>&nbsp;&nbsp;&nbsp; node-1&gt; ./ppserver.py -p 35000 -i 192.168.0.101 -s &quot;mysecret&quot;<br /></strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-2&gt; ./ppserver.py -p 35000 -i 192.168.0.102</strong><strong> -s &quot;mysecret&quot;</strong></p><p><strong>&nbsp;&nbsp;&nbsp; node-3&gt; ./ppserver.py -p 35000 -i 192.168.0.103</strong><strong> -s &quot;mysecret&quot;</strong></p><p><em><strong>On the client</strong></em> <br /></p> <p>2) Import pp module:</p><p><strong>&nbsp;&nbsp;&nbsp; import pp</strong></p><p>3)&nbsp; Create a list of all the nodes in your cluster (computers where you&#39;ve run ppserver.py) </p><p><strong>&nbsp;&nbsp;&nbsp; ppservers=(&quot;node-1:35000&quot;, &quot;node-2:</strong><strong>35000</strong><strong>&quot;, &quot;node-3:</strong><strong>35000</strong><strong>&quot;)</strong><br /></p><p>4) Start pp execution server with the number of workers set to&nbsp;the&nbsp;number&nbsp;of&nbsp;processors&nbsp;in&nbsp;the&nbsp;system, <br />list of ppservers to connect with and secret key to authorize the connection:</p><p><strong>&nbsp;&nbsp;&nbsp; job_server = pp.Server(</strong><strong>ppservers=</strong><strong>ppservers</strong><strong>, secret=&quot;</strong><strong>mysecret</strong><strong>&quot;)&nbsp;</strong></p><p>5) Submit all the tasks for parallel execution:</p><p><strong>&nbsp;&nbsp;&nbsp; f1 = job_server.submit(func1, args1, depfuncs1, modules1)</strong></p><p><strong>&nbsp;&nbsp;&nbsp; f2 = job_server.submit(func1, args2, depfuncs1, modules1) </strong></p><p><strong>&nbsp;&nbsp;&nbsp; f3 = job_server.submit(func2, args3, depfuncs2, modules2) </strong><br /> </p><p>&nbsp;&nbsp; ...etc...<br /></p><p>6) Retrieve the results as needed:</p><p><strong>&nbsp;&nbsp;&nbsp; r1 = f1()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r2 = f2()</strong></p><p><strong>&nbsp;&nbsp;&nbsp; r3 = f3()&nbsp;</strong> </p><p>&nbsp;&nbsp;&nbsp; ...etc...</p><p>&nbsp;7) Print the execution statistics:<br /></p><p><strong>&nbsp;&nbsp;&nbsp; job_server.print_stats()</strong></p><p>To find out how to achieve efficient parallelization with pp please take a look at <a href="http://www.parallelpython.com/content/view/17/31/" title="Parallel Python Implementation Examples">examples</a> </p><hr /><h1 id="COMMANDLINE">&nbsp; Command line options, ppserver.py </h1> <pre>Usage: ppserver.py [-hda] [-i interface] [-b broadcast] [-p port] [-w nworkers] [-s secret] [-t seconds]<br /> Options:<br /> -h                 : this help message<br /> -d                 : debug<br /> -a                 : enable auto-discovery service<br /> -i interface       : interface to listen<br /> -b broadcast       : broadcast address for auto-discovery service<br /> -p port            : port to listen<br /> -w nworkers        : number of workers to start<br /> -s secret          : secret for authentication<br /> -t seconds         : timeout to exit if no connections with clients exist<br />  </pre><hr /><h1 id="COMMANDLINE">&nbsp; Security and secret key<a name="SECURITY" title="SECURITY"></a></h1><p>Due to the security concerns, in Debian we decided to disable default password authentication, and require to run <strong>ppserver</strong> with a non-trivial secret key (<trong>-s</strong> command line argument) which should be paired with the matching <em>secret</em> keyword of PP Server class constructor. An alternative way to set a secret key is by assigning <strong>pp_secret</strong> variable in the configuration file <strong>.pythonrc.py</strong> which should be located in the user home directory (please make this file readable and writable only by user). The secret key set in <strong>.pythonrc.py</strong> could be overridden by command line argument (for <strong>ppserver</strong>) and <em>secret</em> keyword (for PP Server class constructor). Note that passing the password on the command line allows every user to see it (e.g. using <strong>ps(1)</strong>), and that running it on an open/untrusted network can be a security problem as traffic, including the password, is not encrypted.</p>
- 
- 			</td>
- 		</tr>
-diff -urNad parallelpython~/ppserver.1 parallelpython/ppserver.1
---- parallelpython~/ppserver.1	2008-06-03 05:48:34.000000000 +0200
-+++ parallelpython/ppserver.1	2008-08-16 23:08:47.331929304 +0200
-@@ -40,12 +40,11 @@
- documentation, examples and support forums
- .br
- .SH SECURITY
--Due to the security concerns it is highly recommended to run ppserver.py with an non-trivial secret key (-s command line argument) which should be paired with the matching secret keyword of PP Server class constructor. An alternative way to set a secret key is by assigning 
--.B pp_secret 
--variable in the configuration file 
--.B .pythonrc.py 
--which should be located in the user home directory (please make this file readable and writable only by user). 
--The secret key set in .pythonrc.py could be overridden by command line argument (for ppserver.py) and secret keyword (for PP Server class constructor).
-+Due to the security concerns, in Debian we decided to disable default password authentication, and require to run
-+.B ppserver
-+with an non\-trivial secret key (\fB\-s\fR command line argument) which should be paired with the matching secret keyword of PP Server class constructor. An alternative way to set a secret key is by assigning \fBpp_secret\fR variable in the configuration file \fB.pythonrc.py\fR which should be located in the user home directory (please make this file readable and writable only by user). 
-+The secret key set in \fB.pythonrc.py\fR could be overridden by command line argument (for \fBppserver\fR) and secret keyword (for PP Server class constructor).
-+Note that passing the password on the command line allows every user to see it (e.g. using \fBps(1)\fR ) and that running it on an open/untrusted network can be a security problem as traffic, including the password, is not encrypted.
- .SH AUTHOR
- This manual page was written by Sandro Tosi <matrixhasu at gmail.com>,
- for the Debian project (but may be used by others).
 diff -urNad parallelpython~/ppserver.py parallelpython/ppserver.py
---- parallelpython~/ppserver.py	2008-08-12 18:27:19.000000000 +0200
-+++ parallelpython/ppserver.py	2008-08-16 23:08:47.331929304 +0200
-@@ -56,7 +56,11 @@
-     def __init__(self, ncpus="autodetect", interface="0.0.0.0",
+--- parallelpython~/ppserver.py	2009-02-23 10:04:29.000000000 +0100
++++ parallelpython/ppserver.py	2009-03-05 23:43:13.162381054 +0100
+@@ -66,8 +66,12 @@
                  broadcast="255.255.255.255", port=None, secret=None,
-                 timeout=None, loglevel=logging.WARNING):
--        Server.__init__(self, ncpus, secret=secret, loglevel=loglevel)
-+	try:
-+        	Server.__init__(self, ncpus, secret=secret, loglevel=loglevel)
-+	except Exception, e:
-+		print >> sys.stderr, str(e)
-+		sys.exit(1)
+                 timeout=None, loglevel=logging.WARNING, restart=False,
+                 proto=0):
+-        Server.__init__(self, ncpus, secret=secret, loglevel=loglevel,
+-                restart=restart, proto=proto)
++        try:
++            Server.__init__(self, ncpus, secret=secret, loglevel=loglevel,
++                    restart=restart, proto=proto)
++        except Exception, e:
++            print >> sys.stderr, str(e)
++            sys.exit(1)
          self.host = interface
          self.bcast = broadcast
          if port is not None:

Modified: packages/parallelpython/trunk/debian/rules
===================================================================
--- packages/parallelpython/trunk/debian/rules	2009-03-05 23:03:27 UTC (rev 7799)
+++ packages/parallelpython/trunk/debian/rules	2009-03-05 23:04:37 UTC (rev 7800)
@@ -45,9 +45,9 @@
 	dh_testdir
 	dh_testroot
 	dh_installchangelogs 	CHANGELOG
-	dh_installdocs		ppdoc.html
+	dh_installdocs		doc/ppdoc.html doc/example.config 
 	dh_installexamples	examples/*
-	dh_installman		ppserver.1
+	dh_installman		doc/ppserver.1
 	dh_pysupport
 	dh_compress
 	dh_fixperms




More information about the Python-modules-commits mailing list