[Pkg-puppet-devel] [SCM] Puppet packaging for Debian branch, upstream, updated. 2.6.1rc1-141-gcdb2b90

[markus (none)]

The following commit has been merged in the upstream branch:
commit f54d843e4e585274f724c97f1b10288d8798a63b
Author: Brice Figureau <brice-puppet at daysofwonder.com>
Date:   Fri Jul 30 22:12:46 2010 +0200

    Fix #4461 - attempt to fix another performance issue
    
    During a profiling of a 2.6.1rc1 puppet master, I found that we
    spend a lot of time and efforts in Puppet::Util::Autoload#module_directories.
    Since this method is doing a bunch of filesystem access, this process is
    slow.
    
    In fact each time we were evaluating a resource or trying to find if a given
    resource was a builtin type we ended up scanning the whole module directories
    for the given environment.
    
    This patch attempts to fix this performance issue by caching the module_directories
    output for the either the time of the compilation or an agent configurer
    run (since this stuff looks like to be shared for both compilation and
    catalog evaluation).
    
    With this patch, my compilation time for 2k resources went from 5.91s to
    3.71s (second run each time to allievate parsing time)..
    
    Signed-off-by: Brice Figureau <brice-puppet at daysofwonder.com>
    
    Edited to fix a typo [#4434]
    
    Signed-off-by: Jesse Wolfe <jes5199 at gmail.com>

diff --git a/lib/puppet/configurer.rb b/lib/puppet/configurer.rb
index 327955b..31d31c2 100644
--- a/lib/puppet/configurer.rb
+++ b/lib/puppet/configurer.rb
@@ -158,6 +158,10 @@ class Puppet::Configurer
       return
     end
   ensure
+    # Make sure we forget the retained module_directories of any autoload
+    # we might have used.
+    Thread.current[:env_module_directories] = nil
+
     # Now close all of our existing http connections, since there's no
     # reason to leave them lying open.
     Puppet::Network::HttpPool.clear_http_instances
diff --git a/lib/puppet/parser/compiler.rb b/lib/puppet/parser/compiler.rb
index 61bb13c..98cad95 100644
--- a/lib/puppet/parser/compiler.rb
+++ b/lib/puppet/parser/compiler.rb
@@ -23,6 +23,7 @@ class Puppet::Parser::Compiler
     # We get these from the environment and only cache them in a thread 
     # variable for the duration of the compilation.
     Thread.current[:known_resource_types] = nil
+    Thread.current[:env_module_directories] = nil
  end
 
   attr_reader :node, :facts, :collections, :catalog, :node_scope, :resources, :relationships
diff --git a/lib/puppet/util/autoload.rb b/lib/puppet/util/autoload.rb
index c293ac1..f0dd0a5 100644
--- a/lib/puppet/util/autoload.rb
+++ b/lib/puppet/util/autoload.rb
@@ -131,11 +131,23 @@ class Puppet::Util::Autoload
     # We have to require this late in the process because otherwise we might have
     # load order issues.
     require 'puppet/node/environment'
-    Puppet::Node::Environment.new(env).modulepath.collect do |dir|
-      Dir.entries(dir).reject { |f| f =~ /^\./ }.collect { |f| File.join(dir, f) }
-    end.flatten.collect { |d| [File.join(d, "plugins"), File.join(d, "lib")] }.flatten.find_all do |d|
-      FileTest.directory?(d)
-    end
+
+    real_env = Puppet::Node::Environment.new(env)
+
+    # We're using a per-thread cache of said module directories, so that
+    # we don't scan the filesystem each time we try to load something with
+    # this autoload instance. But since we don't want to cache for the eternity
+    # this env_module_directories gets reset after the compilation on the master.
+    # This is also reset after an agent ran.
+    # One of the side effect of this change is that this module directories list will be
+    # shared among all autoload that we have running at a time. But that won't be an issue
+    # as by definition those directories are shared by all autoload.
+    Thread.current[:env_module_directories] ||= {}
+    Thread.current[:env_module_directories][real_env] ||= real_env.modulepath.collect do |dir|
+        Dir.entries(dir).reject { |f| f =~ /^\./ }.collect { |f| File.join(dir, f) }
+      end.flatten.collect { |d| [File.join(d, "plugins"), File.join(d, "lib")] }.flatten.find_all do |d|
+        FileTest.directory?(d)
+      end
   end
 
   def search_directories(env=nil)

-- 
Puppet packaging for Debian