Developing a Comprehensive Autoloader

May 1, 2009

Tutorials, Zend Framework

In this article, I’ll discuss the development and features of
Zend_Loader_Autoloader and its related functionality. However, the
main point of the article is to show the various concerns and design
decisions that go into developing a comprehensive autoloading solution for
your PHP applications. Autoloading, while seemingly a trivial optimization
task, has many facets that are often overlooked.

History

Since before the 1.0 release of Zend Framework, we’ve had
Zend_Loader::autoload() and
Zend_Loader::registerAutoload() (which registers the former with
PHP’s spl_autoload_register()). And during that time, they’ve been
basically broken for one use case or another.

At first, we simply attempted to load the class using straight
include calls. However, if this fails, you get a notice from PHP
indicating the include failed — and this would anger some purists who felt
that an autoloader should never raise any errors as there may be other
autoloaders further in the chain that can resolve the class.

So, we then tried using the suppression operator (‘@’). This gets rid of the
error notices (though they still show up in logs) — but has a really nasty
side effect: if there are parse or compilation errors when
attempting to load the class, nothing is reported, and you end up with a
blank white screen with no information.

We then tried to first check if the file was readable. However, this is a
really expensive process, often requiring multiple stat calls to the server;
the performance issues were enough to warrant returning to plain old error
suppression.

What it came down to was this: it was next to impossible to address all
concerns using this simplistic approach. And thus
Zend_Loader_Autoloader was born.

Zend_Loader_Autoloader: Goals and Design

The initial goals of this component were as follows:

  • Provide namespace matching. If the class namespace prefix is not in a
    list of registered namespaces, just return false immediately. This
    alleviates most issues from the start.
  • Allow the autoloader to act as a fallback autoloader. In the case where
    a team may be widely distributed, or using an undetermined set of namespace
    prefixes, the autoloader should still be configurable such that it will
    attempt to match any namespace prefix.
  • Allow toggling error suppression. We feel — and the greater PHP
    community does as well — that error suppression is a bad idea. It’s
    expensive, and it masks very real application problems. So, by default, we
    want it off. However, if a developer insists that it be on, we
    should allow toggling it on.
  • Allow specifying your own callback for autoloading. Some developers
    don’t want to use Zend_Loader::loadClass() for autoloading, but
    still want to make use of ZF’s mechanisms. We should allow specyfing an
    alternate callback for autoloading.
  • Allow manipulation of the spl_autoload callback chain. The
    purpose of this would be to allow specifying additional autoloaders — for
    instance, resource loaders for classes that don’t have a 1:1 mapping to the
    filesystem — to be registered before or after the primary ZF autoloader.

This last point proved to be a rather interesting one. As part of ZF’s
autoloading strategy, we developed Zend_Loader_Autoloader_Resource
and its more concrete cousin Zend_Application_Module_Autoloader;
these basically allow you to map a namespace prefix and component to a base
directory and subpath on that directory — which allow you do have trees
like the following:

.
|-- forms
|   `-- Guestbook.php        // Foo_Form_Guestbook
|-- models
|   |-- DbTable
|   |   `-- Guestbook.php    // Foo_Model_DbTable_Guestbook
|   |-- Guestbook.php        // Foo_Model_Guestbook
|   `-- GuestbookMapper.php  // Foo_Model_GuestbookMapper

Instances of these autoloaders then have an instance method,
autoload(), that we attempted to register with
spl_autoload(). The problem was that if you wanted to add an
autoloader before it in the stack, or insert one somewhere between it and
the default autoloader… it wouldn’t work. For some reason, the return
value of spl_autoload_functions() does not contain the same
callbacks as passed to it. The only ones that could be re-registered were
callbacks with static method calls — which we didn’t want to use so that we
could have multiple resource autoloaders for different areas of our
projects.

So, to solve this problem, we had to develop a registry within
Zend_Loader_Autoloader for maintaining these autoloader callbacks.
As it turned out, this was an excellent solution, as it allowed us to do
something else: multiple autoloaders could serve the same namespace prefix,
which allows us to selectively process only those autoloaders that match
that namespace prefix when attempting a match.

Zend_Loader_Autoloader: Migration

With the 1.8.0 release of Zend Framework, we’ve now marked
Zend_Loader::autoload() and
Zend_Loader::registerAutoload() as deprecated. The simplest way to
replace this solution in your application is as follows:

// If you had:
require_once 'Zend/Loader.php';
Zend_Loader::registerAutoload();

// In most cases, you can replace it with:
require_once 'Zend/Loader/Autoloader.php';
Zend_Loader_Autoloader::getInstance();

However, something to remember: Zend_Loader_Autoloader is a
namespace-based autoloader. By default, it only registers the Zend_
and ZendX_ namespaces. If you have additional namespaces, you need
to do one of two things:

  • Preferred: Register each namespace with the autoloader.
    You can do so with the registerNamespace() method, which
    accepts either a single namespace prefix or an array of them. Namespace
    prefixes should include the trailing underscore (‘_’) to reduce the
    chance of a collision.

    $autoloader = Zend_Loader_Autoloader::getInstance();
    $autoloader->registerNamespace('Foo_');
    $autoloader->registerNamespace(array('Foo_', 'Bar_'));
    
  • Alternately, if you do not know what namespaces you’re using, or you’re
    using a component library such as PEAR which does not have a common
    namespace prefix, you can set the autoloader to act as a fallback
    autoloader:

    $autoloader = Zend_Loader_Autoloader::getInstance();
    $autoloader->setFallbackAutoloader(true);
    

If you’re adventurous, you can also start using Zend_Application.
Zend_Application accepts a configuration option,
autoloaderNamespaces that can be an array of namespaces to utilize.

Resource Loading

I also mentioned resource autoloading earlier.
Zend_Loader_Autoloader_Resource defines an API whereby you specify
a common class namespace prefix and a base path for files with that prefix.
You then can register “resources”, which map a component prefix to a
subdirectory under the base path. This is particularly useful when grouping
application code, where you may not want a rigid library hierarchy, and
instead want to focus on grouping by responsibility.

How do these relate to Zend_Loader_Autoloader? Quite simply: the
constructor of Zend_Loader_Autoloader_Resource registers the
instance with Zend_Loader_Autoloader, using the provided namespace
prefix. This allows the main autoloader instance to intercept classes with
that namespace prefix and pass them to the resource loader to resolve.

How do you use these resource autoloaders in practice, though? The easiest
way to understand how they work is to look at the internals of
Zend_Application_Module_Autoloader, which extends the resource
autoloader and sets up some recommended mappings:

class Zend_Application_Module_Autoloader extends Zend_Loader_Autoloader_Resource
{
    public function __construct($options)
    {
        parent::__construct($options);
        $this->initDefaultResourceTypes();
    }

    public function initDefaultResourceTypes()
    {
        $basePath = $this->getBasePath();
        $this->addResourceTypes(array(
            'dbtable' => array(
                'namespace' => 'Model_DbTable',
                'path'      => 'models/DbTable',
            ),
            'form'    => array(
                'namespace' => 'Form',
                'path'      => 'forms',
            ),
            'model'   => array(
                'namespace' => 'Model',
                'path'      => 'models',
            ),
            'plugin'  => array(
                'namespace' => 'Plugin',
                'path'      => 'plugins',
            ),
            'service' => array(
                'namespace' => 'Service',
                'path'      => 'services',
            ),
            'viewhelper' => array(
                'namespace' => 'View_Helper',
                'path'      => 'views/helpers',
            ),
            'viewfilter' => array(
                'namespace' => 'View_Filter',
                'path'      => 'views/filters',
            ),
        ));
        $this->setDefaultResourceType('model');
    }
}

A resource is just a name that corresponds to a component namespace (which
is appended to the autoloader’s namespace) and a path (which is relative to
the autoloader’s base path). In action, you’d do something like this:

$loader = new Zend_Application_Module_Autoloader(array(
    'namespace' => 'Blog',
    'basePath'  => APPLICATION_PATH . '/modules/blog',
));

You could then simply instantiate classes in that tree:

// Maps to APPLICATION_PATH . '/modules/blog/models/Comments.php':
$comments = new Blog_Model_Comments();

Note: if you use the Modules bootstrap plugin resource,
and define bootstraps in your modules that extend
Zend_Application_Module_Bootstrap, a module resource autoloader
will be setup for you!

This obviates the need for complex require_once statements, or
complex include_path settings, which also helps you de-couple your
code to a large degree from the filesystem and project structure in which it
resides.

Why autoload at all?

So, why use autoloading at all?

In the past, there were a number of issues with autoloading, and it was
reportedly slower to use it than to simply use require_once and its
ilk. However, during the PHP 5.2 series of releases, this situation changed
dramatically with the addition of a solid realpath stat cache. Additionally,
in large frameworks such as Zend Framework, there are often a number of
classes that will require the same dependencies — and since
require_once still needs to do a stat call on each invocation, this
can lead to significant slowdown, particularly in systems where I/O is
expensive.

In short, autoloading, because it defers loading to the last possible moment
and ensures that a match only has to occur once, can be a huge performance
boost — particularly if you take the time to strip out require_once calls
before you move to deployment.

Another reason I hinted at in the last section: using autoloading, you do
not need to worry about where a class exists in your project. In
many ZF-based projects I’ve worked on or reviewed, there were many complex,
difficult systems used to try and map classes to files, such as altering the
include_path (and remember, the more paths on the
include_path, the more performance can degrade), using
Zend_Controller_Front to find module directories in order to do
relative path lookups, or permutations on the dirname(__FILE__)
theme. Using a resource autoloader, you can choose how you want to organize
a subset of your code, write the rules, and register it — and those classes
are now available anywhere in your application.

Conclusion

Autoloading seems like, and often is, an easy, straight-forward task.
However, creating a comprehensive autoloading solution that suits a variety
of needs can be a challenging task. You need to worry about everything from
playing nicely with other autoloaders to ensuring you get appropriate error
notifications; from allowing configurable autoloaders in your stack to
understanding the limitations and constraints of the SPL autoloader
solution; from mapping PEAR and ZF style classes to mapping application
classes.

Hopefully you know have a better understanding of how autoloaders work, and
some design decisions to consider when building an autoloading solution for
PHP. That said, you may consider using the solution proposed in this
writeup:
Zend_Loader_Autoloader.

Translations of this article:

About Matthew Weier O'Phinney

Matthew is an open source software architect, specializing in PHP. He is currently project lead for Zend Framework, a project with which he has been involved since before the first public preview release. He is a Zend Certified Engineer, and a member of the Zend Education Advisory Board, the group responsible for authoring the Zend Certification Exam. He contributes to a number of open source projects, blogs on PHP-related topics, and presents talks and tutorials related to PHP development and the projects to which he contributes. You can read more of his thoughts on his blog, weierophinney.net/matthew/.

View all posts by Matthew Weier O'Phinney

15 Responses to “Developing a Comprehensive Autoloader”

  1. gavihs Says:

    I am having a Zend Application and I am introducing a concept of customization based on the user who has logged in.
    I am having a Directory Structure as follows
    Application
    *
    \-Modules
    *
    \–Admin

    Library
    *
    \-CUSTOM
    *
    \–Admin

    I have all the custom classes under the CUSTOM\Admin directory
    Now I want the autoloader to check for the class file under the Directory Structure CUSTOM to load a class. If a class is not there then go to the Vanilla Module directory under Application/Modules and load the class from this directory.

    Can anyone help me on how this can be achieved?

  2. zensys Says:

    switch4mac, thanks for your reaction. Very useful to know how to autoload from the library. Currently I use the models version. In addition to what you mention I could only make it work with the following lines in my Bootstrap:

    protected function _initAutoload()
    {
    $loader = new Zend_Application_Module_Autoloader(array(
    ‘namespace’ => ‘Service’,
    ‘basePath’ => APPLICATION_PATH . ‘/modules/service’,
    ));
    }

  3. switch4mac Says:

    I was searching for how to autoload my personnal custom namespace from the library folder then I saw your post with:

    autoloadernamespaces.service = "Service_"

    The correct syntax would be the following for a Service namespace in /library/Service:

    autoloadernamespaces[] = "Service_"

    On the other hand, if you’re trying to instanciate the following model: Service_Models_FooBar, then the model needs to be like so:

    class Service_Models_FooBar{}

    and saved under "application/modules/Service/Models/FooBar.php"

    and don’t forget this line in application.ini if you’re using modules:
    resources.frontController.moduleDirectory = APPLICATION_PATH "/modules"

    Hope it helps.

  4. zensys Says:

    Using zend 1.10.7

    I am trying to enable autoloading for some classes I have in:

    application/modules/Service/models/

    I have tried two ways:

    1. in index.php (after instantiating Zend_Application)
    // Set autoloading for modules
    $autoloader = Zend_Loader_Autoloader::getInstance();
    $autoloader->registerNamespace(‘Service_’);

    in application.ini
    autoloadernamespaces.service = "Service_"

    Neither works:

    Warning: include_once(Service/Model/DocMerge.php): failed to open stream: No such file or directory in /usr/share/php/Zend/Loader.php on line 146 Warning: include_once(): Failed opening ‘Service/Model/DocMerge.php’ for inclusion

    What am I doing wrong? Any help or suggestions are greatly appreciated.

  5. kazemipouya Says:

    You note :
    "if you use the Modules bootstrap plugin resource, and define bootstraps in your modules that extend Zend_Application_Module_Bootstrap, a module resource autoloader will be setup for you!"
    but it does not work for default module, as I checked there is a "if" clause in
    Zend_Application_Resource_Modules that exclude the default module from this rule,
    Can you explain why? and what I have to do for loading resouces in default module?

  6. kazemipouya Says:

    You note :
    "if you use the Modules bootstrap plugin resource, and define bootstraps in your modules that extend Zend_Application_Module_Bootstrap, a module resource autoloader will be setup for you!"
    but it does not work for default module, as I checked there is a "if" clause in
    Zend_Application_Resource_Modules that exclude the default module from this rule,
    Can you explain why? and what I have to do for resouces in default module?

  7. iarchitect Says:

    Does this mean that it is possible that a well configured installation of ZF will now mean that you don’t see Debug Warnings in Zend Studio like the following?

    -Ed

    Debug Warning: /bycios2/library/Zend/Loader.php line 165 – fopen(./views/helpers/HeadTitle.php) [<a href='function.fopen'>function.fopen</a>]: failed to open stream: No such file or directory

  8. trippzend Says:

    Hi,

    thanks for this tutorial!

    I tried to follow it and included this in my Bootstrap.php (ZF 1.8):

    protected function _initAutoload()
    {
    $autoloader = new Zend_Application_Module_Autoloader(array(
    ‘namespace’ => ‘Blog’,
    ‘basePath’ => APPLICATION_PATH . ‘/modules/blog’,

    ));
    return $autoloader;
    }

    In a Controller I typed this:

    $comments = new Blog_Model_Comments();

    However I get:

    Fatal error: Class ‘Blog_Model_Comments’ not found in /home/XXXX/application/controllers/TestController.php on line 16

    What did I do wrong?
    Thx!

  9. weierophinney Says:

    Thanks for the feedback, everyone!

    @Pieter: That’s an excellent point about action helpers; could you
    add a feature request to the ZF issue tracker for this, please?

    @jshpro2: mea culpa; while I compiled the feature list for the
    release notes, I neglected to add this fairly major detail. We already have
    plans to create a better release checklist for future minor releases to
    ensure this sort of thing doesn’t happen again.

    ivan-ov: Merging all PHP files into one has been shown in a variety
    of benchmarks to have significant benefits. However, for requests where
    you’re actually using only a small subset of that functionality, it means
    you have much more overhead than really necessary. The best solutions I’ve
    seen use comprehensive autoloader support with an opcode cache — which
    solves both problems (load only what you need; loading a file already cached
    has little to no overhead).

  10. _____anonymous_____ Says:

    Thanks for the article. Module autoloading makes it a lot easier to implement the module structure.
    What I have been wondering about is why action helpers are not registered in the autoloading process?
    At the moment I added the following to my bootstrap to allow loading action helpers from all the modules:

    Zend_Controller_Action_HelperBroker::addPath(APPLICATION_PATH . ‘/helpers’, ‘Helper’);
    $modules = $this->frontController->getControllerDirectory();
    foreach($modules as $name => $path) {
    Zend_Controller_Action_HelperBroker::addPath($path, ucfirst($name) . ‘_Helper’);
    }

    Adding the Helper namespace as a resource type actually doesn’t work.

  11. jshpro2 Says:

    I just spent so much time trying to figure out why errors were broken, I’m sitting there with my debugger running dumping the value of ini_get( ‘display_errors’ ), greping my code for error_handlers, etc… It took me several hours to figure this out, went to go search the issue tracker and saw this on the homepage of the ZF site. I wish I saw it sooner it would have saved me a few good headaches. It doesn’t help to be fighting a bunch of failing unit tests while you discover the new fun error suppression ( if you havent switched your code over to the new loader yet ).

    Anyways the new component is worth my troubles, I was also pleasantly surprised to see the new Zend_Navigation component, looks promising

  12. ivan-ov@yandex.ru Says:

    Many of tests show us that merge all php files in one is great performance gain. Could you comment this?

  13. weierophinney Says:

    Yes — the plan is to remove the require_once calls eventually. Removing them right now would be a BC break, so we will need to likely offer packages with and without for some time before fully moving over.

  14. st4lk3r Says:

    @crazyj: There is no final decision about doing this at this point.

    Removing all the require_once’s in the code is a _major_ BC break, and even though I’d start with it here and now if I cound, we _must_ wait until 2.0 to implement this if we do.

  15. crazyj Says:

    Does the new autoloader mean that eventually Zend Framework will remove all of the require_once calls throughout the libraries?