Programmatic page-traversal in a Grav-plugin

And turning the pages into a HTML-list

For version 2 of my DirectoryListing-plugin I wanted to move away from the folder-traversal I had previously utilized. This post details how to programmatically iterate through Grav’s pages recursively, and gather information about page-media along the way. By the end, we’ll have produced a structure like this:

Features
├── default.md
├── Download
│   └── default.md
├── Documentation
├── Typography
│   └── default.md
├── Block Variations
└── Particles

In version 1 I used PHP’s built-in scandir()-function to recurse through Grav’s /user/pages/-folder and build a HTML-list from it. This worked well enough because all of Grav’s content is essentially kept in and below this folder, but the result was limited to folder- and file-names.

To improve upon this, iterating through Grav’s actual Pages-object will return much more useful data, in particular titles, routes, and paths. However, rather than access this object directly I am accessing it by creating a collection from the current page. This allows using page-header variables and sorting like normal pages.

Creating Utilities

Rather than construct a large and convoluted directorylisting.php, I split the core functionality into Utilities.php, which is also used by the plugin’s Twig-extension (more on that later). This file declares a class – “Utilities” – with four methods:

  • build(), which initiates the process and resolves configuration
  • buildTree() recursively iterates through pages and media
  • buildList() recursively iterates through the previously created tree to form a HTML-list
  • arrayExcept(), a helper-method to remove a key from an array

The class is instantiated from directorylisting.php, and returns the directorylisting Twig-variable. This is done in the “onTwigPageVariables”-event. In addition to the default plugin-configuration, any page can override it. This is facilitated through:

/**
 * Builds hierarchical HTML-list of pages and media
 * @return string HTML-List
 */
public function output()
{
    $config = (array) $this->config->get('plugins.directorylisting');
    $page = $this->grav['page'];
    $route = $page->route();
    $config = $this->mergeConfig($page);
    $utility = new Utilities($config);
    $list = $utility->build($route);
    $this->grav['twig']->twig_vars['directorylisting'] = '<div class="directorylist">' . $list . '</div>';
}

First, it retrieves the plugin’s default-configuration, the Page-object, and the page’s route. Then it merges any page-configuration, and constructs the “Utilities”-class. Finally, it builds the HTML-list and outputs it through Twig.

Building the tree

Internally the merged configuration is passed to the class, wherein build() takes over.

/**
 * Sorts out configuration and initiates building
 * @param string $route Route to page
 * @return string HTML-list from page-structure
 */
public function build($route)
{
    $config = $this->config;
    $tree = $this->buildTree($route);
    if ($config['include_additional']) {
        foreach ($config['include_additional'] as $include) {
            if (is_array($include)) {
                $include = $this->buildTree($include, '@page.self');
                $tree = array_merge($tree, $include);
            }
        }
    }
    if ($config['exclude_additional']) {
        foreach ($config['exclude_additional'] as $exclude) {
            if (is_array($exclude)) {
                $this->arrayExcept($tree, array($exclude));
            }
        }
    }
    if ($tree) {
        $list = $this->buildList($tree);
        return $list;
    } else {
        return false;
    }
}

Here a tree-structure of pages is constructed, and additional pages are included or excluded based on the retrieved configuration. The tree is constructed thuswise:

/**
 * Creates page-structure recursively
 * @param string $route Route to page
 * @param integer $depth Reserved placeholder for recursion depth
 * @return array Page-structure with children and media
 */
public function buildTree($route, $mode = false, $depth = 0)
{
    $config = $this->config;
    $page = Grav::instance()['page'];
    $depth++;
    $mode = '@page.self';
    if ($depth > 1) {
        $mode = '@page.children';
    }
    $pages = $page->evaluate([$mode => $route]);
    $pages = $pages->published()->order($config['order']['by'], $config['order']['dir']);
    $paths = array();
    foreach ($pages as $page) {
        if ($config['exclude_modular'] && isset($page->header()->content['items'])) {
            if ($page->header()->content['items'] == '@self.modular') {
                continue;
            }
        }
        $route = $page->rawRoute();
        $path = $page->path();
        $title = $page->title();
        $paths[$route]['depth'] = $depth;
        $paths[$route]['title'] = $title;
        $paths[$route]['route'] = $route;
        $paths[$route]['name'] = $page->name();
        if (!empty($paths[$route])) {
            $children = $this->buildTree($route, $mode, $depth);
            if (!empty($children)) {
                $paths[$route]['children'] = $children;
            }
        }
        $media = new Media($path);
        foreach ($media->all() as $filename => $file) {
            $paths[$route]['media'][$filename] = $file->items()['type'];
        }
    }
    if (!empty($paths)) {
        return $paths;
    } else {
        return null;
    }
}

In buildTree() the Pages-object is constructed. Line 11-15 does a couple of important things: The $depth-parameter is increased, to indicate what level of depth the recursive array is at, but also to set the header used by the collection. It should only target itself at the first level, and thereafter the current page’s children.

The next two lines creates the Pages-object as a collection, and sets the order of pages. The following foreach-loop first excludes modular pages, if set in the configuration, then proceeds to add parts of each page to $paths – named thus because each page is identified by its route, which is unique within Grav.

Notice, on line 32-37, that if the page was created it then searches for children by calling itself again through $children = $this->buildTree($route, $mode, $depth). This is what makes the method – or any function in any programming-language really – recursive. It calls itself, and in this case passes the current page’s route, the “@page.children”-mode, and current depth. This process continues until no further pages below the current page is found.

Additionally, on line 38-41, the page’s media is iterated over. Every media-file, including photos, audio, videos, etc., is included and identified by its filename. Whilst this is not inherently a unique value, no filesystem will allow files with the same name in the same folder, so it is virtually unique.

Building the list

Now, as we saw previously, buildList() is called with the tree as the parameter. This method does much of the same:

/**
 * Creates HTML-lists recursively
 * @param array $tree Page-structure with children and media
 * @param integer $depth Reserved placeholder for recursion depth
 * @return string HTML-list
 */
public function buildList($tree, $depth = 0)
{
    $config = $this->config;
    $depth++;
    if ($config['builtin_css'] && $config['builtin_js'] && $depth == 1) {
        $list = '<ul class="metismenu metisFolder">';
    } else {
        $list = '<ul>';
    }
    foreach ($tree as $route => $page) {
        $list .= '<li class="directory';
        if ($page['depth'] <= $config['level']) {
            $list .= ' active';
        }
        $list .= '">';
        if ($config['builtin_css'] && $config['builtin_js']) {
                $list .= '<a href="#" aria-expanded="true" class="has-arrow">' . $page['title'] . '</a>';
        } else {
            if ($config['links']) {
                $list .= '<a href="' . $page['route'] . '">' . $page['title'] . '</a>';
            } else {
                $list .= $page['title'];
            }
        }
        if (!$config['exclude_main']) {
            if ($config['links']) {
                $list .= '<ul><li class="file page"><a href="' . $page['route'] . '">' . $page['name'] . '</a></li></ul>';
            } else {
                $list .= '<ul><li class="file page">' . $page['name'] . '</li></ul>';
            }
        }
        if (isset($page['children'])) {
            $list .= $this->buildList($page['children'], $depth);
        }
        if (isset($page['media'])) {
            $list .= '<ul>';
            foreach ($page['media'] as $filename => $type) {
                if ($config['links']) {
                    $list .= '<li class="file ' . $type . '"><a href="' . $page['route'] . '/' . $filename . '">' . $filename . '</a></li>';
                } else {
                    $list .= '<li class="file ' . $type . '">' . $filename . '</li>';
                }
            }
            $list .= '</ul>';
        }
        $list .= '</li>';
    }
    $list .= '</ul>';
    return $list;
}

The first lines take care of configuration and depth, as well as assigning classes if built-in CSS and JS is used.

First it checks the configuration and figures out the current depth in the Pages-tree. It also assigns classes if the built-in CSS and JS is used. This check is done throughout to add necessary functionality for metisMenu to work. When the tree is iterated over, each page gets a list-item – with optional link to the page if enabled – with its media listed within it. As before, notice how line 38-40 refers to the buildList-method and passes the page’s children and the depth, thus recursing until no more pages are found.

A Twig-extension

To use directorylisting from any Twig-template, a DirectoryListingTwigExtension.php is added. Simply put, this goes through exactly the same process in building the tree and the list. The difference is that rather than passing configuration through the page’s FrontMatter or the plugin’s configuration-file, it is passed directly to the Twig-function. This follows the same format and looks like this:

{% set settings = {
  'route': '/features',
  'exclude_main': false,
  'exclude_modular': true
} %}
{{ directorylisting(settings) }}

The output

When all this executes the following list is returned:

<div class="directorylist">
    <ul class="metismenu metisFolder">
        <li class="directory active"><a href="#" aria-expanded="true" class="has-arrow">Features</a>
            <ul aria-expanded="true" class="collapse in">
                <li class="file page"><a href="/features">default.md</a>
                </li>
            </ul>
            <ul aria-expanded="true" class="collapse in">
                <li class="directory"><a href="#" aria-expanded="true" class="has-arrow">Download</a>
                    <ul aria-expanded="false" class="collapse">
                        <li class="file page"><a href="/features/download">default.md</a>
                        </li>
                    </ul>
                </li>
                <li class="directory"><a href="#" aria-expanded="true" class="has-arrow">Documentation</a>
                    <ul aria-expanded="false" class="collapse">
                        <li class="file page"><a href="/features/documentation">default.md</a>
                        </li>
                    </ul>
                </li>
                <li class="directory"><a href="#" aria-expanded="true" class="has-arrow">Typography</a>
                    <ul aria-expanded="false" class="collapse">
                        <li class="file page"><a href="/features/typography">default.md</a>
                        </li>
                    </ul>
                </li>
                <li class="directory"><a href="#" aria-expanded="true" class="has-arrow">Block Variations</a>
                    <ul aria-expanded="false" class="collapse">
                        <li class="file page"><a href="/features/block-variations">default.md</a>
                        </li>
                    </ul>
                </li>
                <li class="directory"><a href="#" aria-expanded="true" class="has-arrow">Particles</a>
                    <ul aria-expanded="false" class="collapse">
                        <li class="file page"><a href="/features/particles">default.md</a>
                        </li>
                    </ul>
                </li>
            </ul>
        </li>
    </ul>
</div>

Which will look like this:

Directory Listing

And that’s “all” there is to it. The same tree could be used anywhere you’d need an exhaustive list of Grav’s pages and their associated media. As a final tip, you can use Grav::instance()['debugger']->addMessage($tree) to explore the tree in the debugger if needed, for example after line 25 in the build()-method.

A more concise tutorial covering only the traversal of pages is also available in Grav’s Docs.

php grav plugin

Optimizing a Grav-Installation

When the term “Web 2.0” became popular after 2004, the focus was largely on design, usability, and new media. In the past few years, however, web development has focused a lot on more on speed and per...