Friday, February 3, 2017

Adding search to your Moodle plugin - post mortem

Post Mortem - Production ready search

I ended up in a lot of discussions with other Moodle developers on the best way to implement the question content search. Originally, I created a separate search area, but was hampered by the fact that I couldn't reliably keep a record per question linked to a questionnaire instance.

The search API didn't allow me to create my own index other than an integer. If it would have allowed me to construct my own, I could have stored information that identified the questionnaire and question. This resulted in the creation of the Moodle tracker item MDL-57857.

However, as we discussed this particular problem more, it really seemed that if I was searching for questionnaire instances that contained the searched for question content, then the index should really be part of the activity search. This resulted in the update I posted at the end of the last post. While that worked, the return results displayed weren't very attractive and, as was pointed out, would also include question content that may never have been displayed to the use. Questionnaires can choose questions to display based on answers to other questions.

At that point, I realized that the most likely use case for searching for question content would be for users who can access all of the question content of a questionnaire. This would be users who could actually edit the questions and users who could see all of the responses to any question.

So, it made more sense to again separate out the question search area into its own search area. This allows for two important things:

  1. Allows the administrator of the site to determine if they want to even allow question content searching.
  2. Allows the access for this content to be limited to users who have the ability to see all of the question content for a questionnaire.
You can see the final code I decided on here. The question area still stores one record per questionnaire activity, like the activity area, but it uses a different set of access rules. And, since it is a separate area, it can be enabled/disabled globally, and filtered on separately.

I believe that until there is a way to store information in the search index that allows me to go directly to the question within a specific questionnaire, this is the best solution.

Wednesday, February 1, 2017

Adding search to your plugin - Part three

Part Three - Indexing More of your Plugin

In the last post, I added more searchable content to my plugin's search indexing function, so that additional information fields specific to each instance of a plugin were included. Now I'm going to try and add question content to the search indexing.

The documentation tells me that if I'm going to index other activity data, I should use the base_mod class instead of the base_activity class. The base_mod class is located in the core file /search/classes/base_mod.php. Looking at that class, I see that it only contains one method: get_cm. All the rest come from the class it extends, base. The functions I used before, get_recordset_by_timestamp() and get_document() are abstract functions of base. This means I must provide their implementations. I also note that three other abstract functions, check_access(), get_doc_url() and get_context_url() that I will likewise need to provide implementations for. In my previous work, these three functions were provided to me by the base_activity class.

Back to the documentation, the first step it points out is that if I am going to add a new search area for my plugin, I need to provide a name for it in my language strings. Since I am going to index the questions, I add the string:

    $string['search:question'] = 'Questionnaire - questions';

to my lang/en/questionnaire.php file.

Next, I need to provide the indexed content from my questions to the search engine, using the get_recordset_by_timestamp() function. Recall that this function is responsible to construct the necessary database query for all of the content I want available for searching, execute that query, and then return the resulting recordset. For this, I create a new file called 'classes/search/question.php' and create a new question class extending the \core_search\base_mod class.

For questionnaire questions, I will need to join two tables. Question data is contained in a table that contains a reference to a survey id. That survey id is also referenced in the main questionnaire table. Since question data is not accessed outside of a survey context, I will need to create SQL that returns each question with an associated questionnaire and survey.

The function I create looks like this (full file here):

    public function get_recordset_by_timestamp($modifiedfrom = 0) {
        global $DB;

        $sql = 'SELECT qq.id, q.id AS questionnaireid, q.timemodified, ' .
            'q.course AS courseid, q.introformat, qq.name, qq.content ' .
            'FROM {questionnaire} q ' .
            'INNER JOIN {questionnaire_question} qq ON q.sid = qq.survey_id AND ' .
            'qq.deleted = \'n\' ' .
            'WHERE q.timemodified >= ? ' .
            'ORDER BY q.timemodified ASC';

        return $DB->get_recordset_sql($sql, [$modifiedfrom]);
    }

Note that I use the question id field, 'qq.id', as the first field. Moodle requires the first field of a query return to be a unique field that is used as an index for the resulting data array. Since there will be multiple questions per questionnaire and survey record, I can't use those. I will need the questionnaire id field though, so I rename it to 'questionnaireid' in the returned data. The question name and content fields, 'qq.name' and 'qq.content', contain the data that will actually be searched. This function will provide the data that will be indexed by the search function.

Now I need to provide the get_document() function. This is the function that looks at an indexed record returned from a search query and constructs the document object that is displayed on the search results screen. A document object contains a 'title' and 'content' field. I set the question 'name' data to the document 'title' and the question 'content' data to the document 'content'. The rest of the object is pretty much boilerplate from the base_activity class code and the documentation.

The function I create looks like this (full file here):

    public function get_document($record, $options = []) {
        try {
            $cm = $this->get_cm('questionnaire', $record->questionnaireid,
                $record->courseid);
            $context = \context_module::instance($cm->id);
        } catch (\dml_missing_record_exception $ex) {
            // Notify it as we run here as admin, we should see everything.
            debugging('Error retrieving ' . $this->areaid . ' ' . $record->id .
                ' document, not all required data is available: ' .
                $ex->getMessage(), DEBUG_DEVELOPER);
            return false;
        } catch (\dml_exception $ex) {
            // Notify it as we run here as admin, we should see everything.
            debugging('Error retrieving ' . $this->areaid . ' ' . $record->id .
                ' document: ' . $ex->getMessage(), DEBUG_DEVELOPER);
            return false;
        }

        // Prepare associative array with data from DB.
        $doc = \core_search\document_factory::instance($record->id, $this->componentname,
            $this->areaname);
        $doc->set('title', content_to_text($record->name, false));
        $doc->set('content', content_to_text($record->content, $record->introformat));
        $doc->set('contextid', $context->id);
        $doc->set('courseid', $record->courseid);
        $doc->set('owneruserid', \core_search\manager::NO_OWNER_ID);
        $doc->set('modified', $record->timemodified);

        return $doc;
    }

Prior to constructing the returned document object, the search API needs to verify that the requesting user has the right permissions to access the Moodle item. This is done with check_access() function. Generally speaking, what this function should do is to extract the Moodle item in the correct Moodle context, and then check that item and context with the requesting user's capabilities. In my case, a question's data is tied to the questionnaire it belongs to. So determining if a user can see that is based on their access to the specific questionnaire instance.

For questionnaire, this is complicated by the fact that a survey (which contains the questions) can belong to more than one questionnaire (in the case of public questionnaires). While this is an uncommon use, it can still exist. The check_access() function only provides me with the unique id of the data that was saved. For questions, this is the question id field. This means that I do not know which questionnaire instance I need to check when I receive this id data. So my function will need to "guess" at which questionnaire to return by looking for one that contains the question data that is accessible to the user.

The function I create looks like this (full file here):

    public function check_access($id) {
        global $DB;

        try {
            // Questions are in surveys and surveys can be used in multiple questionnaires.
            $question = $DB->get_record('questionnaire_question', ['id' => $id],
                '*', MUST_EXIST);
            $questionnaires = $DB->get_records('questionnaire',
                ['sid' => $question->survey_id]);
            if (empty($questionnaires)) {
                return \core_search\manager::ACCESS_DELETED;
            }
            $cmsinfo = [];
            foreach ($questionnaires as $questionnaire) {
                $cmsinfo[] = $this->get_cm('questionnaire', $questionnaire->id,
                    $questionnaire->course);
            }
        } catch (\dml_missing_record_exception $ex) {
            return \core_search\manager::ACCESS_DELETED;
        } catch (\dml_exception $ex) {
            return \core_search\manager::ACCESS_DENIED;
        }

        // Recheck uservisible although it should have already been checked in core_search.
        // Use the first one that is visible. Otherwise exit.
        $cmvisible = false;
        foreach ($cmsinfo as $cminfo) {
            if ($cminfo->uservisible !== false) {
                $cmvisible = $cminfo;
                break;
            }
        }
        if ($cmvisible === false) {
            return \core_search\manager::ACCESS_DENIED;
        }

        $context = \context_module::instance($cminfo->id);

        if (!has_capability('mod/questionnaire:view', $context)) {
            return \core_search\manager::ACCESS_DENIED;
        }

        return \core_search\manager::ACCESS_GRANTED;
    }

Without going into too much detail, the basic function is:
  • find the question data and all questionnaires that might contain it,
  • get a valid course module for each questionnaire,
  • find the first course module that is visible to the user and assume this is the one,
  • check that the user has the proper capabilities to view that questionnaire.
For this post, the functionality will suffice to show how this function works. Unless the function finds reason not to grant access, then granted access is returned.

(Note - I may be able to modify my functions to return a search id field that I construct to identify both the question and the questionnaire. I will look into this later, as I am not happy with "guessing" which questionnaire is used.)

The last two functions I need to provide are get_doc_url() and get_context_url(). These functions provide the link to the resulting page that will display the found content in its proper Moodle context. Since there is no way to look at only a question instance, my functions will provide a link to the questionnaire. Both functions will provide the same link. I write the code for the get_context_url() and just call that from the get_doc_url() function.

The get_context_url code looks like this (full file here):

    public function get_context_url(\core_search\document $doc) {
        $context = \context::instance_by_id($doc->get('contextid'));
        return new \moodle_url('/mod/questionnaire/view.php',
            ['id' => $context->instanceid]);
    }

Pretty straightforward.

Since I have everything I need, I load the new code, reindex all of the site contents and give it a test.

One of the first things I notice is that there is a new "search area" in the dropdown called "Questionnaire - questions":


Next, I search for a term that I know is only in some questions. Sure enough, I see the question name and the question content where matches occur. And clicking the link takes me to the appropriate questionnaire.

Updating your plugin to use the new global search feature is a fairly easy process, especially if all you do is provide the basic search. There should be no reason not to do it!

This concludes my series on adding global search to your Moodle plugin. Feel free to leave comments and suggestions. I will look at improving the question results as I noted above, and will provide updates when I have completed that.

Checklist for building additional plugin content searching:
  1. Create a new class with the name of your area in a similarly named file in your 'classes/search' directory.
  2. Add a language string name using "search:AREANAME" for the search area in your plugin language file.
  3. Create a get_recordset_by_timestamp() function to retrieve the records you want to index.
  4. Create a get_document() function to return a completed document object to the search function.
  5. Create a check_access() function to determine a matched document's availability to the user performing the search.
  6. Create a get_context_url() function to return an appropriate Moodle URL to the matched document.
  7. Create a get_doc_url() function to return an appropriate Moodle URL to the matched document.

Update - 2017.02.02
After a lot of discussion with other developers, I redid the activity search to include the question content. Since the question content was helping to find the questionnaire instance that contained it, it really made sense for it to be a part of the activity search and instance return. Since there could be many question records per questionnaire instance, I had to do the database search in the get_document() function and add the question data there. You can see the resulting work here.


Thursday, January 26, 2017

Adding search to your Moodle plugin - part two

Part Two - Indexing Custom Information

In the last post, I created the basic code required to allow Moodle's global search feature search my plugin's "name" and "intro" information fields. Now I want to extend that functionality to include the "subtitle" and "additional info" fields.

Reading the documentation, this looks like it should be an extension of the functionality I wrote for the basic case. The information fields I want to return, both belong to an activity instance. And the case I already created is for that activity instance. So I want to look at extending the class I already created.

If I decide to provide index data from other sources, then I would be creating new classes as described in the documentation following sections.

One of the concerns I have with my approach of continuing extending the base_activity class, rather than the base_mod class, is that the documentation says that I should use base_activity "to index Moodle activities basic data like the activity name and description". And that base_mod should be used for "other specific activity data". The two extra fields I want to include, aren't part of the main activity table of my plugin, but rather are stored in a second table called "questionnaire_survey". There is a one-to-one relationship though, and the information is considered part of the activity information, so I still think this the correct approach. But, to get that extra data, I will have to override the database function that gets all of that data.

Looking at the /search/classes/base_activity.php file, I see that the main function to retrieve and return the activity data is the get_recordset_by_timestamp function:

    public function get_recordset_by_timestamp($modifiedfrom = 0) {
        global $DB;
        return $DB->get_recordset_select($this->get_module_name(),
                   static::MODIFIED_FIELD_NAME . ' >= ?',
                   array($modifiedfrom),
                   static::MODIFIED_FIELD_NAME . ' ASC');
    }

This is the one I will override in my class. In its current form, it returns all of the fields from the main activity table according to the "timemodified" field. Since I want to JOIN the "questionnaire_survey" with this table, I am going to have to be selective about the fields that are returned, as there is overlap.

I open my questionnaire/classes/search/activity.php file and add the following function (full file here):

    public function get_recordset_by_timestamp($modifiedfrom = 0) {
        global $DB;

        $sql = 'SELECT q.*, s.subtitle, s.info ' .
            'FROM {questionnaire} q ' .
            'INNER JOIN {questionnaire_survey} s ON q.sid = s.id ' .
            'WHERE q.timemodified >= ? ' .
            'ORDER BY q.timemodified ASC';

        return $DB->get_recordset_sql($sql, [$modifiedfrom]);
    }

Since I haven't told the search engine anywhere that I want to include the two new fields I've added, there must still be some changes to make. I install the new code, reindex the global search and test that things work. While there are no failures, I also cannot get any matches from the "subtitle" or "additional info" fields, confirming that there is more to do.

Looking at the documentation again, the next function that is presented is get_document. From the description, this function takes a record from the query results of get_recordset_by_timestamp function, and constructs a \core_search\document object with all of the data needed for indexing. My guess, is that I need to add my two new fields here. I'm going to override this function in my class.

The main function returns a constructed object with the data added to it as variables. I believe I can override this and take the returned function from the parent, add my new data to it, and return it. When I'm done, the function looks like this:

    public function get_document($record, $options = []) {
        // Get the default implementation.
        $doc = parent::get_document($record, $options);

        // Add the subtitle and additional info fields.
        $doc->set('subtitle', content_to_text($record->subtitle, false));
        $doc->set('additionalinfo', content_to_text($record->info, $record->introformat));

        return $doc;
    }

I add the code to my site, purge all the caches and reindex the search engine. But, searching for these fields is not returning my data. On the search areas admin page, I have been using "Update indexed contents". But, since none of my questionnaire activities have actually changed, nothing is being re-indexed. Instead, I need to use the "Reindex all site contents" button. When I do this, I get an error indicating:

    Coding error detected, it must be fixed by a programmer: "subtitle" field does not exist.

It would appear that using "subtitle" is not correct.

The documentation shows an example of using get_document(), and it uses "description1" and "description2" as the fields being set. Its likely that I can't use "subtitle" and "additionalinfo" as  field names in the $doc->set() function.

The \core_search\document class is defined in the /search/classes/document.php file. The set() function uses a static array to look for specific field names it is allowed to set. The names I used, "subtitle" and "additionalinfo" are not part of those. However "description1" and "description2" are. This confirms that I need to change those functions. The set() function throws an exception when an unknown field is used, and that explains the error I saw.

My new function now looks like this (full file here):

    public function get_document($record, $options = []) {
        // Get the default implementation.
        $doc = parent::get_document($record, $options);

        // Add the subtitle and additional info fields.
        $doc->set('description1', content_to_text($record->subtitle, false));
        $doc->set('description2', content_to_text($record->info, $record->introformat));

        return $doc;
    }

When I load it, and reindex all of the site contents, my search works. I now see content found from a subtitle.


So it seems that I can add indexed fields to the a activity level search, but only two. And they need to be referred to as "description1" and "description2". I'm guessing to add more than that, I would need to create more classes using base_mod instead of base_activity.

I'll explore that more in the next post, as well as how to search for more parts of my plugin.

Monday, January 23, 2017

Adding search to your Moodle plugin

Part One - The Basics

In Moodle 3.1, the "Global Search" feature was introduced. This feature provides the ability for a user to search for text within the entire site for entered terms. The results obey all role and capability rules, so only parts that can be legitimately accessed by the requesting user are returned.

As delivered, core activities are included in the global search. But plugins need to provide code to the search API in order to be included in the search results. In this series of blog posts, I am going to add the search feature to my questionnaire plugin.

Some resources that will help with this task are:


The global search feature uses search engine plugins to provide the search functions. Currently, there is only one plugin, that uses an external technology called Solr. In order to use search, and enable me to test my work, I will install it in my development environment. Moodle provides documentation to do this. Since I am using OSX with Macports, I follow those instructions. After that, I use the administration functions in my Moodle site to enable global search and verify that it is working. I'm not going to go into the details of that here, but the documentation links were enough for me to successfully do this.

Before I start, I'm going to use the search function to see how it works with the existing activities, and verify that it is not working for mine. I'm using Moodle 3.2 with the Boost theme. The search function is located in the upper right part of the navigation bar, with a text box entry next to a magnifying glass icon as below:


Once I enter my search text, for example "test", the screen displays everything it found and some further options. Under the "filter" section, there is a "Search area" drop down that lists all of the areas available to search within (see image below). My plugin does not appear there, so I presume this will be one of the places that I will see a change once I have enabled search in my plugin.


I also notice that in those filter choices, some of the activities show only "activity information" while others show this and other levels of the activity. These correspond to what content an activity returns to the search function. Looking at the documentation, I see that returning the "activity information" is the easiest case, and does not require much work at all. I am going to start there.

I have created a working branch off of my latest Moodle 3.2 branch called GLOBALSEARCH_32 as my starting point. You can follow along on Github here.

The documentation makes this look pretty straight forward. I create a subdirectory called search in my plugin's classes subdirectory. In that new directory, I add a file named activity.php. The contents, are basically this (you can see the entire file here):

    namespace mod_questionnaire\search;

    class activity extends \core_search\base_activity {
    }

I'm going to update my development site with this new code, and see what happens. After copying, I check the filter dropdown as above, but I don't see the questionnaire in the list. In all likelihood, I need to purge all of the caches. After doing that, the page reloads, but gives me the following error:

Invalid get_string() identifier: 'search:activity' or component 'mod_questionnaire'. Perhaps you are missing $string['search:activity'] = ''; in mod/questionnaire/lang/en/questionnaire.php?
  • line 349 of /lib/classes/string_manager_standard.php: call to debugging()
  • line 7076 of /lib/moodlelib.php: call to core_string_manager_standard->get_string()
  • line 146 of /search/classes/base.php: call to get_string()
  • line 62 of /search/classes/output/form/search.php: call to core_search\base->get_visible_name()
  • line 204 of /lib/formslib.php: call to core_search\output\form\search->definition()
  • line 59 of /search/index.php: call to moodleform->__construct()
It looks like I need to define a specific language string as well. The error nicely tells me exactly what I need. I add the line:
    $string['search:activity'] = 'Questionnaire - activity information';
to the lang/en/questionnaire.php file and try again. This time, I have no error, and the drop down shows my activity in the list:


However, when I perform a search that should include a questionnaire, it does not show up. The most likely reason is that the search engine has not indexed the plugin into its database yet. The global search can be scheduled to reindex its database, and it can also be manually reindexed. At the main global search administration page, admin/settings.php?section=manageglobalsearch, there is an "Index data" link. Reviewing that page, I can see that there is a section for "Questionnaire - activity information", that is showing as never having been indexed. When I click the "Update indexed contents" button, also on that page, my questionnaire now shows as having been indexed. I perform my search again, and I can now see results from my questionnaire.

Adding the basic search functionality turned out to be extremely easy. I simple had to add one file that extended a class, and one language string. In the next post, I'll add extra levels of searching to include questionnaire "subtitle" and "additional info" sections.


Tuesday, January 17, 2017

Part 5 - Further discussion

After I completed my first attempt at adding renderers and templates to my questionnaire plugin, I went on to use them much more thoroughly in the actual release. The Moodle 3.2 release of questionnaire uses templates for all of the major pages. This is a work in progress, and I have taken some shortcuts to make the transition easier while allowing me to release the code as it is worked on.

Most noticeably, since the output of questionnaire is generated by specific functions that return HTML pieces, I have created the templates to accept that HTML. This means that only the top level formatting of a page is templatable. But, I am working on extracting more and more of the output into the templates as the work continues.

In the RENDERER_32 work in progress branch, I have refactored and extracted almost all of the question specific output into individual question type templates. These templates are used with renderer functions to return the HTML to the main renderer. I have coded the renderer function so that it will use question templates as I define them, or fallback to the old method if they are not present. If you have questions about this work, contact me or post a question here.

If you wander through those templates, you'll see some other aspects of templates used that I didn't discuss in this series. For example, in the navbaralpha.mustache file you can see the use of the language string {{# str}} helper function. This is a way to put a language string directly into the template. You can find more information on that here.

In the question_check.mustache template, you can see the use of dotted notation for tag names. For example, choice.id. While not strictly necessary, I find that this notation helps to document the structure of the context data being used in the template. In this example, the fact that id is a subelement of the choice structure.

Regarding data structures, if you recall in "Part three", I mentioned that the template data could be formatted as arrays or objects and that both would work. The code I provided used strictly arrays. But it could have used objects as well as the one array needed for the rows element. Below is an alternate coding using objects instead of arrays where possible:

    /**
     * Prepare data for use in a template
     *
     * @param \renderer_base $output
     * @return stdClass
     */
    public function export_for_template(\renderer_base $output) {
        $data = new \stdClass();
        $data->headings = new \stdClass();
        foreach ($this->headings as $key => $heading) {
            $data->headings->{$key} = $heading;
        }
        $data->rows = [];
        foreach ($this->rows as $row) {
            list($topic, $name, $responses, $type) = $row;
            $item = new \stdClass();
            $item->topic = $topic;
            $item->name = $name;
            $item->responses = $responses;
            $item->type = $type;
            $data->rows[] = $item;
        }
        return $data;
    }

Lastly, I never got into using javascript with templates. I have personally not spent enough time with this to do that topic justice. If you wish to explore that though, start with the documentation. Additionally Damyon Wiese from Moodle HQ gives a great demonstration that covers how to include javascript and Ajax with templates, as well as a great introduction to templates in general. If you want to skip ahead to where javascript comes into play, fast-forward to around 14:10 in the video.

If you want to discuss any of this series, or recommend alternate approaches, please feel free to leave comments or contact me directly.

Friday, January 13, 2017

Part 4 - Using the Template

In the previous posts, I created and used a renderer, and created a simple mustache template for my plugin. In part four of the "Adding renderers and templates to your Moodle plugin" series, I will present how to code the template classes and then use them in the renderer.

To use a template, I need a few different constructs.

Firstly, I need a new class that implements the core renderable and templatable classes. The file and class name should be the same as the template file name to make things easier. And the file must be located in the classes/output/ directory.

Since my template file is called indexpage.mustache, I create a new file called indexpage.php in the classes/output directory, and add a class called indexpage that implements \renderable and \templatable.  The class declaration must look like this (the entire file is available here):

    class indexpage implements \renderable, \templatable {
    }

The purpose of this script is to collect the output data for the template and make it available to the renderer. The only real required function is the export_for_template function, as renderers require that function to retrieve the data structure that is passed into the template. Other functions can be provided as necessary including those that process the data for the template. The code for the export function looks like this:

    public function export_for_template(\renderer_base $output) {
        $data = ['headings' => [], 'rows' => []];
        foreach ($this->headings as $key => $heading) {
            $data['headings'][$key] = $heading;
        }
        foreach ($this->rows as $row) {
            list($topic, $name, $responses, $type) = $row;
            $data['rows'][] =['topic' => $topic, 'name' => $name,
                'responses' => $responses, 'type' => $type];
        }
        return $data;
    }

For my version, I pass all the necessary data for the template to the constructor, and then process it for output in the export function. The class contains two property variables, $headings and $rows, which will hold the table headings and each row of content for the index page based on what is passed into the constructor. My export_for_template function processes these variables and creates the expected data construct for the template.

If you recall from the last post, my template expects a data structure with two top level elements: #headings and #rows. In the export_for_template function, I create a PHP array called $data, and add the keys 'headings' and 'rows' to it, each of which is initialized to an array. Then, I load up the headings array with a value for each 'title[n]' index and the rows array with an array for each content row containing the topic, name, responses and type elements. So after processing, the function returns a structure that looks something like this (for example):

    $data['headings' =>
             ['title1' => 'Topic',
              'title2' => 'Name',
              'title3' => 'Responses',
              'title4' => 'Questionnaire Type'
             ],
          'rows' =>
             [0 =>
                 ['topic' => 'Topic1',
                  'name' => 'Questionnaire1',
                  'responses' => 42,
                  'type' => 'Private'
                 ],
             [1 =>
                 ['topic' => 'Topic2',
                  'name' => 'Questionnaire2',
                  'responses' => 11,
                  'type' => 'Private'
                 ]
             ]
         ];

This structure is what gets sent to the template to meet its requirements for data.

Now that I have the necessary implementation of the core \templatable class, and a template, I need to code the renderer to use them. Currently, the renderer's render_index function, creates the HTML output from Moodle core helper functions, specifically the \html_table class. I need to change that so it uses the template and template class.

To do that, I change the declaration of the function to take one argument, a \templatable object. And I get rid of the current code, and replace it with the object's export_for_template function and a call to the renderer class function render_from_template. The new function looks like this (you can see the entire file here):

    public function render_indexpage(\templatable $indexpage) {
        $data = $indexpage->export_for_template($this);
        return $this->render_from_template('mod_questionnaire/indexpage', $data);
    }

The new render_index function expects an object created from the classes/output/indexpage.php::indexpage class. With that object, it will call the export_for_template function to get the data necessary to pass into the template. Then it will pass that data to the template to get the HTML output from the core render_from_template function. The render_from_template function takes two arguments: the template to use and the data to pass the template. In my case, the template to use is defined by the argument 'mod_questionnaire/indexpage'. The 'mod_questionnaire' portion tells the function that this is for an activity module plugin, and therefore the template should be found in the directory '/mod/questionnaire/templates/'. The 'indexpage' portion indicates that there should be file named 'indexpage.mustache' in that directory. After that, the function executes the necessary magic to pass the data into the template and return the final formatted HTML.

The final step I need to do in order to use all of this new template code is change the way I am using the renderer in the index.php file. In this case, I only need to change the code where I am actually using the renderer to output to the screen. I need to do two things: change the arguments being passed to the render_index function and create an indexpage templatable object for that function to use.

The old code looks like this:
    echo $output->render_index($headings, $align, $content);

The new code looks like this:
    $indexpage = new \mod_questionnaire\output\indexpage($headings, $content);
    echo $output->render_indexpage($indexpage);

Note, although I changed the function name from render_index to render_indexpage, it was not necessary. I changed the name only because I felt it better reflected the functionality.

I now have everything I need to output my plugin's index page using renderers and templates. And, other developers can now easily change the way that page display looks using alternate templates or renderers in their themes without having to change any of the plugin's code. The entire plugin's code using this template can be viewed here.

In my next post I will look at alternate coding strategies and how I used renderers and templates for the rest of the plugin's output code.

Wednesday, January 11, 2017

Part 3 - Setting up Templates

In part three of the "Adding renderers and templates to your Moodle plugin" series, I will present how to set up and begin using templates with your plugin renderer.

At this point, in parts one and two, I have created a renderer for my plugin's index page, and modified the output code for the index page to use the renderer.

There are a number of parts to implementing templates. I need a renderer. I did this in the earlier parts of the series. I need output classes based on Moodle's \renderable and \templatable classes. And I need Mustache templates to define the actual HTML output.

The first thing I am going to do is construct the mustache template. A mustache template is a file that looks very much like an HTML file. This makes it easy for designer/developers, familiar with HTML to create a layout display. They do not need knowledge of PHP or Moodle internals.

In my plugin, I need to create a new subdirectory called "templates" to hold the mustache templates I create.  Any template file I create requires the ".mustache" extension. Moodle's output system is set up to use the component name and the templates subdirectory name to locate any templates I specify in the specific functions. Since I am creating one for the index page, I create a file called templates/indexpage.mustache.

If you recall from the renderer posts, my index page was essentially an HTML table, listing the questionnaire instances in the course it was being accessed for. I used the html_writer::table function to generate the HTML of this table with the data I loaded. My new template needs to do something similar, except I will have to write the HTML for the table listing using HTML code in the mustache file.

The file I create contains the following HTML/Mustache code (you can see the whole file here):

<table class="generaltable">
  <thead>
    <tr>
{{#headings}}
      <th class="header" style="text-align:left;" scope="col">{{title1}}</th>
      <th class="header" style="text-align:left;" scope="col">{{title2}}</th>
      <th class="header" style="text-align:center;" scope="col">{{title3}}</th>
      <th class="header" style="text-align:left;" scope="col">{{title4}}</th>
{{/headings}}
    </tr>
  </thead>
  <tbody>
{{#rows}}
    <tr>
      <td class="cell" style="text-align:left;" scope="col">{{topic}}</td>
      <td class="cell" style="text-align:left;" scope="col">{{{name}}}</td>
      <td class="cell" style="text-align:center;" scope="col">{{responses}}</td>
      <td class="cell" style="text-align:left;" scope="col">{{type}}</td>
    </tr>
{{/rows}}
  </tbody>
</table>

You can see from this that the mustache template file looks very much like part of an HTML file, except for the portions contained within multiple brace brackets ("{{ }}"). Anything contained within the brace bracket structures, are mustache constructs that will be replaced by real data when the template is executed.

When a template is executed, it is passed a data structure. The data structure is where the template gets the real data to substitute in place of its brace tagged variables. Deconstructing this template, I have two main data structures: headings and rows. These structures are contained within opening and closing tag pairs, specifically:
    {{#headings}} {{/headings}}
    {{#rows}} {{/rows}}

Each of these structures contains other data defined by tags. The headings structure contains
{{title1}}, {{title2}}, {{title3}} and {{title4}}. The rows structure contains {{topic}}, {{name}}, {{responses}} and {{type}}. This level of the structure is the actual data. When the template is executed, it will expect a data structure that contains these structures passed into it.

Putting the data into section tags allows for some logic to occur within the template. If the section tag exists as a variable in the passed in data, then the output contained within the tag will be displayed. If the section tag does not exist, then the output contained within the tag will not be displayed. Additionally, if the section tag variable is an array, then the output within it will be repeated for each item of the section tag array. For my template, this is important for the rows variable, since there may be multiple rows of data.

In the case of the headings section, if the data contains no headings variable, a table row with no columns will be displayed in the <thead> section. In the case of the rows section, one row containing four columns each (one each of topic, name, responses and type) will be displayed for each element of the rows array. If rows is not present in the data, then no content rows will be displayed.

For this template, since I know I am always going to be passing in the table headings, I really don't need the #header section and structure at all. I could simply pass in the title variables as standalone variables in the template data structure. In that case, I would rewrite the table header section without the opening and closing header tags.

While I haven't discussed the code that constructs the data to be sent to the template yet, I just want to point out some details about how that data can look. For this, assume $data is the data variable I am constructing to pass to the template.

The template will expect the data that is passed in to be either an object with each template tag defined as a property (object variable), or as an array with each template tag defined as an array index. In the case of this template, the data structure expected will consist of either:
    $data->headings
    $data->rows
or:
    $data['headings']
    $data['rows']
or a combination of both.

The headings variable, likewise can be either an object or an array. So for example, either
headings['title1'] or headings->title1 will work successfully with the template data.

For this template, if there are multiple rows of data to be displayed, then the rows variable must be an array structure. Each array item can be either another array or an object however. So, for this template for example, we expect either rows[0]['topic'] or rows[0]->topic. If rows is not an array, there will only ever be one row displayed.

The data structure required is documented as a JSON structure in the template file as comments before the code.

One other thing to point out before I move on. You may have noticed that all of the template variable tags are enclosed in two brace brackets, except for one. The name variable is enclosed in three brace brackets - {{{name}}}.  When a variable enclosed in two brace brackets is substituted for the real data, it is HTML escaped. That means only text will be displayed, and any HTML constructs contained in that variable will be converted to escaped text and displayed verbatim rather than interpreted. When a variable is enclosed in three brace brackets is substituted for the real data, the data is included raw. That means the HTML will be interpreted and displayed accordingly. This is useful when you want to display a block of HTML formatted content.

In the case of my template, the name variable also contains a link to the actual questionnaire instance, so I want the HTML to be displayed as I passed it. In actuality, I should probably rewrite the template so that the pieces of the link are passed instead of the entire HTML structure. Generally speaking, it is safer and better practice to pass only unformatted text.

In the next post, I will begin modifying the renderer code to use the template.