Implementing Moodle's Privacy API in a Moodle Plugin - Part 2

In part 1, I began implementing Moodle's Privacy API in my questionnaire plugin, in order to meet the requirements of the GDPR. In this post, I will add the specific code to do this.

I have a skeleton file in place, that includes all of the class and function specifications that I need. Next, I need to describe each data table that includes user data. The questionnaire has several tables that do this, namely:

• questionnaire_attempts
• questionnaire_response
• questionnaire_response_bool
• questionnaire_response_date
• questionnaire_response_other
• questionnaire_response_rank
• questionnaire_response_text
• questionnaire_resp_multiple
• questionnaire_resp_single

It looks like I need to add each of these to the $collection variable. And, each table and relevant field will require a language string, as shown in the documentation example. To start with, I'll implement just the questionnaire_attempts table.

Adding this table to the get_metadata function means defining the relevant fields. In this case, this table stores the user id, the question id, the response id and the time stamp of when the latest submission for this attempt occurred. Each of these fields can be considered private data, although the question id points to the actual question which really only provides context for a specific question response. I'll stay on the side of providing too much information rather than too little and include it. My function now looks like:

public static function get_metadata(collection $collection) : collection {

    // Add all of the relevant tables and fields to the collection.
    $collection->add_database_table('questionnaire_attempts', [
            'userid' => 'privacy:metadata:questionnaire_attempts:userid',
            'rid' => 'privacy:metadata:questionnaire_attempts:rid',
            'qid' => 'privacy:metadata:questionnaire_attempts:qid',
            'timemodified' => 'privacy:metadata:questionnaire_attempts:timemodified',
        ], 'privacy:metadata:questionnaire_attempts');

    return $collection;
}

And, I add each of the privacy strings to the language file as:

$string['privacy:metadata:questionnaire_attempts'] = 'Details about each submission of a questionnaire by a user.';
$string['privacy:metadata:questionnaire_attempts:userid'] = 'The ID of the user for this attempt.';
$string['privacy:metadata:questionnaire_attempts:rid'] = 'The ID of the user\'s response record for this attempt.';
$string['privacy:metadata:questionnaire_attempts:qid'] = 'The ID of the questionnaire record for this attempt.';
$string['privacy:metadata:questionnaire_attempts:timemodified'] = 'The timestamp for the latest submission of this attempt.';

Now that I have added the metadata, I should be able to see them at the "Plugin privacy registry" page of the site. Navigating to that page, and opening the section on questionnaire, I do indeed see the definitions I just added:

Next, I need to provide a way to retrieve and return the list of contexts for which my plugin stores user data. For my plugin, the only context is CONTEXT_MODULE. And I can determine the context module id for each questionnaire a user has responded to by the qid field in the questionnaire_attempts table and joining tables back through the course_modules table to the context table using SQL. My function looks like this:

public static function get_contexts_for_userid(int $userid): \core_privacy\local\request\contextlist {
    $contextlist = new \core_privacy\local\request\contextlist();

    $sql = "SELECT c.id
             FROM {context} c
       INNER JOIN {course_modules} cm ON cm.id = c.instanceid AND c.contextlevel = :contextlevel
       INNER JOIN {modules} m ON m.id = cm.module AND m.name = :modname
       INNER JOIN {questionnaire} q ON q.id = cm.instance
        LEFT JOIN {questionnaire_attempts} qa ON qa.qid = q.id
            WHERE qa.userid = :attemptuserid
    ";

    $params = [
        'modname' => 'questionnaire',
        'contextlevel' => CONTEXT_MODULE,
        'attemptuserid' => $userid,
    ];

    $contextlist->add_from_sql($sql, $params);

    return $contextlist;
}

Next, I need to provide a way to export user data. The documentation doesn't provide an example, but I can find examples in the core code.

There are a number of data types that must be exported mentioned in the documentation, but questionnaire only needs to worry about the "data" part. The documentation section also describes using the \core_privacy\local\request\content_writer but the code examples in the documentation use \core_privacy\local\request\writer. Looking at the /privacy/classes/local/request/content_writer.php file, I can see that is an interface, while the /privacy/classes/local/request/writer.php is a class described as a "factory class used to fetch and work with the content_writer". So I think the "writer" class has been provided as a shortcut.

Looking at the exporter code for choice and forum, it appears that there is no specific format for the output of a module. The data is structured as JSON, but the elements seem to be up to the plugin. This makes sense, since any plugin can have very different data.

For example, a choice activity export looks like this:

{
    "name": "Choice One",
    "intro": "",
    "completion": {
        "state": 0    
    },
    "answer": [
        "Choice 2"    
    ],
    "timemodified": "Wednesday, 3 May 2017, 6:28 pm"
}

While a forum post looks like this:

{
    "subject": "My New Post",
    "created": "Friday, 1 June 2018, 3:15 pm",
    "modified": "Friday, 1 June 2018, 3:15 pm",
    "author_was_you": "Yes",
    "message": "<p>Hi. This is my new post. I hope you like it.</p>
}

Before I implement an exporter, I will need to decide what the data should look like. I'll stick with my simple attempts data for now. Since any questionnaire instance can have multiple attempts by a user, it makes sense to create a structure organized by the instance; in this case the course module id. So my structure should look like this:

{
    "name": "Questionnaire name",
    "intro": "Complete this questionnaire",
    "completion": {
        "state": 0    
    },
    "attempts": [
        {
            "responseid": "rid1",
            "timemodified": "Wednesday, 3 May 2017, 6:28 pm"
        },
        {
            "responseid": "rid2",
            "timemodified": "Thursday, 4 May 2017, 9:31 am"
        }
    ]
}

Looking at the choice activity code for the exporter, I create a function to create the JSON structure I am aiming for. You can see the code here. This code uses several functions provided by the API that are not documented in the wiki. The documentation is really in the class files themselves.

The following line displays the time and date in a readable form:

'timemodified' => \core_privacy\local\request\transform::datetime($attempt->timemodified),

You can find the datetime function in the /privacy/classes/local/request/transform.php file.

The following line gets a structure containing general data for the activity and user that can be merged with the data more specific to the activity:

$contextdata = \core_privacy\local\request\helper::get_context_data($context, $user);

This function is contained in the file /privacy/classes/local/request/helper.php. Following through that code, it creates the part of the JSON structure I need, prior to the 'attempts' array.

The following lines, merge the specific data I want to export with the general data and then writes that JSON data to the export function:

$contextdata = (object)array_merge((array)$contextdata, $attemptdata);
\core_privacy\local\request\writer::with_context($context)->export_data([], $contextdata);

The with_context function is contained in the file /privacy/classes/local/request/writer.php,  and calls the export_data function which is ultimately located in the /privacy/classes/local/request/moodle_content_writer.php file.

The end result of this is an exported structure in JSON form.

Now, to test this, Moodle has provided some scripts that can be created and executed from the CLI. The one I want to use is the "Test of exporting user data" script, provided on that page. So, I create that script on my test site, and execute it. When I execute it, there is a lot of output. Scanning through the output, I see:

"Processing mod_questionnaire (4/15) (Friday, 1 June 2018, 8:37 pm)"

which is positive.

And the last line says:

"== File export was uncompressed to /moodledevsite/moodledata/temp/privacy/3d5750c5-4d5b-4e96-9e86-663cbc9ed177".

This means that there is data located in my moodledata directory, that should contain the exported data. A visual structure of that area looks like this:

I have opened it to the questionnaire I am testing. The "data.json" file will contain the data I exported. When I open the JSON file, I see:

{
    "name": "Test Questionnaire",
    "intro": "<div><p>A wonderful description of the questionnaire.<\/p><\/div>",
    "completion": {
        "state": "1"    
    },
    "attempts": [
        {
            "responseid": "66",
            "timemodified": "Friday, 18 November 2016, 8:14 pm"        
        },
        {
            "responseid": "88",
            "timemodified": "Tuesday, 11 April 2017, 8:50 pm"        
        },
        {
            "responseid": "89",
            "timemodified": "Tuesday, 11 April 2017, 8:54 pm"        
        }
    ]
}

Which appears to match what I wanted.

That's some good progress. In Part 3, I'll add the delete data portion of the API.