Monday, October 27, 2014

OneNote, the Surface and Moodle

I recently (through my company, Remote-Learner) became involved in a partnership project with Microsoft Open Technologies. This project has the ambitious goal of providing tighter integration of a variety of new Microsoft products with Moodle. I'm very excited about this.

The products we are focusing on initially include OneDrive, OneNote, Outlook Calendar and the various pieces these integrations will support, such as the Office 365 applications and Azure Active Directory.

For me, this meant getting myself reacquainted with Microsoft technology and systems. I chose to buy a new Surface Pro 3 running Windows 8.

The Surface is an interesting device. From my perspective, it’s a tablet that tries to be a laptop when you want it to be. But in my experience so far, I like it far better as a tablet. And for me, the killer app is OneNote on the Surface.

If you are not familiar with OneNote (I wasn't), it has been around for a lot longer than you think - since at least 2003. If you're familiar with Evernote, its a very similar application. To put it simply, it is your collection of notebooks, available to you on all your devices. You organize each notebook into sections, and add pages to the sections. Pages can include your created notes, media, clipped information from the web and photos you take on the device. Kind of a notebook / scrapbook hybrid.

OneNote is integrated pretty tightly with the Surface. It comes pre-installed as a Surface app, meaning it works well with the tablet interface of Windows 8 and the Surface stylus. The stylus allows you to call up OneNote with a click of the button on the end (like clicking a ballpoint pen), and then write your note using handwriting, on the open page. For myself, I have really gravitated to using the stylus and OneNote to take quick notes, in the same way I used to use a pen and a pad of paper. And if you upgrade to OneNote 2013, you can use the "Ink-to-text" feature to turn your handwriting into a digital text document.

Marking up existing documents in OneNote is a great feature. For example, you can import a PowerPoint slide into a note, and then mark it up with your comments and drawings (if you use the stylus). Likewise with a clipped web page. And add-ons like Office Lens, let you use your device camera as a scanner, capturing notes on paper or whiteboards directly into your OneNote notebook. And you can then mark them up as needed. In fact (I haven't tried this yet), I believe you could even capture audio markup and place it in the page (I will try this and confirm in a later post).

You can share your notebooks with other collaborators, via email addresses. People you share with don't even need the OneNote app; they can use OneNote on the web as a web-based application. The OneNote notebooks and their content are all stored in the OneDrive cloud, making them available to any connected device.

Prior to using OneNote with my Surface, I was unsure what an integration of OneNote with Moodle would be good for, or how it would be useful. Now I really want to make this happen.

From a learning system standpoint, consider the markup and collaboration. As a Moodle assignment, OneNote could be submitted for grading. If the OneNote interface was used by all participants, teachers could mark up the submission directly on the pages. And, as a collaborative assignment, multiple users could work on the submission contributing and marking up existing content before submitting.

And that is where we have our focus right now. In the works are integrations to allow OneNote to become an assignment type. To be fair, OneNote can be used to create assignment documents to be submitted by upload to Moodle right now. OneNote allows the exporting of its notes as PDF's, Word docs or as single web pages. But we want to make it work without that extra step.

So, for the next little while, we will be working with the Microsoft Open Technologies group to build out the necessary Moodle pieces. As a start, these will include:

  • SSO with Azure Active Directory - necessary to make the account access between Moodle and the cloud seamless for the user.
  • OneNote and OneDrive repository plugins - give easy access to document in OneDrive and specific parts of OneNote.
  • OneNote assignment plugins.
I will post regular updates to our progress along with my thoughts and experiences with the technologies and how they can be further used with Moodle and the learning environment.

Watch this space for more information on what's coming and how you can be involved.

Friday, April 11, 2014

Chromecast - What have we got here?

Starting out

I'll be honest, I wanted to try the Chromecast out primarily to see how it would work as an entertainment piece at home, but I am curious to see if it has other potential applications for presentation in business and education. I already have a 1st gen AppleTV that has been jailbroken so it can run XBMC, and a Raspberry Pi that can run XBMC but this little dongle for $39 piqued my curiosity. Can it replace all of that and provide a highly accessible streaming point for any media source? If it can, and is as portable as it appears, it could be a device that sits in the classroom and any place where collaborative presentation can happen.

Setting up

To start, I can see that Google has taken a page from Apple's book and created attractive, fun-to-open packaging. The box is much heavier than expected, but I now see the weight is primarily the box; the device and adapter are as light as I expected.


The instructions were simple; there are three of them detailed on the inner box lid:

  1. Plug into TV & power.
  2. Switch TV input.
  3. Set it up at chromecast.com/setup.

And, it really was that simple. I plugged the dongle into the side HDMI port on my TV, plugged the power into the wall (turns out I could actually just plug the USB end of the power into the available USB port on my TV; I'll do that later), switched the TV to HDMI4, and I'm looking at a Chromecast setup screen:


The website link takes you to a download link to get the app you need to set it up. You can also download this app on any Android and iOS device. The only thing this app is needed for is to set it up and configure it later.

I used the app on my Nexus 7 Android tablet. The app found the new Chromecast device right away, and I had to enter the code it displayed on the TV to verify the connection. The only other things I had to do were change the name of the device if I wanted to (I did), and give it my WiFi password. After that, everything was automatic. A couple of automatic updates and a reboot later and it was up and running. I don't think it could be much easier.

Time to play:

I had the advantage of a room full of people when I first started it up. We had several devices and systems, Android and iOS. Each of us looked at our Youtube apps, and there was a new icon. This icon was similar to the Airplay icon on iOS devices, and selecting it allowed us to choose the new Chromeplayer device as output. Again, this was available on an Android phone and tablet, as well as an iOS phone and tablet, without doing anything to those devices first. The one exception was an iPhone in the group that had not been regularly updated. But, a quick App Store update to the Youtube app, and it too could use Chromecast.

Sticking with Youtube, we each were able to target a video play to the Chromecast. It is interesting to note that the way this works with Chromecast is not what I originally thought. When you play, you are not streaming the content from your device to the Chromecast. Instead, you are telling the Chromecast Youtube app to play the content you requested. In that sense, each of us is using a remote control. It also means that any device can request the play and then carry on doing anything else (I didn't check if I could power down, but I suspect not).

Testing other apps, all of the devices could similarly use Netflix, Vevo and Songza. The Android devices could also use Google Play Music and Google Play Movie. I believe the Google Play apps may allow streaming from the device but I have not verified that. The native music player on the iOS devices could not select the Chromeplayer however.

While playing with the Youtube app, we all discovered that the app on our device would show us what was playing on the Chromecast device, regardless of whether we were the ones who played it or not. We then realized that there was an "add to queue" option instead of "play". This allowed each of us to add a video to the playlist and not stop the one that was already playing. The queue was similarly visible to all of us. We quickly set the ground rules that no video (music only) could be aborted or removed from the queue without two thirds vote. And we each began to plan the music.

The queueing feature seemed to be Youtube specific, and did not exist on the Vevo app. I think this feature would be one that would be copied by many other Chromecast apps. It would be really interesting if it could become a standard part of the Chromecast itself, allowing all types of play requests to be queued.

Observations

We were all pretty excited about being able to queue up videos on the Youtube app, but we also would have liked to be able to send anything we had - music, videos and photos. It would also be good to have a "mirroring" option like with Airplay. These might exist, or might be in development. I still need to do a lot more looking and playing.

The fact that the basic functions were simply "there" for everyone without having to do anything was huge. Since the device is so portable, this makes it something that can be carried with you to other locations and easily used. All you need is an HDMI device, WiFi and power.

Looking ahead

The entertainment value of the device was very obvious. But to be true to my subject, could this be used for any kind of learning environment?

I think, this could be huge if it can just fill in the gaps. It has the ability to be a collaborative presentation tool replacing projectors, cameras, laptops, etc. Every person has a controller, a recorder, a camera and a media library. And they can all present to the common screen.

Why this can be huge, and some other things that are needed...

  1. Be completely cross platform. Out of the box, its pretty close. If it could allow direct streaming from all platforms as well, this would be a slam dunk. Cross platform would also make this a far better option than Airplay on Apple TV.
  2. The price point is amazing. $39 in Canada. This falls into just about everyone's budget. The more people who buy this and play with it, the more clever people will come up with great applications.
  3. It needs to be able to mirror the device screen. Airplay on iPhones and iPads can do this. Consider the power of using your device as a live camera in a walk around the room way. Consider the power of demonstrating your work.
  4. The collaborative queueing feature in Youtube gives a hint as to how this could be used in an active, exciting way. Imagine collaborative groups getting their items into the play queue, and then have feedback/voting actions on the queue. Queueing and queue control at the device level would be very interesting.
The best part of this is that any of these outstanding needs could be filled by the community of app creators. I think I need to go learn app development.


I'll post again about this when I see or learn more.

Saturday, March 16, 2013

Setting up a Moodle Development Environment on a Mac

Having been an Ubuntu user for years, I am switching my main computer to a new Macbook Pro. This means that I now need to change the way I do things, and learn how to set up a Mac to be a developer station. So, here I go...

To begin with, I consulted with my colleagues +Justin Filip and +Amy Groshek, both Moodle developers and both power Mac users. They both pointed me to MacPorts as a starting point. Amy further advised me that although OSX comes preinstalled with PHP and Apache, I probably want to replace them both with MacPorts versions of the software. An opinion shared by a blog post Amy shared with me on setting PHP up using MacPorts. I'm going to use that blog post, and the MacPorts installation page to start my setup.

Now, one thing I need to be careful with is that this blog post is for up to version Lion. I'm running version Mountain Lion; so there may be some differences.

...and the first thing I run into is those differences...

The blog post tells me to turn off Apache:
"go to System Preferences, type Apache in it’s spotlight. It will highlight Sharing, but you just need to press return. Just make sure Web Sharing is disabled."
Unfortunately, "Apache" turns up nothing. And accessing "Sharing" direct doesn't offer "Web Sharing". I do a search for "Mountain Lion" and "Apache", and turn up several articles describing how to enable Apache in the absence of the "Web Sharing" option. It turns out that Apache is disabled by default with OS-X Mountain Lion. I'm guessing, I can skip this step.

The next thing I am asked to do is to install Xcode. This should be available in the Mac App Store, so I'm going to check there first. Searching for 'xcode' in the App Store I immediately find it. It defines itself as "everything developers need to create great applications for Mac...". Checking the "Installing MacPorts" page on the MacPorts site, the "Xcode" requirement is confirmed. So, I install it from the App Store.

Next, the MacPorts site is telling me I need to have the Command Line Developer Tools. The instructions say they can be installed from within Xcode 4, which I just installed. I find Xcode in my Applications folder and run it. It takes me through more installation steps, and then allows me to start it. According to the information on that page, simply running and accepting the license should be enough. But I find another page that points me to the "Preferences -> Downloads" section where I need to install the "Command Line Tools".

Next, both sites tell me to install MacPorts by downloading the dmg from the site. I think I'll follow that advice. The download runs me through a standard installation process, and I choose the standard method. It all seems to install just fine.

Having done that, I move on to installing MySQL. The blog post I'm reading recommends installing the MySQL server installation using MacPorts. After reading others, I decide to do just that. The recommended commands are:
sudo port install mysql5-server -- installs all necessary software
sudo port load mysql5-server -- configures MySQL to start at bootup
sudo -u _mysql mysql_install_db5 -- sets up necessary database tables
/opt/local/lib/mysql5/bin/mysqladmin -u root password 'new-password' -- sets a root password 
Next, I will install PHP, Apache and some required libraries.
sudo port install php5 +apache2 +pear +fastcgi php5-mysql +mysqlnd -- installs all necessary software
sudo port load apache2 -- configures PHP to start at bootup 
Then I will configure PHP with development settings:
cd /opt/local/etc/php5/
sudo cp php.ini-development php.ini
 
And enable it in Apache:
cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so 
Next, I can change where my Apache sites will run from. By default, they are in "/opt/local/apache2/htdocs", but I want them to be in subdirectory off my home. To do this, I edit the "/opt/local/apache2/conf/httpd.conf", and change the two lines below as follows:
DocumentRoot "/opt/local/apache2/htdocs"
to
DocumentRoot "/Users/mikechurchward/www"
and
<Directory  "/opt/local/apache2/htdocs">
and
<Directory "/Users/mikechurchward/www">
This allows me too store my web, and PHP files in my home directory. Some other changes in that file:
<IfModule dir_module>    DirectoryIndex index.html index.php</IfModule>
AddType application/x-httpd-php .phpAddType application/x-httpd-php-source .phps
This should give me everything to run web and PHP.

Now, I reboot my computer and try to access a web page on my machine. It works.

Next, I need git. This looks like an easy task for MacPorts. A quick search concludes that "git-core" is what I should need, so I issue the command:
sudo port install git-core +bash_completion
This installs git-core as well as a variant known as bash_completion, which will allow the "tab" key to help complete git commands.

Now I use git to checkout a Moodle installation, and it works!

So, I will now navigate to my new Moodle codebase, and:
Moodle requires the iconv PHP extension. Please install or enable the iconv extension.
Okay. So I need to add more PHP extensions. Luckily, Moodle documents what I need.

To add the iconv extension, I enter:
sudo port install php5-iconv
then restart Apache:
sudo /opt/local/apache2/bin/apachectl restart
Once more to the site and I actually move on the installation page!

To make my life easier, I think I will add phpMyAdmin before moving on.
sudo port install phpmyadmin
Then, since I have moved my webroot, I move it:
sudo mv /opt/local/www/phpmyadmin ~/www/
Then reconfigure it with:
cd /opt/local/www/phpmyadmin/sudo
cp config.sample.inc.php config.inc.php
 
$cfg['Servers'][$i]['auth_type']     = 'config';    // Authentication method (config, http or cookie based)?
$cfg['Servers'][$i]['user']          = 'root';      // MySQL user
$cfg['Servers'][$i]['password']      = '';          // MySQL password (only needed with 'config' auth_type)
When I try to run it, I get an error about socket connections not working. After a bit of research, I come up with this:
Create /opt/local/etc/mysql5/my.cnf, add the following to it and save    [mysqld_safe] socket = /tmp/mysql.sock
...and...
Also to remain compatible with other programs that may have been coded to look for the socket file in its original location then add this symbolic link:
    sudo ln -s /opt/local/var/run/mysql5/mysqld.sock /tmp/mysql.sock
That gets phpMyAdmin running. Now onto install Moodle.

I step through the initial steps of Moodle until I hit the "Server checks" page. This page tells me exactly what other PHP extensions I'm missing.
sudo port install php5-curl
sudo port install php5-openssl
sudo port install php5-xmlrpc
sudo port install php5-soap
sudo port install php5-intl
And reload the page. Success. The installation continues, and runs successfully.

I now have most everything I need... But I still need an IDE.

I've been an Eclipse IDE user for a very long time, but both +Amy Groshek and +Justin Filip are telling me that Sublime Text 2 is the way to go, so I'm going to see what its like. Now, to install and use it, I'm just going to use Stuart Herbert's page. No need to recreate that here.

So, at this point, I have a fully working development environment for Moodle working on my Macbook Pro. I'm sure I'll need more as I go along, but this is a good starting point.

Friday, November 9, 2012

Adding Moodle 1.9 Block Restore to Moodle 2.3 - Part 6

In my last post, I had managed to get a working system that would successfully convert blocks in a 1.9 backup file to a Moodle 2.3 restore. I need to do a bit more testing, and then look for a solution for blocks that have specific backup/restore needs.

Testing uncovered that trying to restore a 1.9 course that had multiple versions of the same block installed within it (such as multiple HTML blocks) would cause an error, indicating that a file already existed. Referring back to the new code, this is likely due to the new process_block() function. In that function, a new XML file is created using the name of the block. This means that if more than one of the same block is converted, the code would try and create more than one file with the same name. I need to change that code to make sure it creates a unique file name in this situation.

I change the code:
$this->open_xml_writer("course/blocks/{$data['name']}/block.xml");
...
$this->open_xml_writer("course/blocks/{$data['name']}/inforef.xml");
...
$this->open_xml_writer("course/blocks/{$data['name']}/roles.xml");
to:
$this->open_xml_writer("course/blocks/{$data['name']}_{$instanceid}/block.xml");
...
$this->open_xml_writer("course/blocks/{$data['name']}_{$instanceid}/inforef.xml");
...
$this->open_xml_writer("course/blocks/{$data['name']}_{$instanceid}/roles.xml");
By adding $instanceid to the file path, I guarantee that each XML file will have a unique name. A quick test proves that this is now working.

The next issue to tackle is what to do with blocks that have their own backup/restore needs. The one Moodle 2 block that seems to have this requirement is the block rss_client. It will need its own backup converter.

In the tracker Paul Nicholls started to tackle this by creating a new "blocks/rss_client/backup/moodle1/lib.php" file with an extended class. When he tested the first version, a bug was discovered in my moodle1_handlers_factory::get_plugin_handlers() function changes. If a block does have its own class extended, the file containing it is not included, meaning that the handler class is unknown and an error occurs during the conversion process. Paul has also provided a fix for this error using his own git repository. To fix my copy, I will use the power of git collaboration.

First, I need to be able to pull code from Paul's repository into my local one. I have already made this possible by adding his repository as a remote named "pauln" to my local one. The commit that fixes the code is identified, so I cherry-pick it into my repository with:
git fetch pauln
git cherry-pick 630756fe1063e96cc7a6057bd512aa4afa53bc61

This modifies the code from:
if ($type != "block") {
    if (!file_exists($handlerfile)) {
        continue;
    }
    require_once($handlerfile);
} else {
    if (!file_exists($handlerfile)) {
        $handlerclass = "moodle1_block_generic_handler";
    }
}
to:
if (file_exists($handlerfile)) {
    require_once($handlerfile);
} elseif ($type == 'block') {
    $handlerclass = "moodle1_block_generic_handler";
} else {
    continue;
}
Paul has also provided a new version of the rss_client block with the new handlers. Again, I cherry-pick using:
git cherry-pick d89ade28060ae858356ff4a42cf946a4c99878c0
This provides me with a working copy of the conversion code that will correctly add the extra functionality that the Moodle 2.3 rss_client block requires.

For the record, here is the code for the rss_client conversion handler:
class moodle1_block_rss_client_handler extends moodle1_block_handler {
    public function process_block(array $data) {
        parent::process_block($data);
        $instanceid = $data['id'];
        $contextid = $this->converter->get_contextid(CONTEXT_BLOCK, $data['id']);

        // Moodle 1.9 backups do not include sufficient data to restore feeds, so we need an empty shell rss_client.xml
        // for the restore process to find
        $this->open_xml_writer("course/blocks/{$data['name']}_{$instanceid}/rss_client.xml");
        $this->xmlwriter->begin_tag('block', array('id' => $instanceid, 'contextid' => $contextid, 'blockname' => 'rss_client'));
        $this->xmlwriter->begin_tag('rss_client', array('id' => $instanceid));
        $this->xmlwriter->full_tag('feeds', '');
        $this->xmlwriter->end_tag('rss_client');
        $this->xmlwriter->end_tag('block');
        $this->close_xml_writer();

        return $data;
    }
}
I don't think that this handler will be typical of 1.9 blocks that need to restore their own data though. As it turns out, although the rss_client block does store its own data (feed definitions) in its own table, the 1.9 code never backed that data up. So all 1.9 backups contain only the standard block information for these blocks. But, in Moodle 2.3, the restore operation expects that the feed definition data will be there, and if its not included (even empty) then it causes an error. To work around this, the conversion handler creates an empty version of the expected feed definition XML file for all 1.9 conversions. I will do an extra post later to show how a block that really has extra data would provide the correct conversion code.

There are no other core Moodle blocks that require any specific restore conversion code, so I am essentially done the coding that is required by the tracker issue. Next, I'm going to ready the code for use by the HQ integrators and complete the tracker data so that they can consider it for inclusion into core.

First things I need to do is make sure my git repo is in a good state. To start with, I will make sure that the main branch of Moodle my development branch was based off of is up to date with the latest upstream Moodle code:
git checkout MOODLE_23_STABLE
git fetch upstream
git pull upstream
git push origin MOODLE_23_STABLE
This part isn't necessary, but I want to make my development branch's name a little more manageable, so I rename my MOODLE_23_STABLE_MDL-32880 branch to MDL-32880_2.3. This is purely for optics.

Now, I want to compress all of the work I have done on this issue into one commit. This makes for easier documentation and understanding, since several of my commits were code tests that were later undone. The "rebase" function of git allows me to do this. From the branch MDL-32880_2.3, I issue the command:
git rebase -i MOODLE_23_STABLE
This command will merge all of my commits with the base branch specified, keeping them in my branch, but allowing me to compress some things. The "-i" portion stands for "interactive" and brings up an editor screen that allows me to change what happens:
pick 00cab4e MDL-32880 - Playing with ideas.
pick c7849a8 MDL-32880 - Playing with more ideas.
pick 5036ed8 MDL-32880: moodle1 backup converter: add basic block handler
pick b346f7e MDL-32880 - Adding generic block conversion handlers.
pick 6a3045e MDL-32880 - Named block XML file with instance, and corrected use of currentmod to currentblock.
pick 7b9e3ec MDL-32880: moodle1 backup converter: Include custom block handers if present
pick 6fbf1fb MDL-32880: Add moodle1 backup converter for rss_client block.
pick 1012e75 MDL-32880 - Trying to use convert API in specific block instance.
pick 0b86f82 Revert "MDL-32880 - Trying to use convert API in specific block instance."

# Rebase ce44bf4..2bf7777 onto ce44bf4
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
This screen shows all of my commits in my development branch with "pick" and their information next to them. I want to compress all of these changes into one commit, so I change all but the first "pick" to "squash", and save the edits. Once I do this, I have told git to combine and merge all of the commits into one diff commit. The next thing git does is allow me to create a new commit message for the new combined commit, which I simplify to:
"MDL-32880 - Adding 1.9 block conversion to M2 restore."

Now my MDL-32880_2.3 development branch is up to date with the main Moodle code, and contains just one committed change containing all of the necessary changes for the block restore conversion. This will make it easier for others to review and include in their testing.

The last things that need to be done in order to have this considered for inclusion into core is to complete the rest of the tracker information. Refer to the tacker item MDL-32880 once again to see the results. In this case, Paul has completed all of the necessary information which includes:
  • Testing instructions: Specific step-by-step instructions on how to verify that the code will do what it says. Paul has also attached a 1.9 backup file containing all core blocks.
  • Pull from Repository: Git repository containing the new code.
  • Pull X.X Branch: The name of the branch in the repository containing the code for the specified "X.X" version. For mine, it would be "MDL-32880_2.3".
  • Pull X.X Diff URL: A link to the repository that specifies a "diff" between the codebase and the changes, so that reviewer can see a compare view. This is made easy in github, using the "branches" tab and selecting your base branch from the drop-down, and the selecting the specific branch to compare against. For mine, it produces this link: https://github.com/mchurchward/moodle/compare/MOODLE_23_STABLE...MDL-32880_2.3
 I will continue this series next by providing an example of a 1.9 block that does convert its own data.

Monday, November 5, 2012

Adding Moodle 1.9 Block Restore to Moodle 2.3 - Part 5

In my last post, I had determined that I could make things happen correctly through the correct use of the get_paths() function creating convert_path objects. This led me to believe that I could now plan out the changes necessary to complete this task.

If you have followed along with the MDL-32880 tracker issue, you will notice that there has been some activity from other participants. Particularly, Paul Nicholls has done some work in his repository and proposed some changes. While I won't use all of them, some of them will make my work easier.

I know that the moodle1_handlers_factory::get_handlers() function (in file "/backup/converter/moodle1/handlerlib.php") uses the get_plugin_handlers() function to load all handlers it finds for blocks. But, since it expects to find specific class files and functions for any block to convert, it doesn't do anything for blocks (since none of the blocks provide this function). Paul's solution involves adding the appropriate file and function to each block in core to make the conversions happen. I still believe I can do this for every existing block without having to create new class files for every block.

I have looked closely at the functions in "handlerlib.php", and my plan is to do almost all of the work in there. My plan is:
  • Use the existing moodle1_block_handler class as the main handler. It will define all of the default, necessary processes for any block conversion.
  • Extend that class as moodle1_block_generic_handler, for use by blocks that don't require their own conversion handlers. This extended class will simply call all of the default functions.
  • Any block that requires more than the default conversions can extend and override this class with their own derived class, using the convention moodle1_block_[blockname]_handler.
  • Modify the get_plugin_handlers function to load the generic or block specific handler depending on the block. If a specific block handler class exists it will load it, otherwise it will load the generic class.
For my plan, the best place to start is with the main handler, defining the default processes. This would be adding a get_paths and a process_block function to the moodle1_block_handler. At this point, I want to take a look at what Paul Nicholls has proposed.

In Paul's changes, he has provided a moodle1_block_handler::process_block() function that pretty much does everything we need for all of the blocks. He has also provided a specific get_paths() function in every block to define the use of the process_block function. I can use the first part, but I will create a generic get_paths() function instead.

To use Paul's process_block function, I could just cut and paste his code into my version of the file. But instead, I am going to use the strength of Open Source collaboration and pull his changes into mine using Git.

I already have a local version of my Git repository checkout from my "github" repository, with a working branch for this issue. My local repository also has the main Moodle repository added as a remote named "upstream" to allow me to get the latest Moodle code from HQ. I can add Paul's as a new "remote" to my local one allowing me to "pull" commits from his. From the tracker issue, Paul's repository is https://github.com/pauln/moodle/ and he has documented specific commits for his work. So, in my local Git repository I issue the commands:
git fetch upstream
git pull upstream MOODLE_23_STABLE
This makes sure I have all of the latest Moodle 2.3 code in my local repository before I do any more changes. Next, I issue:
git remote add pauln git://github.com/pauln/moodle.git
git fetch pauln
This attaches Paul's repository to mine as a remote, and gets his latest update information. Now, the next command is the important one. I can grab just the changes he made to moodle1_block_handler by cherry picking the commit that modified that code, using the specific commit hash associated with it. I issue:
git cherry-pick 76a7c44491120a696104cd5a4755890c1968777b
This pulls in Paul's changes that added the process_block function to the moodle1_block_handler and maintains his commit information so that his changes are credited to him. This allows me to use his work without having to duplicate it. Once complete, I have the function:
public function process_block(array $data) {
    $newdata = array();
    $instanceid     = $data['id'];
    $contextid = $this->converter->get_contextid(CONTEXT_BLOCK, $data['id']);

    $newdata['blockname'] = $data['name'];
    $newdata['parentcontextid'] = $this->converter->get_contextid(CONTEXT_COURSE, 0);
    $newdata['showinsubcontexts'] = 0;
    $newdata['pagetypepattern'] = $data['pagetype'].='-*';
    $newdata['subpagepattern'] = '$@NULL@$';
    $newdata['defaultregion'] = ($data['position']=='l')?'side-pre':'side-post';
    $newdata['defaultweight'] = $data['weight'];
    $newdata['configdata'] = $data['configdata'];

    // block.xml
    $this->open_xml_writer("course/blocks/{$data['name']}/block.xml");
    $this->xmlwriter->begin_tag('block', array('id' => $instanceid, 'contextid' => $contextid));

    foreach ($newdata as $field => $value) {
        $this->xmlwriter->full_tag($field, $value);
    }

    $this->xmlwriter->begin_tag('block_positions');
    $this->xmlwriter->begin_tag('block_position', array('id' => 1));
    $this->xmlwriter->full_tag('contextid', $newdata['parentcontextid']);
    $this->xmlwriter->full_tag('pagetype', $data['pagetype']);
    $this->xmlwriter->full_tag('subpage', '');
    $this->xmlwriter->full_tag('visible', $data['visible']);
    $this->xmlwriter->full_tag('region', $newdata['defaultregion']);
    $this->xmlwriter->full_tag('weight', $newdata['defaultweight']);
    $this->xmlwriter->end_tag('block_position');
    $this->xmlwriter->end_tag('block_positions');
    $this->xmlwriter->end_tag('block');
    $this->close_xml_writer();

    // inforef.xml
    $this->open_xml_writer("course/blocks/{$data['name']}/inforef.xml");
    $this->xmlwriter->begin_tag('inforef');
    // TODO: inforef contents if needed
    $this->xmlwriter->end_tag('inforef');
    $this->close_xml_writer();
     // roles.xml
    $this->open_xml_writer("course/blocks/{$data['name']}/roles.xml");
    $this->xmlwriter->begin_tag('roles');
    $this->xmlwriter->begin_tag('role_overrides');
    // TODO: role overrides if needed
    $this->xmlwriter->end_tag('role_overrides');
    $this->xmlwriter->begin_tag('role_assignments');
    // TODO: role assignments if needed
    $this->xmlwriter->end_tag('role_assignments');
    $this->xmlwriter->end_tag('roles');
    $this->close_xml_writer();

    return $data;
}
The other piece I need is a generic get_paths function, that creates convert_path objects for each block without having to add it to the specific block's code base. Since moodle1_block_handler is an extension of moodle1_plugin_handler, the name of the plugin (the block in this case) is stored in an object variable called pluginname. I can use this to create a default, block-aware get_paths function:

public function get_paths() {
    $blockname = strtoupper($this->pluginname);
    return array(
        new convert_path('block', "/MOODLE_BACKUP/COURSE/BLOCKS/BLOCK/{$blockname}"),
    );
}
This pretty much gives me everything I should need to convert default blocks to Moodle 2 restore data. Now, since moodle1_block_handler is an abstract class, I need to extend a new class that uses it so that it can be executed. This will be my moodle1_block_generic_handler class. I create:

/**
 * Base class for block generic handler
 */
class moodle1_block_generic_handler extends moodle1_block_handler {

}
At this point, that's all I need in  that class, since it will just be using the default functions.

The last thing I need to do is to modify the get_plugin_handlers function to load the proper handling class for each block it finds. Currently, it only loads a class if a specific handler exists for each block. I want to change it so that it loads a specific handler if it exists, otherwise it loads the generic class. I modify the existing get_plugin_handlers class as follows:
protected static function get_plugin_handlers($type, moodle1_converter $converter) {
    global $CFG;

    $handlers = array();
    $plugins = get_plugin_list($type);
    foreach ($plugins as $name => $dir) {
        $handlerfile  = $dir . '/backup/moodle1/lib.php';
        $handlerclass = "moodle1_{$type}_{$name}_handler";
        if ($type != "block") {
            if (!file_exists($handlerfile)) {
                continue;
            }
            require_once($handlerfile);
        } else {
            if (!file_exists($handlerfile)) {
                $handlerclass = "moodle1_block_generic_handler";
            }
        }

        if (!class_exists($handlerclass)) {
            throw new moodle1_convert_exception('missing_handler_class', $handlerclass);
        }
        $handlers[] = new $handlerclass($converter, $type, $name);
    }
    return $handlers;
}
Essentially, if the plugin type is not a block, continue doing what it always did. If it is a block, and there is no specific block handler defined, then load the generic block handler moodle1_block_generic_handler.

If you want to see the specific code I used, you can see it by the specific commits:
https://github.com/mchurchward/moodle/commit/5036ed80ad57c65c8cc18409e0b59307e0061c52
and
https://github.com/mchurchward/moodle/commit/b346f7e63c9f105dc27fc47707349b90e172ad52

With these changes, I rerun a test restore of a 1.9 backup file with blocks, and it works! The restored course contains the blocks that were in the 1.9 site. But I'm not done yet. I have to handle the blocks that have extra restore needs. That will be in the next post.

Monday, October 29, 2012

Adding Moodle 1.9 Block Restore to Moodle 2.3 - Part 4

At this point, I think I need to step back and try to understand how the conversion process is executed so that I can make some good decisions on how to structure the programming solution. So, I'm going to make one more run through the code to see what is happening. I'll warn you now, this post will be difficult to read, and attempts to document my search-and-reverse-engineer the code for 1.9 conversion.

I feel like if I could just figure out how the moodle1_question_bank_handler class is used I could be most of the way there... It seems like most of the work is done in this class's process_question function, but I can find no place in Moodle where this function is actually called, meaning it must be a constructed function in the Moodle code.

As an aside, what I mean by a "constructed function" is that the function name that is called is actually determined by logic at execution time and stored in variable names. This is a strong programming feature in PHP that is used in Moodle. It allows for function calls to be dynamically constructed from variable names. While a powerful programming tool, it can make reverse engineering very difficult.

Given this, it is likely that the act of registering the question bank handler in moodle1_converter::register_handler()causes some chain of events that allows process_question to be called somewhere!

Inside moodle1_converter::register_handler() is where it begins to get obscure, but it looks like the handler information gets stored in the moodle1_converter::pathelements variable as well as within the moodle1_converter::xmlprocessor object. Searching for use of the pathelements variable, I find it processed in the moodle1_converter::process_chunk(), path_start_reached() and path_end_reached() functions. All of these functions have specific code sections set up to handle blocks in a similar way to modules.

Likely, the function call is created in process_chunk by the use of the get_processing_object() and the get_processing_method() functions. The only place I can find a likely definition of the get_processing_method() function is in the convert_path class. And convert_path objects are loaded and returned by the get_paths function in every handler including the moodle1_question_bank_handler.

And there it is! The handlers' get_paths function creates new instances of convert_path (located in "/backup/converter/moodle1/lib.php"), loading a name and a path argument. The name argument of this function is used to construct the "process_[name]" function. This in fact happens in the convert_path constructor with the line:
$this->set_processing_method('process_' . $name);
Also noted, is that this function definition is a default definition, meaning any implementing class can change this. This may come in handy when I am adding all blocks to be converted.

Looking specifically at the question handler again, it instantiates a convert_path object, passing the name "question" to it, which is used to define the processing method as "process_question". This function is then returned in the get_processing_method function call made in the moodle1_converter::process_chunk function call before dispatching the function itself. Now, I just need to determine what calls process_chunk.

Searching further, process_chunk() is called by moodle1_parser_processor::dispatch_chunk() which appears to be only called by the parent classes of this object.

Stopping to think, at this point, I probably don't have to worry about where the calls are made from. I can derive that, the act of loading convert_path objects with appropriate function call information will at some point, trigger their execution. So, I just need to jump back to the block handler code and redesign it so that it loads paths for all blocks. The act of loading functions for these will allow me to control the conversions! Eureka!

I think I now know enough to design the programming structure for the block conversion.

Friday, October 12, 2012

Adding Moodle 1.9 Block Restore to Moodle 2.3 - Part 3

This is part three of my series on writing the Moodle 1.9 to 2.3 block restore conversion code for Moodle core.

Up to this point, all I've really done is figured out some of the basic operation of the Moodle 1.9 restore converter, and verified my findings using some new test code. This exploration has shown me that the heavy lifting I need to do is convert the data format from a 1.9 backup file to the data format used by Moodle 2. Essentially, I need to look at what data was stored in a 1.9 backup, compare it to what is in a 2.x format and plan for that transformation.

Looking at the backup XML first, in 1.9 I see:
<BLOCK>
  <ID>319066</ID>
  <NAME>participants</NAME>
  <PAGEID>26328</PAGEID>
  <PAGETYPE>course-view</PAGETYPE>
  <POSITION>l</POSITION>
  <WEIGHT>0</WEIGHT>
  <VISIBLE>1</VISIBLE>
  <CONFIGDATA>Tjs=</CONFIGDATA>
  <ROLES_OVERRIDES>
  </ROLES_OVERRIDES>
  <ROLES_ASSIGNMENTS>
  </ROLES_ASSIGNMENTS>
</BLOCK>
In 2.3, I see:
<block id="10" contextid="278" version="2012061700">
  <blockname>participants</blockname>
  <parentcontextid>16</parentcontextid>
  <showinsubcontexts>0</showinsubcontexts>
  <pagetypepattern>course-view-*</pagetypepattern>
  <subpagepattern>$@NULL@$</subpagepattern>
  <defaultregion>side-pre</defaultregion>
  <defaultweight>2</defaultweight>
  <configdata></configdata>
  <block_positions>
    <block_position id="1">
      <contextid>16</contextid>
      <pagetype>course-view-topics</pagetype>
      <subpage></subpage>
      <visible>1</visible>
      <region>side-post</region>
      <weight>1</weight>
    </block_position>
  </block_positions>
</block>
I need to analyze this data and structures and determine how I migrate the old to the new.

The easy ones (I believe) should be able to transfer "as is":
  • ID => id
  • NAME => blockname
  • CONFIGDATA => configdata
The others will require some transformation processing. I'll look at each one.

contextid - This would be the context instance id value of the block on the system it was backed up from. Since this is not kept in 1.9 backups, I'll need to create one. This is a common problem with all 1.9 converters, so there should be examples I can borrow.

version - This would be the version string of the block when it was backed up. This is not in the 1.9 backup information. I'll have to find what I can use in its place, possibly leaving it blank.

parentcontextid - This is likely the context instance id of the context the block is installed on. This is not part of the 1.9 backup, but the page id (PAGEID) is. I may be able to use it, since what is probably most important is consistency for any block on that page.

showinsubcontexts - This is a new feature of blocks previously not available in Moodle 1.9. I should be able to just default this to zero.

pagetypepattern - This is a new setting for blocks in M2. This looks to be similar to PAGETYPE, but is a setting that impacts what pages it will displayed on. This will require a bit of investigation.

subpagepattern - This is a new feature of blocks previously not available in Moodle 1.9. It looks like I should be able to default this to $@NULL@$.

defaultregion, defaultweight - These are other new settings for M2 and can probably be set to default values.

block_positions - The block positions structure's elements pagetype, visible, region and weight will be based on the PAGETYPE, VISIBLE, POSITION, and WEIGHT elements but will require some transforms. The other elements in there, id, contextid and subpage will require more investigation.

So, I have some idea of what I need to do with the block data. Now I need to plan for where that will happen (so I can play with code and see what I need to do).

There is already a moodle1_block_handler class defined in the "handlerlib.php" file. This has been created primarily to be extended by any block that needs it. Recall from my previous post, that all handlers get registered for use in the moodle1_handlers_factory::get_handlers() function. In this function, any block that provides a moodle1_block_handler instance also gets registered. I need to also register a handler for all blocks to do the generic work associated with every block.I'm thinking, this should all be part of the moodle1_block_handler class, such that every block is managed by functions in this class. And that class should be responsible for calling any specific block backup code provided by those blocks. I'm going to try designing this approach to see if it makes sense.

After I've bounced this idea of some of the developers in the tracker, I'll come back to this.