Blog/20131106 Connecting YouTube and MediaWiki

From Bjoern Hassler's website
Jump to: navigation, search

More from B's blog:

Some older entries are here.

Connecting YouTube and MediaWiki

A day at the rock face.

One of the points of OpenCast (and the Steeple project) were to develop simple ways of posting one piece of media out to different "media channels". That is to say, to post a single video onto a central (university-wide) repository, perhaps onto a departmental webpage, as well as to YouTube and iTunesU. (These days we might want to consider social media sites as well, but often the video is just embedded there from YouTube.) Unfortunately we aren't using OpenCast as institutional solution, nor does our institutional solution provide those features, so we needed to find our own solutions.

So why do we want to use YouTube in the first place? YouTube provides

  • a good set of different bandwidths (see YouTube bitrates) including both high defintion and low bitrate video;
  • often there is a national YouTube cache (in the form of a Google-provided "black box" situated at an internet exchange point) which means that there's fast access to YouTube over wired connections (effectively a Content Delivery Network);
  • other useful features, like subtitles.

So it makes a good video repository, and solves a bunch of potential problems for us.

Our scenario is then to post video both onto our OER4Schools resource, as well as to our YouTube site, i.e. to both

  1. our OER4Schools resource (hosted in MediaWiki);
  2. and our OER4Schools YouTube site

This is how the connection between those works in practice: Consider this video on YouTube: embedded right here with the MediaWiki YouTube widget

It's embedded in the same way on this page [[Video/Eness_vertebrates_6.mp4]] on the OER4Schools resource. Elsewhere on that resource, rather than embedding the video using the widget every time, we just embed the video using the widget once (i.e. on [[Video/Eness_vertebrates_6.mp4]]), and for any other occurrences we use 'transclusion', i.e. in wiki syntax


This means that we only need to maintain a single point of contact with YouTube. We'll come back to this. For now, note that the video on the wiki links to YouTube, and the YouTube video links back to the wiki. The wiki also links to our 'download manager', under the link "local play / download options". The links in the wiki pages are embedded as "/downloadmanger" (without a host name), so for offline versions of the wiki, these will point to a local script, that can pick up copies from a local server. That's exactly how the OER4Schools programme (at Chalimbana Basic School) uses videos at the moment. There's an offline copy of the wiki available on a local server, and the video is played back from that server. So that's all fine, and as it should be.

However, we still need to connect YouTube with our MediaWiki, and that's where things need to be done manually. That's not great, and the Steeple project has shown that beyond a certain number of videos, this becomes unfeasible. What we do is to

  • first upload a video to YouTube;
  • then create a wiki page (with whatever metadata we have);
  • then use that wiki page (via transclusion) elsewhere (and in the context of our resource);
  • we also upload a copy of the video onto the our wiki server, so that it can be picked up by the download manager (for use in offline versions).

So there are quite a few manual steps in this. One problem is that the title, description, and playlist membership needs to be set up both on the wiki and YouTube.

Ultimately, it would be desirable if there was an automated process

  • that started with a video file, with some initial metadata (e.g. on dropbox);
  • this would then be uploaded to YouTube;
  • a wiki page is created from the metadata, linking to YouTube and vice versa;
  • the different versions of the video are downloaded from YouTube to our own server (for offline use);
  • and changes to the wiki page are kept in sync with YouTube.

The original file should be kept as an archive. Through Steeple, we always advocated keeping the original files, which in our workflow might be a ProRes file. That's really not part of the above automated workflow, and it may be too expensive to get cloud storage for those files to integrate this easily. Our local institutional DSpace repository may be possible, with local disks as temporary storage before upload to DSpace.

However, we don't have that automated workflow at the moment. Lack of an automated workflow means keeping data and metadata manually in sync, and that is not straight forward. Hence the day at the rock face.

Over time, some of our data had gone out of sync, and so we needed to fix that. In essence, I wrote a script that fetches our video metadata using the YouTube API, then fetches the video data from the wiki (which is maintained using Semantic MediaWiki, and can thuse be exported), and then compare the two. Importantly, metadata is then copied across from the wiki to YouTube, again using the API. I've done this in a fairly ad-hoc fashion, but it would be much nicer if we had some developer time for this to actually solve the problem in a robust way. For now, we're fixing and checking certain things automatically and certain things manually, and that's going to work for our 200 and something videos. With the next large batch, we're going to need more automation, and need to find a better way forward.

By the way, note that the above video also has subtitles, see 20131104 New OER4Schools videos on YouTube.

Notes: Related articles, discussing similar problems over five years ago(!): Multimedia Encoding Workflow, Multiformat Media Delivery, Video Distribution.

2013-11-06 | Leave a comment | Back to blog