4. Discussion and Findings

MSc Report - Discoverability of Open Educational Resources - Kelly TerrellIn total, 35 UKOER projects from the JISC/HEA OER programme were identified for inclusion in this study which were analysed with the objective of extracting evidence of approaches to Disseminating OERs and Discovering OERs which are presented in the following two sections.

4.1 Disseminating OERs

The first strand of analysis was focused on identifying the methods of sharing and dissemination implemented by projects which released OERs.

Of the 35 projects, 11 were led by HEA Subject Centres, many of which have since closed as a result of changes to funding, with the remaining 24 projects led by 21 distinct institutions across the UK. Using a narrative review approach to analysis, project reports and the platforms referenced were analysed to extract key data and methods which were implemented, which returned the following results.

4.1.1 Hosting and dissemination platforms
Hosting platforms refer to the services or software used to store and serve the actual OER content released by each project. Such platforms include repositories, content management systems and web 2.0 sharing sites such as YouTube, SlideShare and Flickr and may hold standalone items, complex objects or collections of related items to reflect the nature of the OERs being shared. Some platforms, particularly web 2.0 sharing sites, focus on holding one type or format of content, YouTube for videos, SlideShare for presentations and Flickr for images, whereas others are designed to hold a diverse collection of many different types, sizes and formats.

Across the 35 UKOER projects, as shown in Figure 1, 47% were found to be using a single platform for hosting their content and 53% distributed them across multiple platforms in an effort to increase exposure.

Figure 1: Projects using Single vs Multiple hosting platforms for OERs

The hosting platforms themselves that were used, shown in Figure 2, intentionally exclude those instances where a platform was used to hold metadata only to describe the OERs and provide a redirection link back to where it is stored. These dissemination platforms were captured separately and will be covered later.

Figure 2: Hosting Platforms for OERs

It should be noted that submission to Jorum13, the national repository service for use by UK further higher education to collect and share OERs, was mandated for all UKOER projects, which is why this platform appears so highly as a chosen platform therefore little was read in this choice. Four projects were found to be using Jorum as their primary hosting platform (OpenStaffs by Staffordshire University and three projects from Subject Centres) whose reasons included local technical difficulties preventing delivery of a self-hosted platform in the timescales to a preference for a nationally supported service. A number of projects across the programme did not submit to Jorum at all due to technical difficulties with automating submission to the system. The results shown do demonstrate a preference for management systems – particularly repositories used by 51% (18) of projects and content management systems (CMS) used by 26% (9) of projects. Though the justification for platforms choices was not consistently recorded by projects in many cases existing infrastructure, in-house experience and familiarity appeared influenced the choice. The projects OCEP from Coventry University, Unicycle from Leeds Metropolitan University, ChemistryFM from the University of Lincoln, and Learning from WOeRK from the University of Plymouth all chose to use their existing institutions repository already in place for a number of academic uses. In the case of OCEP existing content in the repository was made open by creating a new publicly accessible record once appropriate controls had been lifted.

The other standout hosting platform in use across the projects was YouTube, used by 29% (10) of projects, though little justification was evidenced in the reports for this choice being made. In the case of MMTV from the University of Westminster, which focused on the release of multimedia training videos, YouTube was referred to as ‘an obvious choice’ and this perhaps succinctly sums up the current dominance of YouTube as the place on the internet where videos are found.

As well as platforms used to host the content, a number of additional platforms were used for dissemination purposes only. These platforms were used to store metadata about resources exposing their existence from multiple access points. They then redirect the user back to the main host location if they wish to download the actual content itself. Figure 3 shows the dissemination platforms used by the UKOER projects combined with the existing hosting platforms to illustrate the entire range and degree of use.
Figure 3: Hosting & Dissemination platform of OERs combined

Excluding Jorum usage, Facebook, Twitter and the use of Blogs were the main platforms used for dissemination beyond their hosting platform. The top five platforms for hosting and dissemination by the projects were:

  1. Repository (51%)
  2. Blogs (34%)
  3. YouTube (29%)
  4. Content Management Systems (26%)
  5. Twitter (20%)

4.1.2 Platform dissemination and discoverability features
Platforms such as YouTube, SlideShare and Flickr are specifically designed with massive communities of users in mind and as such have a number of features built in to aid sharing and discovery. This includes recommendations based on browsing histories, highlights of the most popular content as well as clear categories for manual browsing. Rather than passive use of resources, YouTube allows any registered user to rate and comment on videos, save to a personal playlist and subscribe to future releases by the same provider.

Figure 4 shows the key features identified on the platforms chosen by the UKOER projects to aid discovery and encourage user interaction.

Figure 4: Tactics used to assist discoverability (excluding web 2.0 platforms)

Tactics in use included RSS feeds, SEO and optimising metadata to better describe the content all of which reflect a focus on pushing content out from the platform so it may be discovered from different access points. Tactics of using tag clouds, facilitating local searches and showing related resources to assist and encourage users to interact with the platform were also found though these still focused on passive interaction to consume the content. SEO tactics were found to have been considered by only 14% of projects which is somewhat surprising given the widespread view that search engines are the primary source for information discovery. Though software platforms such as repositories will often promote themselves as having SEO built into their very design, it was never the less surprising to find so few projects discuss this explicitly as part of their considerations for content release. This may reflect an assumption by projects that the software is handling this requirement, though more detailed investigation would be required to establish if this is fact.

The model and methods of dissemination demonstrated on social media and networking platforms such as Twitter, Facebook, YouTube, Google+, Pinterest and LinkedIn which comprise of millions of users14 presents a huge opportunity for OER to reach a globally distributed audience. The ability to quickly share content and information with others is fundamental to their success and one way this has been facilitated is through the use of an add-on tool referred to as a Share this widget15 allowing the person viewing content to quickly and conveniently share what they have found on selected social networks in just a couple of clicks.

Figure 5: Projects utilising a Share This widgetAcross the 35 projects, 49% were found to be utilising such a sharing mechanism. This is of particular relevance given the prominence of repositories, content management systems and blog-based websites leaving visitors with no quick way of disseminating OERs they find valuable any further. Given that social networks and recommendations from peers and colleagues features in the literature on user behaviour of resource discovery this would seem a significant missed opportunity for the 51% of projects and platforms not utilising this simple add-on feature.

4.1.3 Target audience

The target audience for OERs released by each project, as shown in Figure 6, identified that most projects were aiming their resources at academics (89%) and enrolled students (66%). Enrolled students were considered to be students enrolled at the institution releasing the OERs or within another institutions studying the same or similar subject and therefore already part of education system. Initially projects which identified their audience as being outside of these institutional boundaries were categorised as having a target audience of ‘anyone’. It became apparent that many projects had released OERs to target potential students with taster content to encourage them to enrol formally. MMTV was one such project who declared their primary motivation was to increase recruitment numbers on a course that had been suffering from reduced interest for a number of years. Further examination identified 23% of projects releasing OERs were aimed at potential future students and 34% released them explicitly for anyone in the world to use.

Figure 6: Target audience for OERs

In the majority of cases, projects released OERs declaring multiple target audiences, surprisingly though 10 projects (29%) declared the audience of their OERs as only academic staff and a further 10 projects declared they were for staff and enrolled students indicating use by other audiences would be as a side-effect rather than explicit intent. The projects, Your Business Future from Southampton Solent University, and Finding a Voice through Open Resources from the Language Linguistics & Area Studies Subject Centre were the only projects which declared they were releasing their OERs for use by anyone and particularly with self-directed learners in mind.

Though of course by placing the content online, as with the examples given earlier in the literature review, regardless of primary audience it may be used by anyone and in this case with creative commons license applied provides the permission for use. This may however have implications for the approaches ultimately chosen to disseminate the resources which may hinder their open discoverability for use by anyone which is ultimately the purpose of OER. This brings us back to the earlier point on the dimensions of openness by Hilton et al (2010) if resources cannot be found they may as well be closed.

Figure 7 below illustrates the hosting and dissemination platforms used by projects combined with the target audiences they declared. Though the ‘potential students’ and ‘anyone’ audience groups appear low, this simply reflects the lack of declaration by projects of these groups as a target audience.

Figure 7: Relationship between hosting and dissemination platforms and target audiences

A strong link between the platform and the occurrence of academic staff and enrolled student being declared as the target audience can be seen across all the approaches, though further analysis would be needed to determine the strength of this link and any potential impact on the other audience groups.

From this the top platforms to reach each audience in use were identified:

Anyone Potential Students Enrolled
Academic Staff
1  Repository (6)  CMS, Website (3)  Repository (11)  Repository (16)
2  SlideShare, YouTube (3)  Repository, Twitter, Facebook (2) CMS, YouTube (7)  CMS (9)
3  CMS, Website, Blog, iTunes, Scribd (2)  Virtual Learning Environment (VLE)/LMS, YouTube, iTunes, Vimeo (1)  Facebook (6)  YouTube (8)
Table 7: Top three hosting and dissemination platforms for each target audience group

The use of repositories and YouTube figured strongly across all projects and target audiences, though it was somewhat surprising not too see more widespread utilisation of social networking platforms (Twitter and Facebook). In particular the projects Your Business Future and Finding a Voice through Open Resources made no use of social media or networking sites in their hosting or dissemination platform choice, instead utilising an Open VLE and repository as their sole platform respectively and neither applied a Share This mechanism to encourage sharing by users with a wider community.

4.1.4 Current status

Figure 8: Current status of OER hosting platforms (exc. web 2.0 platforms)A theme evident throughout the examination of this topic has been that the field does not stand still. Technologies change, new social networks appear and search engine algorithms increase in their complexity. One of the key recommendations of the JISC Spotlight project was that content must be looked after carefully to adapt to these changes and prevent content from sliding in to oblivion (JISC 2014a).

Of the 35 projects, the platforms and collections created were examined to determine if open content was still being added or promoted in any way. As shown in Figure 8, nearly half of the projects had OERs on platforms that were no longer active or being managed, rather they had become a static collection of resources for reference.

4.2 Discovering OERs

To apply greater meaning to these figures we return to the literature on discoverability tools, user behaviours towards information discovery and the practices developed in the field to address this challenge.

The use of repositories figures heavily across the UKOER projects. In his analysis of open education resource repositories Rory McGreal describes them as:

A digital database that houses learning content, applications and tools such as texts, papers, video, audio records, multimedia applications and social networking tools. (McGreal 2011)

This definition could also be applied more broadly to content management systems or any hosting platform that has been utilised in this way across the UKOER projects. Many repositories used by UKOER were already established at the lead institution, satisfying multiple purposes for holding content such as research outputs as well as closed access teaching materials. The diversity of the content, purpose and target audience however must be recognised if platforms are to be shared in this way.

Atenas and Havemann (2014) identify a number of quality indicators for a repository platform hosting OERs which provides a valuable comparison to approaches taken by the UKOER projects. Though Atenas and Havemann describe 10 quality indicators only four have been focused on as being relevant for the topic of discoverability of educational resources.

1) Showcasing resources in the repository can help to expose materials and is a method in evidence on many social media platforms, enabling the prominent display of resources considered high quality by others. YouTube offers this by promoting the most popular videos based on viewing numbers. Though there was some evidence in the analysis of projects promoting collections, this was enabled using a manual approach of posting a link on a webpage and was not driven by the dynamic generation of data based on user interactions.

2) Evaluation tools to allow users to rate a resource on some level demonstrates trust in the community and can help engage users to support quality control. This however leads to a tension between this social media practice and those of academia. Whereas in other markets the end-user is viewed as the judge of quality of value, in academia the profession is built on authorities making the knowledge and learners are typically deemed unable to judge the profession without guidance (McMartin 2008). McGreal (2011) describes this shift as work in progress requiring a combination of cultural change as well as better technologies. By the UKOER projects making use of social media platforms such as YouTube, SlideShare and Flickr which have such user evaluation tools built in would indicate there was a degree of a comfort in receiving feedback in this form, though the scale cannot be defined. There was however no other platform found to be utilising such mechanisms to allow users, whether academics, students or self-directed learners, to rate the resources in any way. This would however be an incredibly valuable add-on for repositories particularly to build up associated metadata enabling identification and promotion of ‘popular’ resources as well as enabling a feedback loop.

With a reliance on search engines evident across the literature, providers of OER and the platform providers have a responsibility to generate and expose relevant and meaningful data to describe the resources in a consistent manner. 3) Keywords or context descriptions generated by the user of the resource can enhance the opportunities for others to successfully retrieve and evaluate the usefulness. One of the criticisms of OER reuse is that the context of isolated items particularly is unclear and self-directed learners do not necessarily have the capacity to identify how resources should be used. No projects in the analysis displayed methods of adding information to resources, rather they reflected the finished product that were available to use after collaboration had taken place internally.

Kortemeyer (2013) in his article reflecting on the past 10 years since the launch of MIT’s open courseware suggests OER have yet to noticeably disrupt the current model of education, and posits one contributing factor is the disconnected nature of content which has been released making them much harder to find. This position is strengthened by the findings of the JISC Spotlight project which identified less than half of individual items in structured collections were being returned by major search engines.  Kortemeyer, though not backed by research, proposes the production of dynamic metadata by crowdsourcing, by gathering usage based data as educators who release or use OERs identify the sequence of resources and the course they have been used in allowing the system to make recommendations based on educator’s choices. This draws many similarities to the Wayfinder service prototype described by Kalz et al (2008) enabling the ability to share a suggested learning path with the self-directed learner or equally to provide clarity to another educator on the context of its use alongside other resources so adaptations may be more easily identified for local use. In some of the UKOER projects, such as Digital Futures in Education led by Sheffield Hallam University which produced an open text book and Great Writers Inspire from the University of Oxford, structure surrounding the resources were integral to the platform. A visual representation of book chapters for the open text book and playlists to reflect a sequential order of audio files helped convey a relationship between items as well as the wider context of their use. In many others cases particularly with the use of repositories, standalone resources provided no such context.

The final quality indicator offered by Atenas and Havemann of direct relevance to discoverability is 4) Inclusion of social media tools for sharing resources, which 49% of the UKOER projects were found to have implemented through the use of a Share This widget. The notion of sharing is an important theme to remember, and that this act should not stop when the initial provider releases the content. This should lead to the possibility of dissemination much more widely and quickly so more and more people can access resources to solve their problems (Hylen 2006). The power of social networks, identified in literature review as an information discovery source for learners and academic professionals, should not be underestimated in terms of their potential influence on discoverability. Looking at YouTube alone, the community of users leads to more than 1 billion unique visitors each month and over 6 billion of hours of video being watched each month16. The rise of networking sites such as Facebook and Twitter has accelerated this growth with over 700 YouTube videos shared on Twitter each minute and 100 million people taking social actions (to like, share and comment) every week17. The heart of this type of operation lies in a combination of a wide range of content and engagement of its user community. The community provides the content they have created, rate the content they have viewed, comment on the content they like or dislike, save the content for repeat use and share the content they like or believe others will like.

The approaches identified across the UKOER projects would seem to demonstrate there is still some distance to travel to establish this type of collaborative community for education although the volume and diversity in evidence across the projects demonstrates a clear desire by institutions to explore the possibilities.

5. Conclusions >>

13 http://www.jorum.ac.uk/about-us/
14 Most popular social networking sites (August 2014) http://www.ebizmba.com/articles/social-networking-websites
15 http://www.sharethis.com/ , http://www.addthis.com/
16 Youtube statistics http://www.youtube.com/yt/press/statistics.html
17 Youtube Infographic (March 2012): http://techwelkin.com/latest-youtube-fact-and-statisticsinfographic