2. Literature Review

MSc Report - Discoverability of Open Educational Resources - Kelly TerrellIn this section we review the literature on a number of relevant areas for this study, including how OERs differ from existing educational resources accessible online, how they are being released and the ways in which they may be used, as well as our understanding of how people approach the task of resource discovery and the technology choices available to improve discoverability.

2.1 Distinguishing OERs from the rest

The practice of educational resources being shared online by their creator is not a new one. For many years individuals have used their own webpages or other online sharing spaces to share their lecture presentations, tutorial notes and exercises, often with their current cohort of students in mind or indeed to share with their own communities of practice. A quick search on Google for lecture notes under the .ac.uk domain presents a range of examples, one such example comes from the Vallance chemistry group2 at the University of Oxford who have posted a set of chemistry lecture notes and tutorial problem sheets on its website intended for their taught and research students. Another from the University of Strathclyde computer centre3 is from an academic who no longer works at the university, yet the institution has retained this public site acknowledging that the information may still be of value to student or external readers. In many cases posting content online in this way is simply driven by convenience or personal preference over an institutional learning management systems (LMS) which is often less flexible in appearance and applies stricter access controls. By posting such materials online they may then be found and used by anyone – be that the primary audience or through an unpredictable pattern of online networking, sharing and discovery by anyone else around the world. The question therefore follows, are OERs any different from these resources and if so, how?

Such resources are no doubt useful to the few who do find them but with this occurring as a side-effect rather than explicit intent or desire by the creators this would still seem to be some way from meeting the aim and objectives which drive the open educational resources movement. As well as this differentiation in intent, what it means to be open may be contended as the key difference however the term itself is somewhat vague to make this a clear distinction. If we dissect a number of definitions of OERs (as shown in Table 1) although not exhaustive, a theme of ensuring appropriate permissions and rights are granted to the end user recurs through them all.

UNESCO “… open provision of educational resources … for consultation, use and
adaption by a community of users for non-commercial purposes”
OECD “… materials offered freely and openly for educators, students, and
self-learners to use and reuse for teaching, learning and research”
The William and Flora Hewlett Foundation4 “… resources that reside in the public domain or have been released
under intellectual property license that permits their free use or repurposing
by others”
Creative Commons5 “… educational resources that are freely available for use, reuse,
adaptation and sharing”
OER Commons6 “… teaching and learning materials that you may freely use and reuse
without charge …”
Table 1: Highlights of OER definitions references to openness

It is a common misconception that anything found on the internet is free and that without a copyright notice you may take a copy of what you find and use it as you wish7. Though the topic of copyright is too complex to explore in this review, for OERs there is an important distinction to be made between something being free to access because it online, such as the lecture notes from the Vallance chemistry group and something which has a clear declaration of how it may be taken and re-used without charge.

Though no single OER definition is regarded as authoritative, all reflect the expectation that resources will provide users with permission to do more than simply consume them. Having the consent to change, improve, extend and share appears as a fundamental quality. Wiley (2010) summarises these rights as the 4 R’s framework (see Table 2) which he says operationalise the concept of openness, reflecting acts of generosity, sharing and giving.

Reuse The right to reuse the content in its unaltered/verbatim form, for
example taking a backup copy
Revise The right to adapt, adjust, modify, or alter the content itself, for
example translating it to another language
Remix The right to combine the original or revised content with other
content creating something new
Redistribute The right to share copies of the original content, the revisions, or the
remixes with others
Table 2: The 4 R’s framework. Wiley (2010)

In order to make this declaration of permissions for an educational resource clear to users, the common practice is to apply a Creative Commons license. The Creative Commons organisation was founded in 2001 and developed licenses for education and other sectors to minimise the legal, technical and social barriers to sharing. Inspired by the Free Software Foundation’s GNU General Public License a set of licenses are now available to allow certain uses, under certain conditions or to dedicate a resource for public use8.

2.2 Releasing OERs for a global audience

In much of the literature the mission of OERs is portrayed as the removal of barriers and re-balancing the dominance of elite education institutions to provide fairer and greater access to publicly funded educational content. The implication being that this freedom will lead us to a somewhat utopian existence, where anyone around the world wishing to access high quality education may do so freely and easily leading to greater social and economic progression (Downes 2011, Lane 2008, Hylen 2006 and UNESCO 2002). Rather than a simplistic open vs closed status, Hilton et al (2010) referred to openness as being a rich multidimensional construct which he reflected from a perspective of permissions and discoverability though many more aspects have the potential to influence this. Can a resource be truly open if it cannot be discovered? Others have argued that if it cannot be found it might as well be closed (Brown 2008, Dholakia et al 2006 in Hilton et al 2010, p4) highlighting the importance of this aspect for OERs. The focus of most literature and OER definitions typically imply teachers, learners and researchers as distinct user groups though there is little hard evidence so far on who is actually using OERs and how to establish whether this is fact. If discoverability is a dimension of openness we must understand how these diverse groups and users within them will interact with OERs and identify reliable methods of release which will reach them. This however leads us to a paradox which has an uncertain future at this time, as to be able to understand who is using OERs and for what purpose would require the user to make themselves known to the provider in some form, perhaps through registration. Downes (2007) highlights however that this simple act is a barrier in itself and questions whether a resource can still therefore be considered ‘open’. Though registration may not lead to an exchange of money it does lead to payment in the form of access to user’s personal data including activity tracking data, an incredibly valuable commodity in the online world. For some users, concern about how this data may be used may prevent them from continuing.

2.2.1 Opening existing content
Through initiatives such as MITs open courseware, the Open University’s OpenLearn and a number of projects from the JISC/HEA OER programme, pre-existing content used in on-campus delivery was released with open licenses attached. This of course makes sense for an institution which has already spent many teaching hours creating and assessing the quality of their resources through delivery and there is no stipulation that OERs must be created from scratch. Questions of how reusable they are beyond their original context and by whom remains unclear. In a book chapter on opening access to education, McMartin (2008) claims this approach only serves to reinforce a system where knowledge generation occurs within the institutions with the use of that knowledge occurring outside the ‘walls’ and questions whether reliance on existing closed systems can genuinely provide the revolutionary open alternative or simply strengthens those institutions dominance?

2.2.2 Community based OERs
In describing a project for OER development Thille (2008) described that the revolutionary potential of openness will come from the collaborations between a community of experts such as learning designers, software designers and faculty content experts from around the globe where they can work together to develop courses and conduct studies designed to provide feedback to improve the teaching and learning. Brown and Adler (2008) make the compelling argument in their paper Minds of Fire to go a step further that collaboration between learners and practitioners enables individuals to learn to be through participation, rather than simply learn about through access alone. By enabling learners to be an active part of a community allows them to engage with existing practitioners to form their identity as well as master the content of the field. By being exposed to the exchange of ideas and development of OERs learners are able to situate their learning of the subject content into activities and interactions with professionals. Brown and Adler highlight how these principles are robustly exemplified by the open source software communities, where newcomers can work on simple non-critical developments and demonstrate their ability to contribute, with those most proficient invited to join an inner circle of experts working on critical developments.

For OERs this type of community model and shared ownership may also present the opportunity not just for finished content to be released but for drafts or rough and ready content to be made available to allow adaptation and revision to reflect different cultures and languages, helping to remove these significant barriers for the nonwestern parts of the world where they are most needed (Richter and Mcpherson 2012).

To enable teachers to adapt and repurpose resources will also necessitate access to individual parts of OERs as well as the whole. In order to be able to do so the level of granularity, the size and format of individual items which can be referenced, found and used becomes vital. Initially with MITs open courseware a lot of resources were released as .pdf, Atkins et al (2007) posits that such approaches need to be reconsidered more closely given .pdf formats limits the reuse and recommends that OER collections migrate to a richer set of formats such as XML, HTML, PDF as well as embedded multimedia objects and access to sub-objects.

2.2.3 Support structures for OER
In his review of the OER movement Knox (2013) found little evidence of the recognition of the continued role of teaching with OERs with most emphasis placed upon self-directed learning and by the absence of structure implied individuals were expected to be able to manage their own learning. This view is perhaps more aligned to the standalone OERs (presentations, images, audio, videos) than those presented as collections. In a government strategy report for student success in UK higher education, the importance of an environment in which learners feel they belong and can have meaningful interactions with staff and other students are cited as key evidence of retention and success (Department for Business, Innovation and Skills 2014). If we are expecting OERs to satisfy a global demand for education it would seem optimistic to believe that many learners, particularly from non-traditional backgrounds, would have the confidence, educational maturity or intrinsic motivations required to succeed without support in some form. Rather than an ideology of OER being revolutionary on its own, it will be vital they are treated as a component of the process along with learning supports, validation and recognition of progression (D’Antoni 2009).

2.2.4 Massive Open Online Courses (MOOCs)
The emergence of MOOCs is one of the most recent developments on the OER timeline providing an online learning environment which has many similarities to traditional distance learning courses but with no cost to the learner, unlimited student numbers and a peer based approach to support (Kernohan and Thomas 2012). This has led to some of the most feverish explorations so far under the OER umbrella to reach large audiences and has resulted in a great deal of hype from the media which Bayne and Ross summarised as tending to:

Emphasise [MOOCs] as radically transformative, unprecedentedly different modes of higher education teaching that would revolutionise the sector. (Bayne and Ross 2014)

Charged debates are ongoing on MOOCs with themes around provider motivations, completion rates and the degree of openness of some commercial platforms. In a review of the strategy for the William and Flora Hewlett Foundation’s Education Programme, MOOCs have been identified as a risk to mainstream adoption of OER due to the interpretation of ‘open’ resulting in free access but with very restrictive copyrights on the content itself. Progression from this state to one where all the associated materials are released under open license is seen as unlocking huge potential for large-scale mass customisations and adaptions (The William and Flora Hewlett Foundation 2013).

With this in mind, data on the user population of existing MOOCs is only just beginning to emerge to understand who is actually engaging with these developments. In 2013, the University of Edinburgh reported on survey data associated with six MOOCs they had made available and were surprised to find a very high proportion of respondents were already highly educated. 40% had an existing postgraduate qualification and a further 30% had an undergraduate qualification despite five of the six MOOCs being offered at an undergraduate level. Furthermore a third of respondents were identified as being part of the education sector already – 16.8% in teaching or education roles, 14.8% studying full-time at college or university. With the intent of reaching new audiences to provide access to education for learners regardless of qualifications and geographical region, this initial data highlighted that to target specific sub-groups and demographic profiles will require a greater understanding and targeted consideration of how this can be achieved (University of Edinburgh 2013).

2.3 Discoverability

Discoverability refers to the quality of being discoverable, to show the presence of something hidden or otherwise difficult to see and to make it known9. If OERs cannot be discovered, from a practical perspective they may as well be closed or not exist (Hilton et al 2010). The vast volume of OERs which exists in and amongst other content on the web makes the necessity for a thorough understanding of online discovery a critical success factor for the continuation and sustainability of the field.

In his literature review of OERs and declaration that discoverability is a significant unresolved issue, Wiley et al (2014) does not indicate the scale of the problem, whether this is widespread or affecting particular types of resources, models of release, users or regions. Therefore we are left to make assumptions on this. The activity areas he refers to reflect a continuation of technical efforts to improve discoverability of pre-OER Learning Objects through the improvements to metadata and specifications to fit with search engines. They suggest that being able to search by additional descriptive data such as a license or learning outcome will help make OERs easier to find. Though this may be true for those familiar with this information, it may be less useful for unfamiliar users.

In a reflection of the infancy of this field there appears to be a lack of large scale evidence on discoverability, with most articles focusing on the development of new technology standards and services. Less coverage appears to exist exploring the human side of OER or more generic online discovery in order to understand what people are searching for and what factors influence their choices. By not understanding both sides we risk disproportionate attention being placed on areas which may have little impact.

2.3.1 Online user behaviours
A recent report from the JISC innovation project Spotlight on the digital which ran from 2013 – 2014 in collaboration with Research Libraries UK and the Society of College, National and University Libraries conducted an assessment of 217 digital collections (though only 20% were open) to gain a better understanding of their discoverability. The project also commissioned a literature review to understand users’ online behaviour in resource discovery to underpin their recommendations. The summary of their key findings for discovery behaviours are shown in Table 3.

User Information discovery behaviours
Researchers Google used as a starting point, visiting known sources directly (e-journal databases, resource collection site), following bibliographic references and following recommendations from colleagues and peers via email lists, social networks, RSS feeds and other notification methods. Only a few were found to utilise tools such as Facebook or Twitter for discovery.
Teachers No evidence could be found from the review on teacher behaviour, however the theory that teachers will follow a similar path to researchers was proposed but further studies would be necessary to establish whether this was so
Students Google is frequently a starting point with many going no further than the first page of results, Wikipedia is frequently followed and used when appearing in top of Google results, Google Scholar and library sources are used for locating articles, Google Books for finding book previews, rather than full volumes, YouTube and Flickr for resources and recommendations from lecturers (reading lists), peers, classmates and friends and family outside of the education environment
Table 3: Summary of online information discovery behaviours from Spotlight from the digital project

The literature selected for the JISC Spotlight project focused on researchers, teachers and students. No evidence was found on teacher behaviours and it did not include coverage of more generic online discovery behaviours which may provide valuable insights, therefore this bias must be factored before assuming all users will have similar approaches.

Through interviews conducted as part of the JISC/HEA OER Phase 2 Impact Study a small group of teachers were asked about their approaches to finding OERs. Most interviewees confirmed they used Google searches for materials but in one highlighted case, a clear distinction was made in locating general information via Google to its suitability to return genuine OERs. University and subject specific sites were visited as potential sources of OERs, with only two admitting to regularly visiting OER repositories – perhaps reflecting the importance of trust and familiarity for teachers. Most interviewees reported belonging to some kind of OER community which was a vital factor in obtaining new resources (JISC 2011). Due to the small number of participants (9 individuals) and having some experience of OER being the basis of their selection, little can be drawn from this and generalised at this stage to shape our global understanding of teacher characteristics for OER discovery.

The real challenge, as with many aspects of OER, is being able to accurately capture data on a group of unknown users who are predicted to benefit from OERs around the world, all of whom are likely to seek resources very differently. Until we have more information, some assumptions will have to be made on the basis of what we do know but this bias and limitation will always have to be in mind.

2.3.2 Technology choices
When it comes to putting content online there are a vast number of choices with varying features from managing the content to dissemination and optimising discoverability. Few solutions will offer a comprehensive one size fits all and localised circumstances such as available resource, skills and expertise, familiarity and even getting approval from institutional IT departments and others are likely to heavily influence the final choices made. This managerial and policy driven influence on decision making is however leaving many unable to embrace emerging innovations, which risks leaving education unable to adapt to the rapid pace of change (Laurillard 2008).

Wiley et al (2014) highlights search engine optimisation (SEO) tactics, repositories and social networking features such as tagging, ratings, commenting and recommendation systems which will be explored in turn.

Digital Repositories
A digital repository is a software platform for managing and storing digital content and facilitates access for use therefore deriving maximum value from the content10. They can vary widely in the content they hold and through the Open Access research agenda a number of institutions in the UK have some form of repository in place, though the purposes of hosting research compared to OERs in repositories varies significantly.

Wiley et al (2014) highlighted that quality of OERs was an important aspect of discoverability. With so many results being returned by search engines, users need to have mechanisms to distinguish the high quality resources from the lower. Browne et al (2010) in their paper on the challenges of OER to academic practice includes a quote from an academic who said ‘an OER repository definitely does give an impression of the institution and the staff (team) that produces it’. Though this quotation itself may be taken as supporting the earlier claim of reinforcing the institutions as the only providers, there may also be value in the representation of proven high quality providers of education in a world where Wiley et al describes overcoming the perception of quality of resources available for free as one of the hardest challenges for OER. Given a user who is browsing their search results will often make their initial decision on surface data that is displayed on the results screen, recognising an institutions web address may well influence their decision. For subject specific repositories, associated communities of practice are likely to trust a source from a provider that is known and that they perhaps contribute to in other ways.

Once a user has been drawn into visiting one resource hosted within a repository, by its very nature of holding a large amount of related content they will have the opportunity to discover more.

Recommendations and Ratings
Kalz et al (2008) described an approach called Wayfinding which they based on their view that the primary users of OERs being self-directed learners, supporting Knox’s criticism of the field. They suggest such learners require orientation to choose the best suited resources for their learning to reach proximal development. As part of a learner network of registered users, ratings are established based upon the history of all networked users and items which exist and their usage and similarities to one another. This data is then mined to tell a learner where they stand in the curriculum and based on their rating recommends what they should choose next to reach their goal. Kalz et al do highlight shortcomings of this approach in that new users who have not built up sufficient ratings may receive less accurate recommendations. As new users who are likely to be in most need of guidance and direction, this may risk users starting down the wrong or inappropriate development path which without intervention would be difficult for them the identify or rectify. Though this tracking approach may seem more akin to a learning management system, parallels can be drawn to the ratings and recommendation features found in many web 2.0 sites such as YouTube, Flickr and iTunes where the registered community are able to rate and comment on resources and recommendations are made based upon your browsing history.

Search Engine Optimisation
Search Engine Optimisation (SEO) is the process of improving the visibility of resources in search results. The paper from Hawksey et al (2013) to promote approaches to improve OER discovery reflected this tactic across all which were included – SEO optimised platforms for collections, human readable metadata alongside resources, embedding machine readable metadata in the body of resources, and the experimentation of ‘paradata’11. Paradata has emerged as a term used to describe data about the user and their interactions with learning content being used, reused, adapted, tweeted, shared and interacted with via social media. Hawksey et al confirms that though this is a still a concept at this stage this has been adopted and pursued as part of the Learning Registry Metadata Initiative (LRMI) in order to capture this data and is now being explored by Google.

The coverage of search engine optimisation (SEO) across the literature on discovery would seem to reflect the expectation that most people will use a major search engine such as Google as their main channel to locate information online therefore making this the focus for future efforts in this area (Thomas et al 2012). In a blog from the OpenSpires project at the University of Oxford, the opinion shared was ‘easy to discover of course means the content is as high up in Google ranking as possible’12. Results from the JISC Spotlight project however shine an interesting light on this position. In their assessment of 217 digital collections, only searches for entire collections based upon title were accurate with 95% being found however only 56% of searches for individual items returned results. Searches based on simple keywords were also surprising with 64% of collections and 44% of individual items being found (JISC 2014b). This therefore indicates that if search engines are the primary mechanism for discovering content more than half of the items which exist will never be found. The report does not offer reasons why this is, though the manner of sharing, method of exposure and ability to access individual items that make up a collection are likely to be a contributing factors. For OER collections it is incredibly important to optimise these individual items for exposure so they may be found, used and adapted by others.

Returning to the definition of discoverability, technology choices need to show the presence of OERs and to make them known. This includes the parts as well as the whole. Decisions on appropriate technologies will naturally be made at the start of project or initiative and without sustained investment may remain unchanged unless there is a significant justified reason, such as institutional changes, to do so. With the rapid pace of change and to respond as we increase our understanding of user behaviours and expectations it will be necessary to keep abreast of new technologies and interventions so resources do not slowly slide into oblivion after they are released (JISC 2014a).

3. Research Design and Methodology >>


2 Valance chemistry group http://vallance.chem.ox.ac.uk/CVteaching.html
3 C Programming – Steve Holmes http://www2.its.strath.ac.uk/courses/c/
4 The William and Flora Hewlett Foundation http://www.hewlett.org/programs/education/open-educational-resources
5 Creative Commons http://wiki.creativecommons.org/What_is_OER
6 OER Commons https://www.oercommons.org/about
7 10 common misconceptions about the public domain http://www.publicdomainsherpa.com/10-misconceptions-about-the-public-domain.html
8 Creative Commons http://creativecommons.org/about/history
9 ‘Discoverable’ http://www.merriam-webster.com/dictionary/discoverable
10 Repositories Support Project: What is a repository? http://www.rsp.ac.uk/start/before-youstart/what-is-a-repository/
11 https://wiki.ucar.edu/display/nsdldocs/Paradata 12 http://blogs.it.ox.ac.uk/openspires/2011/02/04/oer-discoverability-top-tips-for-searchengine- optimisation-seo/