HEFCE OER Review

By Lou McGill June 17, 2013
evaluation, HEFCE OER Review, OER Pilot Programme, open educational practice, SCORE, synthesis team, UKOER Phase 2, UKOER Phase 3
, , , ,

We are very pleased to launch the completed UKOER/SCORE Review report – Journeys to Open Educational Practice

wordle

McGill, L., Falconer, I., Dempster, J.A., Littlejohn, A. and Beetham, H. Journeys to Open Educational Practice: UKOER/SCORE Review Final Report. JISC, 2013  short url: http://bit.ly/HEFCEoerReview

What’s in it?

If you want to find out why people might become engaged in OER and Open educational practices (OEP) then you might like to look at the Motivations section. If you are interested in looking at the range of models and approaches adopted for OER Release then the Models page may be useful for you. If you want to know about the impact of the HEFCE funding then we have an Impact section. We have drawn together some critical factors to support OEP for those that want some tips on how to go about this themselves. We have a section that highlights tensions and challenges around OEP and the OER journeys section provides an interesting look at the wider context and how the HEFCE-funded initiatives fit into that. We also offer recommendations. If you contributed to our surveys, polls and interviews then we have a series of supplementary appendices and you can look at out methodology and evidence pages – all available from the main report page http://bit.ly/HEFCEoerReview.

We have also produced a summary briefing paper.

Who contributed to the study?

For the past three years our team (Allison Littlejohn, Isobel Falconer, Helen Beetham and Lou McGill) have been supporting evaluation and synthesising outcomes and outputs from the Jisc/HE Academy UKOER Programme. This has been quite a challenge for a programme with so many complex aspects and has resulted in a well populated wiki containing evaluation tools and support, several briefing papers, a guide to terminology and several synthesis reports.

Following this we were commissioned to carry out the UKOER/SCORE OER Review by Jisc and the Open University to provide a cumulative evaluation of both UKOER and SCORE work. Dr Jay Dempster joined the team and brought some excellent knowledge around evaluation and recent Jisc programmes.

The programme had an excellent support model from a range of bodies and individuals:

  • David Kernohan, Amber Thomas, Heather Price and Maggie Stephens from Jisc
  • Sharon Waller, Amy Robinson, Alex Fenlon from HE Academy
  • Lorna Campbell, John Robertson, Phil Barker and Martin Hawksey from JISC CETIS
  • Naomi Korn and  Alex Dawson from Web2Rights
  • JISC Legal team
  • Terry McAndrew from JISC Techdis
  • Frank Manista, Nicola Siminson and the rest of the Jorum Team
  • Chris Pegler from SCORE
  • HE Academy Subject Centres

and loads of great project teams…

The Review study would not have been possible without the work and input of all the individuals and teams mentioned above. We hope they feel it does justice to all their hard work.

The review is published initially as a series of linked wiki pages with several supplementary pages. Each report page can be downloaded as either an open document format or pdf. We anticipate that people will probably dip into it rather than read it as a linear document but also hope to publish it as an ebook aswell.

We welcome feedback and hope you enjoy reading it.

 

UKOER Phase 3 Report

By Lou McGill March 26, 2013
UKOER Phase 3
, , ,

We are very pleased to release our final Synthesis & Evaluation Report for phase 3 of the UKOER Programme.

wikipage

http://bit.ly/UKOER3SynthesisReport

It has been hard work collating the excellent work of the phase 3 projects. Their work with so many diverse stakeholders has made it very challenging to synthesise key lessons – these included private bodies (including commercial publishers), public bodies such as Skills Councils, Charities, Schools and many partners in the education sector.

The report is initially presented on our Synthesis wiki as a series of linked pages. We are also working on an e-pub version so that you can annotate and personalise the content (coming soon).

Due to the nature of the work it is very detailed and comprehensive and links back to project evidence where appropriate. For those who don’t have time to read it we would recommend that you look at the Summary of key lessons page.

OEP and bounded communities

By Lou McGill March 11, 2013
evaluation, OER Pilot Programme, open educational practice
, , , ,

Open Educational Practice: are communities helpful environments for changing professional practice?

Allison Littlejohn, Lou McGill, Isobel Falconer, Helen Beetham

For Open Education week at 11am (GMT) on Thursday at Lou McGill, David Kernohan and Allison Littlejohn will present some of the key findings from the UKOER programme ‘What you can learn from the UKOER experience‘.  The programme anticipated that widespread involvement of faculty and support staff  would bring about a sustainable change in culture from focusing on content ownership, to focusing on open sharing; and that building a critical mass of OER would bring about sustainable change in practices of reuse and re-purposing. The lessons learned from evaluation and synthesis of the programme are available from  http://bit.ly/oerevalsynth

One set of key findings was around the role of communities in the release of Open Educational Resources (OER).  How professional practice is transformed to support activities underpinning the release of OER, sometimes called open educational practice (OEP), is not well understood. Communities of practice provide a positive environment for changing professional practice. Examples of communities are subject discipline communities or communities within an institution. Each community will have members with different roles (for example academics, support staff, learners), regulated by specific rules. These sorts of communities are important if the benefits of a culture of open resources, open knowledge, free sharing and peer collaboration in education are to be realised. The UKOER programme provided a context to explore these tensions and highlight the benefits and limitations of communities in transforming professional practice.

The UKOER Evaluation and Synthesis team, Allison Littlejohn, Isobel Falconer, Lou McGill and Helen Beetham, analysed the contradictions evident in OER release by UKOER project teams. We drew data from our programme-wide synthesis and evaluation (McGill et al, 2010), using project reports and focus group discussions to surface, 1) common issues, key barriers and enablers around OER release,and 2) cultural differences across the sector, detailing evidence of norms, roles, rules and reward structures that foster effective professional practice.  Analysis was through mapping the actions of project team members against an activity framework (see figure 1). In our study, the activity systems were UKOER projects where project team members (subjects) work on OER (object), transforming it into an outcome using technological and conceptual tools (Engeström, 1987 & 2005). The tool-mediated action of the project teams was mediated by rules and the broader social context of the community within which the activity takes place. Labour was divided among the community members (roles). This framework provided an analytic socio-cultural lens for understanding complex relationships across different groups.

activity traingle

Figure 1: activity framework  for a UKOER project

This analysis provided evidence that OER projects made best progress where  project team were within existing communities. Examples included subject communities, where people already sharing teaching materials. However we also found that in projects where people did not have existing, working relationships, new collaborations were difficult to initiate. For example, project teams found it difficult to convince university support staff to allow collaborators from outside their community access to institutional repositories.

A key factor within communities that helped change professional practice was trust. In many cases, when trust was not apparent, peoples’ willingness to open access to resources was reduced (for a more detailed description Falconer et al, 2013). Faculty wanted to retain control over which communities or sub-communities they opened up their resources to, preferring to release content within a closed community. Yet controlled release of resources within closed communities is antithetical to the philosophy of open access, mitigating against the successful release of OER.

In summary, while communities may encourage first steps into open practices, they sometimes seem antithetical to the basic philosophy of open release of resources. We found a contradiction between the aim of the UKOER programme to openly release OER and limited practices within some communities, resulting in release of OER within bounded communities. These contradictions present major barriers to successful OER release.

References

 Beetham, H. (2011) Reflections, blog post, https://oersynthesis.jiscinvolve.org/wp/2011/01/25/reflections/ [accessed 23/12/11]

Engeström, Y. (1987) Learning by expanding, Helsinki: Orienta-konsultit.  Available from http://lchc.ucsd.edu/mca/Paper/Engestrom/expanding/toc.htm  [accessed 23/12/11]

Engeström, Y. (2005).’ Knotworking to create collaborative intentionality capital in fluid organizational fields.’ In M. M. Beyerlein, S. T. Beyerlein, & F. A. Kennedy (Eds.), Collaborative capital: Creating intangible value (pp. 307–336). Amsterdam: Elsevier

Falconer, I, Littlejohn, A., McGill, L., and Beetham, H. (2013) ‘Motives and tensions in the release of Open Educational Resources: the JISC UKOER programme’ Draft available from https://oersynth.pbworks.com/w/page/63710786/Motives%20and%20tensions%20in%20the%20release%20of%20Open%20Educational%20Resources

JISC (2009) HEFCE/Academy/JISC Open Educational Resources Programme: Call for Projects http://www.jisc.ac.uk/media/documents/funding/2008/12/oercall.doc [accessed 18/10/2010]

McGill, L., Beetham, H., Falconer,  I., and Littlejohn., A. (2010) JISC/HE Academy OER Programme: Pilot Phase Synthesis and Evaluation Report. Available from https://oersynth.pbworks.com/w/page/29688444/Pilot%20Phase%20Synthesis%20and%20Evaluation%20Report

OER terminology

By Lou McGill March 3, 2013
OER Pilot Programme, open educational practice, synthesis team, UKOER Phase 2, UKOER Phase 3

The UKOER programme has investigated many aspects of OER release and use over the last three years and detailed discussion of this is available in both the synthesis wiki and the OER infoKit. Watch out for some other posts about synthesis reports for the latest phase (phase3) and the HEFCE OER Review, which we’ve recently been working on.

For over a year Lou McGill has been working on a guide to help people understand the terminology around OER release, use, models and benefits and open educational practice (OEP) that aims to shed a bit of light on how the different terms relate to each other. She uses a thesaurus approach to identify broader terms, related terms and narrower terms.  For each term she offers a brief definition and also points people to other sources. Click on the image below to go to the guide which is on the synthesis wiki.

terminologyguide

One of the challenges for the UKOER programme is that it covers a very wide range of issues and the OER infoKit and the UKOER synthesis and Evaluation wiki both contain loads of really useful resources developed by our team, and also collate those produced by the project teams who participated in the three year programme. So the guide also points you to relevant places on both the OER infoKit and the synthesis wiki.

This guide is informed by the work done by the synthesis team and the wider UKOER community – so acknowledgement is due to all of them and also to JISC Infonet who have funded this guide as an adjunct to the OER infoKit. If you spot something glaring missing (bearing in mind it is impossible to be comprehensive when pointing to general OER resources) please let Lou know.

A guide to definitions and terminology around OER and Open Educational Practice with links to further sources

HEFCE OER Review survey

By Lou McGill July 3, 2012
evaluation, OER Pilot Programme, UKOER Phase 2, UKOER Phase 3
, , , ,

Please fill in our survey

If you have been involved in HEFCE funded UKOER and SCORE initiatives we would like your help.

We plan to revisit some of the key lessons learnt across all strands of work, and gather input from as wide a range of stakeholders as possible. We are interested in reaching partners from all sectors, service support teams, key project personal, institutional and organisational stakeholders (whatever level of involvement you have had). We are also interested to hear from recipients of the outputs, outcomes or activities so please circulate the survey as widely as you can. We hope to draw together some interesting themes from across three years of diverse and interesting activities.  Alongside the survey we will also be carrying out some telephone or skype interviews to ask some more in depth questions of people who have been involved in the initiatives.

This is your opportunity to feed into the review and also an opportunity to tell us which aspects of open educational practice would benefit from any future funding or research activity.

Click here to take survey

The survey will be open until July 19th

 

HEFCE OER review

This survey comprises a part of a review into HEFCE funded OER activities which aims to provide a cumulative evaluation and synthesis  of all phases of UKOER and the Open University SCORE activities. We will be revisiting the findings from the pilot phase and phase 2 of UKOER and synthesising these with findings from phase three which is currently in progress. We will also be producing a synthesis of OU SCORE activities.

More information is available from our Synthesis and Evaluation wiki

 

Evaluation buddy meeting between REACTOR, Opening up a Future for Business and CORE-SET

By Lou McGill April 23, 2012
evaluation, UKOER Phase 3
, , , , , ,

One of the approaches that we have adopted to support projects with their evaluation activities is to establish pair/teams of evaluation buddies.

An evaluation buddy is basically another project who can:

  • check that evaluation plans are sensible, credible, valid, do-able, relevant to programme aims
  • provide peer review of content, interfaces, and other practical outputs from the projects
  • sharing ideas, experiences and interim outcomes of evaluation
  • swaping time and expertise, e.g. analyse data for one another (extra objectivity
  • sharing resources e.g. external consultants buddy up for e.g. workshops, critical moments (writing up etc)
  • act as a sounding board and be a neutral ear to articulate findings, thougts and any concerns about evaluation outcomes.

It is a support mechanism that can develop into long term relationships for future OER work. Of course it does not preclude the development of other useful relationships across projects. We describe a bit more about this approach in the toolkit.

We held the first of a series of meetings with these pairs/groups this week. This meeting included three projects REACTOR, Opening up a Future for Business and CORE-SET. These projects have several areas of commonality which we explored in the meeting.

Each project gave a brief overview of their project and the evaluation questions/issues that thay were looking at.

CORE-SET

Led by University of Liverpool, this project builds on the Pilot Phase project CORE-Materials and expands the remit to various stakeholders outside HE (including private companies and  3rd sector agencies).  They see the value of adopting a baselining approach which offers a more systematic approach to evidence gathering. They have developed an instrument to gather current understanding and practice of their stakeholders and will apply that at various stages throughout the project to evidence change. This is likely to reveal some interesting information about readiness of other sectors to engage with OER and highlight perceived barriers and enablers.

REACTOR

Led by Doncaster College this project builds on the UKOER phase 2 project SPACE and aims to develop 3D interactive resources for environmental technology which will be made available on web from project website, but also to be downloadable onto mobile devices. Also focussing on a range of sectors outside HE this project has also invested significant energy into engaging very different groups of stakeholders  (including sector skills councils, private companies, public sector, as well as HE in FE) Early research to identify what teachers and students within these curriculum areas want or need has been supported by the wider networks of these agencies. They also bring different resources to input to the OERs and offer opportunities to make sure they reflect the curriculum needs of many stakeholders.

Opening up a Future for Business

Led by Southampton Solent University this project has an FE & 6th form college audience as it aims to collate and produce an OER to enable 16-19 year olds, thinking about their future, to move forward with confidence in to studying Business and Management topics in Higher Education. The project emphasised the need to reconceptualise their approach as the project has progressed.  By examining how FE teachers already use OERs they discovered several institutional barriers such as blocking of some sites, to very protective and risk averse management of access to services on the web.  Copyright restrictions on materials produced by other educational institutions have affected what teachers use and what can be adapted as OErs.  The project is looking at how openness spreads within institutions. All of these issues are impacting on platform choice for their disaggregable book – with Moodle emerging as a strong choice as many of their stakeholder institutions use this VLE.

Common themes and strands

The following  areas/linkages  were noted as significant for this group of projects and we anticipate interesting lessons emerging around these:

  • Linking with students as important stakeholder group
    • Students for evaluating and testing
    • Students as project employees
    • Student-generated content
  • Subject area linkages- engineering, environmental technologies, business
  • Linking with a wide range on non HE stakeholders  – not just educators and students in these organisations  (lessons around stakeholder engagement, practice change, challenges/enablers)
  • Choice of Platforms (links to different stakeholder requirements, constraints and influence)

Isobel and I found this meeting really valuable and the projects also highlighted the usefulness of this approach to enable sharing of similar evaluation instruments, sharing processes (not just evaluation related), talking through issues and reflecting on emerging lessons at regular intervals, and in the longer term possibly in presenting evaluation outcomes to a wider audience. We also discussed the evaluation toolkit and how this can help with reflection on lessons learned and preparing for reporting.

This buddy team did note the value of meeting as they prepare their Interim reports and are planning another meeting in June. We will meet again as projects prepare for their final reports too.

Evaluation toolkit

By isobelfalconer March 23, 2012
evaluation, UKOER Phase 3
, ,

We are delighted to launch our new Evaluation Tookit which aims to benefit UKOER projects by providing a structured way to collate their findings and observations and can also be used in the reporting process. It also provided lots of information, advice, examples and videos, on evaluation, sources of evidence, and buddying. To help projects get the most out of the toolkit, we have produced a guide.

UKOER Evaluation toolkit on the wiki

The interactive element of the toolkit provides two alternative routes into the structure – through the programme themes, or through key evaluation focus areas. Both routes utilise the same underlying mechanism to record findings, observations and evidence – a series of google forms and spreadsheets – but the higher level user interface is very different.

  • The key focus area route, via links on dedicated wiki pages, is useful for those familiar with our evaluation framework. Each focus area has an individual wiki page which provides a description of that area and some specific issues and aspects that the programme is investigating.
  • The programme themes route offers a more visual approach via an interactive mind map (using spicynodes) where you can explore the themes of the UKOER3 programme and drill down to find the specific issues and aspects that relate to the themes you are investigating. We have produced a quide to using the spicynodes maps.

Whichever route projects choose, they can opt to receive an email record of their entries for future reference and report writing.

As projects approach writing their interim reports, we hope this provides an opportunity to reflect on, and perhaps reframe, the evaluation questions they started with, in the light of experiences to date, as it becomes clearer where their valuable contributions lie.

We welcome feedback on the toolkit, and expect to revise and improve it in the light of comments.

Open educational practices – what do we mean?

By Helen Beetham March 9, 2012
open educational practice
, , , , ,

To conclude our posts for open education week, I’ve been asked to introduce our new briefing paper on open educational practices.

I think we’ve been talking about open practices since the pilot phase of UK OER – you can find it in some of our recommendations. When we set out, I for one had the idea of a perfectly formed OER as openly licensed, self-contained, professionally tagged with educational metadata, and probably hosted in an open repository. Very quickly it was clear that open release would be messier and more interesting than that. OERs were caught up in a host of other emerging practices, such as guiding students to freely available content (with open licensing being only one way of judging value), or using third party services to support students outside of institutional learning environments. The people who were  interested in making open content had other open agendas too, some of them very radical.

One problem with tracing the links between open practices is that they tend to be ‘owned’ by different agencies. Even within JISC there are fascinating developments in open education at large, in open access publishing, and in open research data management, that I am only just finding out about. Can we build better connections? The Curriculum Delivery programme has reported on new pedagogical models in which open content makes good sense, while the Curriculum Design programme aims to make design teams more effective and collaborative – couldn’t high quality OERs be of some help here??

Finally, I’m interested in how ‘open’ embodies a different set of values for different people. Other members of the team have pointed out to me that open learning is not the same beast as open education. And our work with earlier phases of the programme confirmed that different stakeholders have very different perceptions of benefit and risk. It’s important to acknowledge the complexity of the open landscape and I hope our new briefing sheds a little light. Do also head over to where Amber Thomas has collated some  visualisations of the wider open education landscape. And please comment!

TEPL-SIG webinar

By Lou McGill March 8, 2012
open educational practice
, , , , , , , ,

On 2st February 2012 isobel Falconer and I led a webinar for the TEPL-SIG (Technology Enhanced Professional Learning – Special Interest Group) on Open Practices which aimed to encourage participants to consider their own practices. We considered open educational practice (OEP) but also touched on issues around wider open practices with a focus on how this might differ for people in different stakeholder roles (learner, teacher, employer -or even a mix of these). We were keen to take a broad view to emphasise that open practice in one aspect of your life can lead to or influence openness in other areas. The level of participation was excellent and I left hoping for an opportunity to take this further as we only had time to touch on a few issues.

The group included people from around the world with several different roles and levels of experience with OER or OEP. It was nice to have some of the UKOER project team members there as we were drawing on their work and the work of the UKOER  programme to illustrate our discussions. We made a wiki page for participants to refer to prior to the session with specific reference to the new briefing paper on open practice across sectors. below are the slides that we used for the session.


Before talking about UKOER and findings we asked particpants a series of questions which really got the discussion flowing (see slides 2 and 3). We were basically asking if open practice is a new concept or if our existing practices are just changing – not rocket science to be sure but a useful way to get participants to reflect on their own practices and how these are impacted by organisational cultures. As expected we did have attendees from professional contexts where some organisational knowledge was seen as a commodity that shouldn’t be shared. We discussed issues around degrees of openness where, in some contexts, there were valid reasons for maintaining some restricted access – a good example of this being around patient data in the NHS.

Isobel offered some insights from the UKOER programme which led to discussion around the motivations for open sharing and the benefits in different contexts. We finished with by considering the challenges and barriers for different sectors.

What is always frustrating at events like this where you have a very fast moving text chat is that people say something really interesting and then chat moves on and you don’t have time to really delve into a particular aspect. The conversation touched on areas such as:

  • inequalities and whether OER adds to the rich/poor divide or actually helps to challenge it
  • issues around use – the differences between learner/teacher use and how the means of production impacts on quality and size of OER/OCW
  • organisational showcasing
  • notion of institutions paying ‘lip service’ to the principle of being ‘open’ online but not necessarily considering the value of this in a local community context
  • sustainability and embedding of tools to support open processes (and challenge of integrating these into existing practice)
  • issues around confidence of individual teachers
  • fears of litigation
  • questions about why we are even talking about this – are some initiatives already making our discussions obsolete (Coursera, Udactity, MITx)
  • how current initiativess around assessment and accreditation are going to impact on things

One of the people attending suggested that we hold the webinar again during open education week which has not been possible – but we hope publishing this post will at least provide some useful information to support the activities during the week and you can watch the recording of the session too. we would be delighted if you would like to continue the conversations we started here in the comments section too.

UKOER phase 2

By isobelfalconer March 7, 2012
open educational practice, UKOER Phase 2
, , , , ,

Synthesis activities for the first two phases of the HE Academy/JISC UKOER Programme have included cultural and institutional issues across a number of sectors. The Evaluation and Synthesis team’s final report on phase 2 synthesises evidence and outputs under five main headings:

and includes recommendations to the funders and to the stakeholders represented in the various strands of the programme.  Further, detailed findings from each component strand of the programme addressed how different communities and cultures are progressing towards openness in their educational practice and in their management and use of educational resources:

  • Release – projects undertaking release of new or repurposed OERs in key subject areas
  • Open Materials for Accredited Courses (OMAC) – projects releasing materials linked to the national professional standards for staff who teach in higher education
  • Cascade – projects cascading and embedding their own good practice in OER release to other contexts
  • Collections – projects using a range of technologies to collate existing OER in thematic collections

On practice change, evidence is accumulating that teachers do share content informally, but do not necessarily consider IPR, or share resources openly. Thus they are engaging in some of the practices associated with OER, but they do not necessarily recognise OER terminology, and in asking about use/reuse of OER we may be asking the wrong questions.  This finding has implications for development and release, where cascade strand projects noted that choice of search engine affected which resources were found and used, while collections projects found that their users expected searches to be “google-like” in their ease of use, personalisation, and production of relevant results. Users expressed frustration with the search functionality of many OER repositories. They were also disappointed in the scarcity of relevant OERs, a finding echoed by the release strand PORSCHE project. Most of the collections projects decided to include “grey” or “non” OERs to get round this problem – while clearly labelling them so that their non-licensed status was clear.

Thus a key question that is emerging as important to OER practice is what difference does it make to practices of development/use/reuse and sharing, that the resources are “open”?

Experience during this phase confirmed that OER practice cannot be divorced from other open practices.

In some disciplines, sharing practice through a range of open technologies has emerged as important as sharing resources, and is having an impact on the way subjects are being studied and taught. Considerations of OER use cannot be divorced from these wider changes to disciplinary knowledge practices. Social science subjects, for example, are being changed in radical ways by the availability of public social and research data online as well as the rise of new social/digital practices.

Findings from this phase confirm those from the phase 1 institutional strand, that there are different cultures of openness at different educational institutions. This is not as simple as a single dimension from closed to open: rather there are many different ways in which institutions can support open educational practices.

For all stakeholders, then, a key question is, why engage in open practices? But equally, what is it about a practice that makes it open?

Our final report on phase 2 offers no simple answers, but lots of evidence to support you in exploring these issues.