Learning Analytics for 21st Century Competencies?

safariscreensnapz484We’re delighted to announce a Special Section of the Journal of Learning Analytics, published this week, focusing on the challenge of Learning Analytics for 21st Century Competencies. In our editorial we introduce the nature of the challenge, and after summarising the different researcher and practitioner papers, propose a complex systems approach which takes seriously the ‘layers, loops and processes’ of learning infrastructures and the iterative relationship between the human and the digital, where people learn at the nodes of networked flows of information.neural_network_by_rajasegar-d2xx3w9

Learning analytics is an emerging field powered by the paradigm shifts of the information age. Pedagogy and learning that produce students capable of thriving in conditions of complexity, risk, and challenge by taking responsibility for their own learning journeys, and using technology and analytics to scaffold this process is  at the heart of the challenge. It is an emergent field, still struggling to find its way. These papers represent a unique ‘window’ into this programme from the viewpoint of both users and researchers.

You can enjoy full access to all the articles, since JLA is an open access journal.

I gave an overview of the topic and some of the papers in the above volume in this talk to the Asian Learning Analytics Summer Institute, with thanks to Yong-Sang Cho and the LASI-Asia team for the kind invitation…


Learning Analytics for 21st Century Competencies

Simon Buckingham Shum, Ruth Deakin Crick

Towards the Discovery of Learner Metacognition From Reflective Writing

Andrew Gibson, Kirsty Kitto, Peter Bruza

An Approach to Using Log Data to Understand and Support 21st Century Learning Activity in K-12 Blended Learning Environments

Caitlin K. Martin, Denise Nacu, Nichole Pinkard

Understanding Learning and Learning Design in MOOCs: A Measurement-Based Interpretation

Sandra Kaye Milligan, Patrick Griffin

Practical Measurement and Productive Persistence: Strategies for Using Digital Learning System Data to Drive Improvement

Andrew Edward Krumm, Rachel Beattie, Sola Takahashi, Cynthia D’Angelo, Mingyu Feng, Britte Cheng

Analytics for Knowledge Creation: Towards Epistemic Agency and Design-Mode Thinking

Bodong Chen, Jianwei Zhang

Tracking and Visualising Student Effort: Evolution of a Practical Analytics Tool for Staff and Student Engagement

Robin Paul Nagy

Marks Should Not Be the Focus of Assessment – But How Can Change Be Achieved?

Darrall G Thompson

Scaffolding deep reflection with automated feedback?

We’ve all got used to the idea that computers can understand writing and speech to some degree — Google adverts that match your search queries… asking Siri simple questions… IBM Watson winning Jeopardy! But how does natural language processing fit into learning that goes beyond getting the right answer to a focused question, or matching some key concepts?

Language is clearly front and centre in the way that we learn from others, share our understanding, and narrate to ourselves. However, the idea that computers have any substantive contribution to make to the teaching and assessment of writing elicits strong reactions from educators, and understandably so (learn more from this workshop).

In recent work that we’ve been doing at University of Technology Sydney, we’re exploring to what extent text analytics could support the integrative practices of writing that many are now using to help students reflect on their learning. This is particularly germane to future-oriented pedagogy that seeks to help the learner narrate their own learning journey in relation to their identity (how does my learning change who I think I am becoming?), place them in authentic (often professional) learning contexts (on the ward; in the classroom; in a company), and so forth.

As we explain in this article, there are many pedagogical reasons for valuing deep, academic reflective writing, but it poses significant challenges to educators to teach this well, and to students for whom this is often a new genre of writing. Below is the presentation we gave earlier this year. Current the Academic Writing Analytics (AWA) web tool is available only within UTS as we pilot it, but in the future this may become available to other universities, and beyond that, to schools interested in collaborative research.

Buckingham Shum, S., Á. Sándor, R. Goldsmith, X. Wang, R. Bass and M. McWilliams (2016). Reflecting on Reflective Writing Analytics: Assessment Challenges and Iterative Evaluation of a Prototype Tool. 6th International Learning Analytics & Knowledge Conference (LAK16), Edinburgh, UK, April 25 – 29 2016, ACM, New York, NY. http://dx.doi.org/10.1145/2883851.2883955 Preprint: http://bit.ly/LAK16paper

Learning Power: new research identifies Mindful Agency as central to resilience

For learning in the complex world of risk,  uncertainty and  challenge, what matters is being able to identify, select, collect, collate, curate and collaboratively re-construct information to suit a particular purpose. This is why there has been a sustained and growing interest in learning dispositions and the personal qualities people, teams and communties need to flourish. As Edgar Morin says:

edgar morinWe need a kind of thinking that reconnects that which is disjointed and compartmentalized, that respects diversity as it recognizes unity, and that tries to discern interdependencies. We need a radical thinking (which gets to the root of problems), a multidimensional thinking, and an organizational or systemic thinking

Ruth Deakin Crick 2015After fifteen years of experience in the research and practical application of learning power using a survey tool called the Effective Lifelong Learning Inventory (ELLI), Professor Crick, one of the originators, led the research team in a thorough review and reanalysis of the data.  Supported by the Learning Emergence Network of international researchers, the results are now published for the first time in the British Journal of Educational Studies:

Ruth Deakin Crick, Shaofu Huang, Adeela Ahmed Shafi & Chris Goldspink (2015): Developing Resilient Agency in Learning: The Internal Structure of Learning Power. British Journal of Educational Studies. DOI: 10.1080/00071005.2015.1006574. Open Access Eprint:  http://dx.doi.org/10.1080/00071005.2015.1006574

Interestingly, the support for this re-analysis came from the Systems Engineers in the Engeering Faculty at the University of Bristol  as part of the International Centre for Infrastructure Futures, rather than ELLI’s original home in the Graduate School of Education….where Crick, Broadfoot and Claxton began in 2000.  Perhaps Morin would have something to say about this — we think so!

The new self assessment tool, called the Crick Learning for Resilient Agency Profile (CLARA) identifies Mindful Agency as a key learning power dimension — which predicts the set of active dimensions: Creativity, Curiosity, Sense-Making and Hope & Optimism.   Two distinct Relationship dimensions measure Belonging and Collaboration.  Finally, an Orientation to Learning indicator measures a person’s degree of Openness to change — in contrast to either fragile dependency or rigid persistence.

Internal Structure of LP with simplied view 19 August

The new measurement model represented by CLARA resulted from a detailed  exploration of the patterns, relationships  and interdependencies within the key constructs through structural equation modelling (diagrammatic summary above).  It is a more robust, parsimonious measurement model, with strengthened research attributes and greater practical value. The research  demonstrates how the constructs included in the model link to the wider body of research, and how it serves to integrate a number of ideas that have hitherto been treated as separate. For more details from a user perspective see  Introducing CLARA.

The CLARA model suggests a view of learning that, after Siegel is:

an embodied and relational process through which we regulate the flow of energy and information over time in order to achieve a particular purpose.

Learning dispositions reflect the ways in which we develop resilient agency in learning by regulating this flow of energy and information. They enable us to engage mindfully with challenge, risk and uncertainty and to adapt and change in a way which is positively alinged with our purpose.

Resilient Agency is our capacity to move iteratively between purpose and performance, utilising our learning power and generating and re-structuring knowledge to serve our purpose.

Learning JourneyLearning, from this viewpoint, is a journey which moves between purpose and performance – to put it another way, without having purpose we’re not really going to learn in a context of complexity and information overload. To learn, when the outcome is not known in advance (which is most real world learning) requires that we are able to navigate learning as a journey, utilising our Mindful Agency, restructuring information to achieve the outcome we need.

BlueThe Learning Emergence Network has teamed up with eXplorance Blue, one of the world’s leading survey providers based in Montreal, to create the SOLA platform (Surveys for Open Learning Analytics) which can host CLARA and other assessment tools, and importantly, provide rapid feedback to users for improvement purposes.

Visual feedback to the learner from CLARA

The rapid analytic feedback to users who complete the questionnaire is returned in the form of a spider diagrame which forms a framework for a coaching conversation which can move between learning identity and purpose and the formulation of strategies for change.  The new assessment tool is a focus for research and development around the world. Crick and Buckingham Shum are now based in the pioneering Connected Intelligence Centre and the School of Education at the University of Technology Sydney, where CLARA forms part of a research programme into dispositional learning analytics — alongside other learning analytics approaches designed to make visible – to learners and educators – the dynamics of lifelong learning qualities.

by-nc-nd (1)CLARA, and the knowledge and know-how in the research paper, have been made available for research and development under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives License. This permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.

We welcome all contributions to the ongoing research and development of this work which has applications in education, industry and community.  We have translated CLARA into Chinese, Russian and Spanish. For more details and opportunities for collaborative research and development please contact info@learningemergence.com

Universities: core business (and analytics) in 2030?

Screen Shot 2015-02-23 at 6.31.38 pm Ruth and I have the privilege of working with Randy Bass [blog] and team at Georgetown University. Randy is a leading thinker  around the deep purpose of higher education, and how this entails rethinking student qualities, and analytics.

Jump to 40mins for his closing comments in this keynote envisaging higher ed in 2030. Here’s the gist:

Our calling as a university is the formation of men and women (but many institutions do this of course). However, we do so in the context of a community of enquiry and knowledge creation (fewer institutions do this). Moreover, we do so for the public, common good (fewer still have this explicit mission). These three are interlocked and inseparable.

The railroad companies who thought they were in the business of railroads went bust. The ones who thrived understood they were in the transportation business.

What’s our equivalent?

Let’s call it Formation.
Or Transformation.
Or Integration.

But if we think we’re in the business of Content, Skills or Information Transfer, then by 2030, we’re going to have a LOT of competition.

…or, as we might say, Dead In The Water.

His Formation by Design (FxD) initiative is defining the contours of this new landscape, and their progress report is an inspiring read (disclosure: it includes material from our contributions to a symposium last June). Or check out the video roundtable discussion series he hosted called Reinvent University for the Whole Person. He was also on the team of (what I think is) the largest national ePortfolio initiative in higher education, a reflection of the importance being placed on reflection for transformational learning.

Randy and team: all power to you as we figure out together how we redefine our calling, to help students find theirs. Along the way, lets reinvent the environments and metrics that will constitute the new evidence base in 2030 🙂

EdMedia 2014 keynote on learning analytics

At EdMedia 2014 [#edmediaconf] I enjoyed great AACE and Finnish hospitality 🙂

The critical stance of my keynote there seemed to resonate with delegates, who hear a lot about “Big Data” and analytics, but have reservations about the kinds of learning that such technologies may perpetuate. I sought to deconstruct analytics to clarify the ways in which an approach and how it is used embodies an educational worldview. Knowing this, what kinds of learners are needed for 21st century society, and what role can analytics play in advancing this mission?

Part of this emerging picture is what we’re focusing on here at LearningEmergence.net — redefining metrics that value qualities in the learner that many are talking about, but which are hard to evidence.

Here’s the replay + slides [pdf/pptx].

Abstract: Education is about to experience a data tsunami from online trace data (VLEs; MOOCs; Quantified Self) integrated with conventional educational datasets. This requires new kinds of analytics to make sense of this new resource, which in turn asks us to reflect deeply on what kinds of learning we value. We can choose to know more than ever about learners and teachers, but like any modelling technology or accounting system, analytics do not passively describe sociotechnical reality: they begin to shape it. What realities do we want analytics to perpetuate, or bring into being? Can we talk about analytics in the same breath as the deepest values that a wholistic educational experience should nurture? Could analytics become an ally for those who want to shift assessment regimes towards valuing the qualities that many now regard as critical to thriving in the ‘age of complexity’?

Bio: Simon Buckingham Shum is Professor of Learning Informatics at the Open University’s Knowledge Media Institute, where he is also Associate Director (Technology), overseeing knowledge and technology transfer to the OU. He researches, teaches and consults on Learning Analytics, Collective Intelligence and Argument Visualization. He co-edited Visualizing Argumentation (Springer 2003) followed by Knowledge Cartography (2008, 2nd Edition 2014). He served as Program Co-Chair of the 2nd International Learning Analytics conference, chaired the LAK13 Discourse-Centric Learning Analytics workshop, and the LASI13 Dispositional Learning Analytics workshop. He is a co-founder of the Society for Learning Analytics ResearchCompendium Institute and and Learning Emergence. In August 2014, he joins the University of Technology Sydney as director of the new Connected Intelligence Centre. WWW: simon.buckinghamshum.net