Learning Graph + Reputation Graph = Massive Disruption in Higher Ed?

crackHere’s an outline of a thesis I’m working on that was also my suggested topic for my application to speak at TEDxSFED:

Higher education is on the brink of massive disruption right now. The “cost” of learning has never been less expensive as the Web is flooded with amazing content, tutorials, discussion boards and more. At the same time the cost of credentials (i.e. degrees) is sky-high (out-state tuition alone at a UC school is now $35,000/year…that’s for a public school!) and rising at a rate of 8% per year. The disconnect between these two feels quite unsustainable. So what could be the disruptive factor in this space? In my opinion, the combination of a learning graph and a reputation graph could massive disrupt higher education as we know it.

First, let’s define a learning graph. I think I first came across the term on Kirsten Winkler’s blog. In my mind, a learning graph is the roadmap of all of the things we’ve ever learned. But more than simply a “check the boxes” listing of all the concepts we’ve studied and mastered, it’s a dynamic and fluid measure that includes the rate at which we learned specific things, when we learned them and how much difficulty we encountered along the way. The usable end product of the learning graph would be a dashboard where anyone could abstract the data to end up with clean and concise view of what a person knew and their relative level of mastery.

Sound like Sci-fi? There are a host of people working on building on the learning graph right now. Early pioneers in the space included companies like DreamBox Learning. Recent entrants into the space include Grockit, Knewton and TenMarks (if you have the time, this 7-minute video from Knewton gives a pretty good sense of what the learning graph could look like). Each of these companies is trying to build a platform that will allow students to receive a personalized learning experience that is tracked over time.

Fast forward a bit and you can imagine a university looking at a student’s Grockit or Knewton dashboard when trying to decide whether to admit them to their school. Fast forward a bit more and you can imagine a company looking at a prospective employee’s dashboard to determine whether or not to hire someone. It’s a bit out there but I think it demonstrates the power of the learning graph. In a conversation today with my good friend Ranjit Mathoda he mentioned that if he was trying to figure out how smart someone was one of the first things he would do is find that person’s account on Quora. Indeed, sites like Quora, Stack Overflow, Namesake and others could end up playing major roles in the learning graph.

So why is a learning graph so important and potentially disruptive? I think a major reason is that it could eventually replace assessment. After all, if you’re tracking everything a person is learning the notion of stopping to test them on a particular piece of knowledge seems less important. In fact, it seems like the only real reason we need to test is to track how much a person has learned. The learning graph threatens to put an end to testing and assessment as we know it.

And why that’s disruptive is that educational institutions derive much of their power (and hence, their ability to charge large sums of money) from their ability to assess. Pop quizzes, mid-terms, final exams…it is these things that make school today what it is. But what would the world look like without discrete testing and assessment? Well, my feeling is that in such a world it might matter less where you went to school. Or even whether you went to school at all…

But wait, before we get too far ahead it’s important to realize some potential failings of the learning graph. It can’t measure everything. It does an especially poor job of assessing the “soft skills”. Is someone a good communicator? Are they creative? Do they work well within a team? The learning graph doesn’t tell us much here. I might be able to see from someone’s Quora account that they know a lot about entrepreneurship. Would I want to start a company with them? Not necessarily.

Education, especially at the post-secondary level provides a strong filtering and sorting mechanism for society. There’s a reason why some companies will only hire Ivy League graduates. And it’s a big part of why the top schools have incredible pricing power. But what if you could develop an alternative signaling mechanism that rivaled or even eclipsed what schools currently do? I think that’s precisely what the reputation graph could become. It’s still way early but I could see the reputation graph ultimately playing a very important role in decision-making about people.

Neither the learning graph nor the reputation graph alone could disrupt higher education. However when you put both together things start to get really interesting. The learning graph gives someone great insight into what you’ve learned throughout your time in school. In fact, it probably does a better job than GPA or any of the other rudimentary measures we use today. The reputation graph helps to dramatically strengthen the sorting and filtering piece of the puzzle. It gives us insight into who is the smartest, the most creative, strong communicators, best team players and so forth. Put those two things together and you start to question whether degrees are even important. At a minimum, we can envision a world in which the power of the degree is dramatically weakened.

Just to be clear, a built-out learning graph + reputation graph scenario is many, many years away. But I think it’s interesting to start to talk about what that might look like. To follow the companies who are potentially building the learning graph and those that are playing a role in the building of the reputation graph. And of course this is an over-simplification of matters. Schools serve many other roles (not the least of which is providing a social network for each of its students) and it would be foolish to argue that they’ll be going away anytime soon.

Instead what I see rising up are alternative ways to prove to the marketplace that you’re competent and credible. Historically, universities have had an almost monopolistic grip on that (especially for younger people). I think we’re moving into a world with much more diverse mechanisms for determining who truly is a rock star. And I think that’s actually a very good thing.

11. February 2011 by Jon
Categories: Disruption, Reputation Graph | 31 comments

  • http://www.edumorphology.com mpstaton

    I'm in. Let's build a company…

  • http://mathoda.com Ranjit Mathoda

    I think one day in the future people will be able to learn from a system that also provides a credential that a prospective employer or team would be interested in. Rather than saying “I graduated from Harvard” a person might say “I am certified in ____ by the education company ______.” But I would point out that while the websites listed in your blog post seem to rely on assessment techniques that depend on multiple choice answers heavily, certain types of assessment may depend on the ability to handle a challenge or provide an answer in the context of ambiguity.

  • Ryan Busch

    Wow Jon …i was just having a conversation about this today and now steve sends me this. I must have logged into the disruptors wifi signal this afternoon !

  • http://twitter.com/narkor Narkor

    Destroying the undergraduate teaching system, which basically finances all higher levels of research, will have disastrous long term effects for the economy. A professor can only supervise a small number of Doctoral candidates – unless you want to farm that out to Khan Academy as well. In disrupting the undergraduate system, the supply of professors will be drastically reduced. Reduce the supply of professors and you reduce the ability to produce postgraduates. Reduce the number of people who can study at the postgraduate level and see what exciting things happen to the economy. This scheme is almost republican in its scope to almost completely wipe out learning beyond the graduate level.

  • http://www.facebook.com/chrismanfrank Chris Frank

    A Paul Graham quote: “…the variation between schools is so much smaller than the variation between individuals that it's negligible by comparison”. Right now, where someone went to school is a good heuristic – the average Ivy Leaguer is probably smarter than your average state school graduate. But the combo of these two graphs would make it easier to compare people based on merit. I'm in, but I don't think the big universities will like your plan :)

  • http://www.theascendancegroup.org/ TAG

    Big, big ideas here… perfect for a TEDx talk. ;)

    The challenges in terms of distribution would take 9 figures to overcome. That concerns me, as does the small problem of changing human behavior (most students aren't on LinkedIn / Quora / Namesake and don't realize the importance of building their online reputations until they've already made it through post-secondary education).

    But there is real merit in your ideas… keep 'em coming!

  • Joseph Thibault

    I get it, but what I don't think will happen is that the learning/reputation graphs will replace or unseat the power we all place in assessments. Just because I have reviewed x material or watch y video does not confer understanding. I know that's where the reputation graph is meant to fill the gaps, but the leap from assessment-derived credentials to peer/histogram derived maps will be hard.

    That said, including assessments of all kinds to verify comprehension in the graphs could have HUGE power.

  • http://twitter.com/jonbischke Jon Bischke

    Hey Joseph. I think assessment can happen continuously. For example, the way assessment works with Khan Academy is that you do a certain type of problem until you get 10 in a row right. Once you do you move on to the next more difficult level of problem (until you get 10 in a row right there and so on).

    If we're continuously testing someone's acumen (which we can do with technology) why the need to come back and do separate assessments?

  • http://twitter.com/jonbischke Jon Bischke

    Most students aren't on LinkedIn/Quora/Namesake…yet. I think it's a fallacy (no offense) to assume that just because things haven't worked this way in the past that they won't in the future. Many young kids are on Facebook now. Did people think that would be the case five years ago? Probably not. But times change. And I actually Quora (and similar sites) will play much more of a role in disrupting education than most “education” companies will.

  • http://twitter.com/jonbischke Jon Bischke

    Epic quote. Thanks Chris!

    Big universities might not like it but they might not have a choice. I feel that a lot of education companies are similar to where the record labels where a decade ago. They're trying to pretend that the world isn't changing fast or that the dynamics of change won't affect them. They'll likely continue to do this until it's too late to make meaningful shifts in strategy. And just as companies like Apple have usurped most of the power in the music biz, my guess is that the powerful companies in education in the coming decades will look very little like the ones in power today.

  • http://twitter.com/jonbischke Jon Bischke

    Maybe…but I think what's really interesting about education is that many of those who are teaching would rather not be. They'd rather be researching. So you have a person at the front of the class who would rather not be there and who is evaluated not based on how well they teach but on how much research they publish. Does anyone else see why this might be a problem and why so many students have poor experiences in college?

    I think there's an answer for the scenario you suggest Narkor but I don't think pretending like we have to keep one sub-optimal system propped to support another necessary and vital one is that answer.

  • http://twitter.com/jonbischke Jon Bischke

    The SSID is Hacking_Edu and the password is “revolution”. :)

  • http://twitter.com/jonbischke Jon Bischke

    Aren't you already doing that? ;)

  • http://twitter.com/jonbischke Jon Bischke

    I think you're exactly right. I think we'll see a big shift in how credentialing happens. Right now my sense is that the current system only survives on the virtue of there being no better system. I think very few thoughtful people would agree that the way post-secondary credentialing currently works is ideal.

  • http://www.thecuriousentrepreneur.com AndrewSkotzko

    Jon, you raise some interesting points. I couldn't help but think of the post that Paul Graham did on credentials (http://www.paulgraham.com/cred…) and see a direct tie-in with this.

    One question that comes to mind: what else needs to happen to enable widespread acceptance of the technological replacements for the Ivy-league sorting/filtering function? I'd argue (based on gut alone) that the Ivy League degree derives most of it's value from it's mainstream acceptance. So, besides tracking & making the information widely available, what do you think tech disruptors need to do in order to make their equivalent of the credential accepted (and therefore valued) by mainstream society?

  • http://twitter.com/jonbischke Jon Bischke

    The proof will be in the pudding. When/if companies start to sense that they will have a competitive advantage by hiring based on alternative credentialing systems they'll shift. As I posted to Twitter last week “Would you rather be a graduate of YCombinator or of Stanford?” My sense is that many companies (e.g., Google, Facebook) would prefer to hire a YC grad over a Stanford grad (on average). Now YC isn't a tech-based filter so it's not a perfect analogy but you're seeing an increasing number of employers look at things like StackOverflow participation when making hiring decisions. Change is on the way!

  • http://www.thecuriousentrepreneur.com AndrewSkotzko

    Very true. I've used StackOverflow, Quora, Twitter, and GitHub to add depth to resumes before, and always look for outside school projects (built something cool in spare time, started an organization they believed in, etc). Since startups have to be on the cutting edge to survive, it's probably only a matter of time until these practices & other similar ones tip into mainstream companies.

  • http://www.facebook.com/profile.php?id=670081041 Matt Braynard

    Testing is necessary because even though you could do ten problems in a row of a certain kind yesterday, you can't necessarily do those things tomorrow because you can forget. I did quiet well in calculus but know that if I had any kind of assessment now, I would fail.

    Testing is also necessary because not all people learn equally well. It's the difference between the person who gets an A on a calculus pop quiz and someone who gets a C.

    In terms of reputation 'graphs,' those things become too easy to game/fake. It's much harder to fake acceptance and subsequent graduation from a university.

  • http://twitter.com/jonbischke Jon Bischke

    I agree that when you learned/mastered something is important but I don't know that you necessarily need discrete tests. The fact that you took calculus in the past is good enough for someone to gauge your relative body of knowledge. It's not like we're testing 30-somethings in the workforce on their level of calculus mastery anyway (even if some knowledge of calculus is required for their job).

    As to reputation graphs being gamed, that will be a huge challenge. That being said, the value of the reputation graph presupposes that it is a relatively accurate measure. And while it's harder to fake acceptance and graduation from a university I think we all knew people from school who we wondered how they got there and how they ever managed to graduate! :)

  • http://www.einztein.com Marco Masoni

    You are dead on. Thanks for an extremely insightful article. As you said, there's still a lot to be figured out but the spiraling costs of higher ed, the OER movement, the ubiquity of Internet access and the digitization of ed content are converging for a massive innovation push that will bring change surprisingly fast.

  • http://www.facebook.com/people/Jon-Bischke/500456005 Jon Bischke

    Let's hope so. :)

  • http://www.facebook.com/people/Jon-Bischke/500456005 Jon Bischke

    Let's hope so. :)

  • http://thetrailblazinglife.com Collin Vine

    Have you ever read the book “Weapons of Mass Instruction” by John Taylor Gatto? He proposes that the disease of traditional education lies within standardized testing. His idea is to abolish standardized learning and testing for more self-directed, teacher-facilitated models.

    My roommate asked me recently what I thought would happen to the future of education. With the forefront of education technology and platforms (as simple as it may be) like the Khan Academy, it's easy to see the value decreasing within the 4-wall institutions that we know.

    This article really got me excited about the future potential. I'm excited to think about having better options when it comes time to put my future kids through school (I had previously resided to home schooling; this may provide an even better means).

  • Pingback: Alexander

  • http://pezcandy.blogspot.com Karl

    “There’s a reason why some companies will only hire Ivy League graduates.”

    Since you don't state the reason for us, maybe you could explain what it is.

    I'd wager it's because you believe it's true that Ivies “are better.”  

    If so, I'd ask for the proof of that assertion's truth.

  • http://pezcandy.blogspot.com Karl

    “probably smarter than”?

    What bull-pucky!  The only “proof” that Ivies get higher quality noggins in their rolls is the myth of Ivy superiority.

    High GPAs in K-12 do not prove intellectual ability.

    A resume chock-full of extracurricular “accomplishments” K-12 does not prove intellectual ability.

    Over-achievement is the defining trait of the non-legacy Ivy student's background.  Over-achievement is a sign of deficient intellect, not superior intellect.

    This is what lays at the bottom of the fraud of credentialing, academic-style:  a bunch of myths designed to confer advantage on those who attend Ivy schools.  With over 200 years of myth-making in their favor, the Ivies have done well, almost as well as the Roman Catholics or the Mormons in spreading their mythology.

    A person's ability is determined by doing, not by having a keen parchment.

  • Pingback: Alexander7

  • http://satandcable.com/manufacturers/dreambox Dreambox 800

    To learn about the graph is not so easy that it sounds you have  to made a detailed study. Over here you have shared about the repudiational  graph and how to made it and how higher education need and also defines the important potential failings of the learning graph.

  • http://www.brandvois.com/reputation-management/ Reputation Management

    I think if you combine a learning graph with a reputation graph you have something that could be extremely disruptive.

  • http://stctechnologies.copytaste.com/ STC Technologies

    Good news.

  • http://stctechnologies.copytaste.com/ STC Technologies

    Good news.