Showing posts with label predictions. Show all posts
Showing posts with label predictions. Show all posts

Monday, February 14, 2011

The Tertiary21 Manifesto for Higher Education

If I was a political party, what would my higher education manifesto be? Have spent the last two posts poking at the Hunt Report and the Higher Education Policies of the political parties contesting the upcoming general election, isn't it only fair that I nail my colours to the mast and say what I would do to (for, even) Higher Education when I'm inevitably appointed by mass acclamation as planetary despot-for-life.

I've spent almost four hours working out the detail of my proposals (by my estimate, double the time spent by most of the main parties) I'm entirely open to suggestions for improvements or for odd incentives my framework will create. In a short posting I need to keep it high level - I recognise that every sentence of this probably needs many pages of operational policy to flesh out the details and smooth out the fish hooks.

Autonomy 
Firstly, all higher education institutions need to be entirely autonomous, as non profit entities. I don't care what they teach, who they hire, what they charge, whether they grant tenure or not. Off you go. Enjoy yourselves.

Of course there is a catch.

Funding
Funding will be a state supported student loan model, largely following the Browne Framework. I'd love to simply make it free for all, but I haven't got the money. What I will try to do is make it a cheaper system, through other reforms, turning the screws to drive adoption of new, open learning paradigms. Full economic fees for an Arts Degree are €10,000, which is ridiculous. More on tackling that later.

A state run loans system does give me a bunch of handy levers. I can set some courses at zero (or inflation only) interest if we a short on graduates, or even write down loan balances for people who go on to do necessary, but underpaid jobs like teachers and nurses (that is, if actually paying them decently in the first place remains untenable). Loans will be drawn down module by module, and should be straightforward to access, not means tested.  Income levels for repayments need to be set so that you are only repaying the debt your education actually helped you earn more than a comparison group that didn't borrow or study.

There are lots of potential fish hooks to be watched for. We'll need some provision in the system to ensure people don't abuse it by failing module after module. Loan levels for some modules will need to be capped if the employment outcomes aren't panning out. There will be an incentive in the system to create expensive, cool sounding modules ('Outdoor Adventure Instructors' say) that will draw students, but not, once the job market is saturated, actually help them all that much professionally. We'll need a way of handling dropouts and partial completions.

Funding for living costs is a genuinely thorny issue. We need to support poor (sorry, economically disadvantaged) people who actually get as far as third level and want to study full time. At the same time, we want to discourage people from becoming eternal full time students, living on loans they will never repay.  While cheap loans for tuition go to what is, for now, a small controllable group of Higher Education Institutions which were already state funded, cheap loans for living expenses get spent and go straight into the economy - it's potentially very expensive, and much more macroeconomically active. There's no easy one paragraph answer to this issue. It's going to take some very delicate policy work to find a good solution. We can figure it out, but not over a weekend, and not in a mid length blog post.

I can also set conditions which must be met for a given module to be eligible for loans. You're not going to like them one bit.

Openness
To be eligible for loan funding, all of a modules material - course outlines, lecture content, slides, handouts, booklists, marking rubrics, sample papers, marking rubrics, teaching plans, who will teach it (and their qualifications for doing do) - must be open access, available online under a creative commons licence, ideally free for non commercial reuse (ie the other non profit HEIs can nick it). Course will change from run to run, so planned future variances must go up there too.

If you are not familiar with the ideas of open education, there are a couple of recent pieces in the Irish Times and the Times Higher Ed, or you could read DIY U, or take a look at people like David Wiley , a leading US advocate.

Here are a few rationales for Open Education at Third level in Ireland:

  • Taxpayers have paid for the material, and are entitled to see it. Virtually every piece of higher education teaching content in the state was developed with taxpayers money. You are entitled to access it. Public money should not be used to create private assets.
  • People not attending the course can benefit from open education materials. You might not be able to pay to sit the module, but you might still learn a bit by reviewing at the material online. The feedstock of a knowledge economy is knowledge, and the cheaper and more abundant it is to access, the better for everyone.
  • It shifts the value proposition of a module away from a content driven, transmission model, and on to genuine teaching and assessment. You can't really charge a four figure sum for a lecture based module where the lecturer interaction is limited to a Postgrad marking an essay or two. Either the costs must drop, or the institution must deliver real worthwhile teacher interaction and assessment, or provide access to specific equipment or facilities that justifies the cost.
  • By placing such a mass of courses online, it establishes Ireland as a central node on the global landscape of higher education. Freely available higher education online will speak with an Irish accent. If you liked the TCD course on Genetics online, why not come on over and do the degree in it here? If you like those Tyndall Institute nanotechnology modules, maybe you should take to them about doing some R&D for you company.
  • The Open access requirement will discourage for profit providers. They will be less willing to enter the market if they have to give away the goods. I'm uneasy about government backed loans supporting for profit higher education institutions,  as it hasn't worked out well in the US or elsewhere. This will give them some pause.

Making the content available for free will reduce the inventive for new module development, so we may need to put a funding mechanism in place to provide seed capital for creating new material. I'm conscious also that dumping a weak module online simply creates a weak online module, and there is a long road from there to full, true open education, but the first step on the road is as important as the last. The material can be improved upon over time, and having it in the public domain will create a strong incentive to do so. If it isn't good enough to show in public, how can it be good enough to teach in private? If we believe higher education is a public good, then it must be public. But it must also be good

Quality
A fair proportion of the material put online will, of course, be rubbish. But at least we can all see that now. An open rating and reviewing system, both for the module material, and the lecturers and course delivery, will fairly rapidly sort the wheat from the chaff. Outstanding material and teachers, of which we have many, will rise to the top.
For qualities sake, I would also add a requirement that the module materials get a peer review every few years, which will be published too in the same place online. This means potential students can balance their own opinion of the materials, with scholarly and module graduate perspectives before making their decision to use that module. The peer review can also ensure the modules dependencies (prerequisite or follow modules) are appropriate. Weak modules will wither in the sunlight, and good ones will blossom.
With this model, I can ensure reasonable course quality with relatively little bureaucracy, and empower people to make up their own minds on whether a module is worth doing or not.

We will, alas, need some bureaucratic oversight, targeting audits on modules that are unusually expensive, or have unusually high (or low) completion rates or uptakes, or are getting strange reviews.

Have you noticed there is nothing in this framework so far to stop an individual, or any organisation, designing a course, having it peer reviewed and accredited, putting it ip, running it for free for a cycle to get open reviews, and being eligible for state loan support? Well done. Keep it to yourself.

Deep Modularisation
Underpinning the points of Openness and Quality is an ideal of Deep Modularisation - the Bologna Process taken to it's logical extreme. The Module is now the critical unit, not the degree. Education, and credentialling, can be acquired in module by module chunks, as needed, from any institution, not just in degree sized chunks at the start of a career, when you don't know what you'll need to know. It makes lifelong, distance and part time education the logical central paradigm, and the idea of full time degrees a less logical one. These kinds of changes take a generation to work their way into the mindset, but if we change the framework in which we imagine higher education, that mindset change will come in time.

Deep modulatisation also opens the potential for a module to be more than simply a classroom led learning event. Many of the best learning experiences for students are outside classrooms (In my degree, I learned the most useful things on Fieldwork) and we can open that out further, recognising prior learning and other, non classroom structured learning experiences within the module framework. Could we, for example, use a loan funded module to support an international relations student to work for an NGO like Concern for a bit? I'd certainly like to hear a case made for it. Let's get creative - if it's good, real learning, I'll back it.

Universities will complain to me that this will shift people from accessing full degrees into accessing learning piecemeal. I will understand this is 'people will no longer buy our big expensive things and want smaller, cheaper things instead' which is exactly what I have in mind. I've been rattling on about how the quantum of learning is changing to smaller, more agile units. Deep modularisation recognises this reality.

Access
Modules, of course, can still be stacked up into degrees, just like now, and modules will have prerequisite modules. No point in taking Quantum Mechanics 201 unless you have nailed a bunch of mathematics modules first. But tracing the web of modules down, there is a logical first module, for a 'prime learner, unlearned' which would have no formal prerequisites, and, as an entry level module, it should be possible to complete entirely by distance, just turning up in person for an exam (remote proctoring is, and will remain, technically dicey). So, in time, by picking off the bottom rung prerequisites online first, you can access higher education without bothering with the Leaving Certificate rat race, broadly seen as destructive of minds and (perhaps) lives, at all.

Governance
This should be straightforward. We need three main bits:
A body to administer the student loan and funding systems, it's all straightforward  stuff. In New Zealand, the Social Welfare system looked after that, it's just a bookeeping and payments/repayments management exercise. A couple of years ago, I'd just have tendered this job out to a bank or two to manage. Now, maybe not, but I'm not paying for a whole new agency from scratch for it, not with your taxes.
We'll need a good policy shop to get all over labour markets and outcomes and understand which modules we need to set at zero interest or incentivised in some way, and which we don't need to worry so much about where that sits in government doesn't matter (and, indeed, we can encourage independant research on the matter) once it's good.
Then we need a quality management agency. Their role is to make sure the modules are good, and deliver the learning outcomes they are supposed to, and manage module development grants and so on. They'll also need a rather bloody minded team of cost assessors, who can audit modules and make sure the list price bears some reasonable relation to the cost of delivering them.

Philanthropy and Endowments
I'd like to give Higher Education institution's a little bit of autonomy. After all, if they rely almost completely on loan funding, they haven't really got any independence, even though it is largely students who will control who gets the money. Independence is economic or illusory. Since their administrators will spend most of the next 15 years crying over these reforms we should throw them a bone.
We'll need to make sure that the tax incentives are strong for donations to long term institutional endowments, rather than buildings or isolated, piecemeal scholarships. I'd love to set up our existing Higher Education Institution's with a nest egg, like the US Land Grant system, but I don't have any land. Perhaps there might be something else, a little slot of RF spectrum perhaps, that we could transfer to them on a 100 year lease to give them the potential for revenue. It might be something that seems valueless to us today, as a publicly held good, but having title to it would create an incentive for them to find a valuable use for it. I'm open to ideas. This is a long term strategy, all the more reason to start right away.

That, in a nutshell, is it. An open, actively managed education system supported by government backed student loans. There are, of course, heaps of operational policy issues to address is detail, and whole areas like research and innovation policy, and international education, that I have left off for brevity, as as they sit outside a core educational scope, but I'll happily work out those details for you if needed.

I advocate the system for two reasons:

Firstly, most obviously, I believe it's substantially better than current arrangements. Some of the changes are fundamental, but, like many systems, the state of higher education is strongly dependant on it's history, and it has evolved itself into a dead end, with too much junk DNA from it's 1000 year history. Only an aggressive reform can set it free, and get it moving onwards and updards again. There is much good in our higher education institutions, and we can help set that good free.

Secondly, I think these changes are going to happen anyway. Global higher education systems will look like what I've described by about 2030. Ireland can lead the change, get in ahead, and put our higher education at the head of the global pack, or we can follow, and fall further behind. Leadership carries risks, and demands courage and audacity, but will bring great rewards for our children, and ourselves.

What do you think? 

Tuesday, December 7, 2010

The Four Big Trends: Pecha Kucha Presentation



Here's a presentation I did at the Cork Pecha Kucha event on the four big trends in Higher Education in the 21st Century. The content will be familiar to some readers, but you may find it interesting nonetheless.

Pecha Kucha is a really interesting presentation format if you haven't run across it, it's worth trying. 20 slides, 20 seconds per slide, slides advance automatically. No time for Ums and Awws, and you have to cut to your main point on the slide right up front or you'll miss it. The speed also forces you to gloss the details and caveats a little. Challenging. If you haven't tried it, do.

My apologies for the lighting on this piece. I do in fact have a face as well as voice, but the presentation is perhaps the better for my appearance as a Mysterious Shadow.

Thanks to Nicki Ffrench-Davis for arranging the event, taping it, putting it online, and making me do it over again when the tape ran out halfway through my first run.

Monday, July 5, 2010

Follow the Money

"Follow the Money" Raymond Chandler advised us (via Marlowe), and it's good advice to take when considering how the cost of Tertiary Education effects it's future. It's timely to think about now, as Ireland, from where I write, considers whether to continue with a 'free' fees model, where the state pays, or return to a fee paying model, where some or all of the fees are paid by students directly to the University. How might these choices play out in the long run, and which is the smart one?

All else being equal, the cost of a University education would rise in proportion with inflation. If GDP growth outpaces inflation, as it tends to, University education gets cheaper in real terms. More and more people can afford it, and we all live happily ever after. All else, alas, is not equal. As we get richer, our expectations rise. The school my firstborn starts in on August 31st might as well be on a different planet from the school I started at in '79. It's more reasonable to think of the cost of University education in terms of it's share of GDP.

Whether the cost will rise or fall as a share of GDP depends on who, exactly, is paying.

Where the state funds University education, as in Ireland, it's a single, strong customer with tight pursestrings. That keeps a lid on costs. Unless Universities suddenly get a lot better at picking the governments pocket than they are, the costs can't grow faster than GDP, and will probably shrink as other agendas draw on state coffers. The state will pressure universities to expand enrollments and spend less.  While the slice of national wealth consumed is smaller in relative terms but the cake is growing all the time. So, in absolute terms, Universities do get more money, and deliver a better service. The voting middle classes won't mind them bleeding a little, but won't like to see them bled white, just like schools. While universities command a smaller and smaller wedge of national wealth, more and more people go there, because it's cheap. You end with with near universal tertiary education in the long run excluding only those so already too badly sabotaged by family or school to make it. That's how state funded primary and secondary education works, why should tertiary wind up any different.

Universities won't like this. Having only one paying customer is bad business. Like a pea farmer selling to a big supermarket, you suddenly realize you don't work for yourself anymore, and your profit margin is what they tell you it can be. When they feel a squeeze, you get crushed. Primary schools don't have a lot of autonomy.

Universities would like to work like any other business. They would deliver a service, and charge what it costs, or as much as they can get away with, whichever is more.  As well as helping the top line, making goods more expensive makes them more exclusive. All Universities, deep in their hidden hearts, want to be Harvard when they grow up, and that means being expensive enough to filter out the riff raff, and elite enough to attract the very best talent. Employers, at least ones that potential students would like to work for, like that too. It makes it easy to sort the CV into a slush pile. There are some Universities, who, as a core value, try to keep costs as low as possible, but they are, alas, not economically significant. You can see this model in operation in the US.

Where education is on a fees basis, the prices will rise as fast as the market will bear. Faster, if predatory lending practices, as seen in the US, come into play. If people are borrowing for their education, based on their beliefs around future earnings, fees can logically rise faster than GDP growth. Fees go up, Universities command an expanding slice of the national pie, and (successful) Universities are grand and well resourced, like the US Ivy Leagues. If you can afford to go to the best, or can swing a scholarship, your future is assured. Less and less people can afford to go. Things like community colleges, online and for profit colleges spring up to satisfy that market.

Note that other costs (housing, food, and so forth) are more or less neutral either way. Whether you are in college or not, you've still got to eat and sleep indoors. There is an opportunity cost as well, time spent in lectures and years of earnings foregone while in University. In either model of funding, so long as University can get you a better income than no University it's worth going. That better income might be a better job than a non graduate, or indeed any job. If practically everyone has a degree, not having one would be a fast track to the benefit office, with rare exception.

These two scenarios are of course end members of a continuum of solutions. In Ireland, people suggest an intermediate model, where Universities charge fees but the government provides adequate scholarships to ensure access for people who cannot afford them. This is a great idea, and like many great ideas, it won't fly.  Democracies don't reliably support people who don't vote. Any scholarship programme for the deserving and needy will get whittled away over time. Cutting those programmes keeps them out of college, and not competing for jobs with the children of nice, voting, middle class parents who knows the name of their elected representative and would like the money spent on themselves. Think of the consistent heavy flak affirmative action attracts in the US, and how it survives only because it is supported by a substantial mass of voters who benefit from it.

The other intermediate model is to have some state supported free Universities and some fee paying Universities, so everyone gets what they want. This is not unlike the US system, and has serious consequences for social equity. Your degree will forever record how rich your parents were, and humans being humans, your place in the pecking order is set.

There are, of course, many excellent Moral Arguments for this funding system or that. I'm not concerned in this blog with Moral Arguments, however just. I'm just concerned about what will happen. History has no morals. Nor am I concerned with what decision Society will make. Society does not make decisions, it responds to stimuli.

Universities however, are small enough to make actual choices. Given these two options what should a clever University lobby for? What is likely to ensure your relevance and survival in the long run?

If you are in the top 10%, then lobby for a fees model. You've got a good shot at 'Ivy League like status in the long run. Take it. With the enforced egalitarianism of state funding removed, you can lay claim to the top spot in the public mind. Fees will bring competition, and you're well placed to win it. Call your Minister, book a nice Restaurant and good luck.

If not, think carefully. You're not going to get a big endowment or asset base, like the Land Grant colleges in the US, to cushion you against the hard years. In a free market, they will come. The state might be one big customer, but it's checks don't bounce very often. THe business will start to look a lot like the Private sector. Rising costs will bring in competition, like in the US. Strange movements, like the EduPunks, will challenge your right to exist as an institution. Students incurring big debts are much more likely to cause trouble. If your country grows an Ivy League, and you're not in it, what then?

Monday, June 7, 2010

Black Swans and The Fifth Megatrend.

I've just finished reading Nassim Talebs book ' The Black Swan'. It's essential reading if you are thinking about forecasting, and a lucid and entertaining read as well..

The core idea is that many processes thought to be governed by Gaussian distributions are really governed by Power Law Distributions. For example, over a short term, much financial data looks like it clusters around an average value, in the same way as height or weight does. In fact it's a Power Law, where larger scale events are increasingly rare, but overpoweringly large. Over a short or selective term, a set of data like, say, the the size and frequency of bank collapses, drilling accidents or airline disruptions seems like a simple normal distribution, with the probability of severe events tiny and easily quantified. A day later, events make fools of us all.

This idea impacts on recents posts where I set out four key trends which I thought would drive the story or tertiary education through the 21st century: Demographics, Economics, Telepresence and Artificial Intelligence. In each of these trends I somewhat boldly extend a current trend in a broad swish across the century ahead and consider the effects. As Taleb would argue, I am like the Turkey extrapolating his ever increasing meals and weight gain to dream of the mighty bird he will be come spring - a simple inductive fallacy.

It's worth taking the time to consider the rationale behind each of those trends, and how much inertia they have. Are they indeed driven by fairly dull, Gaussian processes, where, for example economic growth generally flits around two or three percent a year, with the occasional wobble up or down a few percent, or are great surges over and back possible - Black Swans, in Talebs language.

Population growth seems to be at fairly low risk of a major shift. It's a consequence of many factors; how many children people want to have, how many they can have, and how long they live. The biological factors are well bounded, at the bottom (you can't have negative children) and at the top (It's really unlikely you'll have 14 children, or live to 200). The variables are driven by things with a lot of inertia - how many children you would like is keyed to economic factors and social expectations which change slowly. Infant mortality is keyed to economic wealth, and how long you live is usually driven by economics too, through the sum of your lifestyle and health from womb to tomb. So population forecasts have a lot of mass behind them. Given that uncertainties coming from basic factors like birth rates and so on amount to over a billion people either way, it would take a particularly vigorous pandemic, or an enthusiastically conducted nuclear war to make much of a difference.

Economic growth is the most interesting one. Predicting 10 fold increase in wealth feels bold in the current climate, but that's just 2.5% a year. Anything under 1% is declared a disaster by the media. Many developing economies scarcely felt the 'Global Financial Crisis' of 2009 and are still growing at rates well in excess of 2.5%. A lot of this growth is probably technologically driven, but given that a very sizeable minority of the worlds population have yet to benefit from the inventions of the 20th century, even if we invent nothing of note from now on, there is plenty of scope. There are limits to economic growth, particularly in the physical 'economy of stuff'. There is only so much physical stuff we can own or consume, and so much physical stuff we can make. The sustainable economics of information, with a marginal cost of zero, is still a subject of debate. It is bold, but not implausible, to suppose that whatever the overall level of growth in the 21st century, the regional inequalities that arose out of the industrial and colonial age will largely even out.

Good quality Telepresence seems like a no brainer - an engineering problem, with an obvious economic payoff for a solution. I can't see how it couldn't come to pass, all else being equal, although some forward advances in the affordability and speed of physical travel might delay it's adoption. Any kind of adverse system shock, like a high mortality pandemic or war, would hit transport nets more and promote telepresence.

Artificial Intelligence is probably the rockiest potential megatrend. Others might not even consider it - it will arrive as a Black Swan to many. Moore's law is a classic inductive case. The long established historical trend doesn't prove it will continue for one more day, no more than the Turkey feeding schedule does. That said, some of the worlds biggest R&D budgets are feeding this turkey and making ever more powerful chips. It's also noting that the law refers to the cost of a unit of processing power, not pure component density. Even if you hit physical  wall on component density, as is frequently prophesied, there is still plenty mileage in making chips cheaper, and with better utilisation making the power cheaper at the desktop. Think of the difference between a chip on a desktop PC, spending most of it's life running screensavers and Freecell, and the same chip in a server farm, running at optimum 24/7/365. Granted it might be running Farmville, but you get the idea. If anything, the risk for Moores law is that something like Quantum Computing will come out of leftfield and rapidly accelerate the curve.

Raw processing power doesn't necessarily mean human like artificial intelligence, but it will chip away at it, until the differences seem pedantic. When the machines can answer the phone, read your x-ray, drive your car and write your essay for you, who cares whether it's 'real' or 'fake' AI.

Overall, Technology seems to be the most risky quarter, with the potential to pull out Black Swans like rabbits out of a hat. It seems likely that they would be 'positive' Black Swans - developments in unexpected places, rather than negative Black Swans, when expected things fail to happen. That said a negative black swan, like perhaps an elegantly tailored bioweapon, cannot be dismissed.

Of course, Taleb would no doubt argue that focusing on these four trends is missing the point. The key trend, the main event that will shape tertiary education in the next century isn't any of those - it's something we haven't -can't - imagine yet, some surprise out of left field that will change the narrative, for better or for worse - Fifth Megatrend, the Black Swan.

Friday, April 30, 2010

Why Predications Fail.




You can't talk about the future of Tertiary education without making predictions. If you are not prepared to put your neck on the block and say what you think is going to happen, it's a pointless exercise. Predicting puts you in harm way - almost all forecasts about the future are wrong.
Before I make too many predictions, It's worth reviewing why predictions fail.
1. Failure to account for economics as a key driver, rather than technology. 
Flying cars are a great example here. People think because a thing can be done, it will. We don't go to work in flying cars because they are impossible - they're not. They're just to expensive, and it's cheaper to go by road.

2. Failure to consider human factors and rates of change. 
A thing can be done a better way, but you have to wait for the old ones who do it the old way to die off first. This is especially important in Universities, where the great old ones live long.The paperless office comes to mind as an example. I've worked in one, it was great. There weren't any old people there who liked to print things off and scribble on them.

3. Predicting out of area of expertise
People who are experts in one field assume expertise in others. Artists imagining space travel is a nice example. There are nice paintings, but I'm not flying in that, thanks. A lot of people fail on technology driven predictions here, often the devil is in the details. Another reason we don't have many paperless offices is that until recently, the screens just weren't good enough, and the software tools for easy annotation weren't either. Generalists sweeping past miss those kinds of details.
4. Failure to account for changes out of area of expertise
This is really the converse of the above reason. There is a tendency to assume that changes you know about will dominate, and changes you do not know about are unimportant. For comparison, consider Ray Kurzwiel Singularity work - focused on technology with George Friedman's 'The Next 100 years' book - focused on geopolitics. Both are fine pieces of work by experts in their field, well argued and, like all predictions, probably wrong. Both focus heavily on developments in the authors own area of expertise, and miss, or err, on key topics outside the field.
5. Wishful thinking. 
Confusing 'We Can' 'We Should' and 'We aught" with what will probably occur. A prediction is not a wish, or a hope, it is a cold, rational analysis of what is likely to occur, whether we like it or not.
6. Predicting the Weather, not the Climate
When people think of predicting, they think of predicting earthquakes, or the stock market, or the weather. You can't predict these in any detail. They are essentially random noise in a pattern. You can predict where earthquakes are likely, that stock markets will exist and be useful, and that there will be weather. Very specific predications often fail not (just) because they are more specific bets on a random future, but because they are attempting to predict things on too fine a scale. Big trends have a mass, an inertia to them that is often the elephant in the room, too big to see. How the big trends collide and play out is important, but as humans we get lost in the human scale details. It's said no one predicted the First World War (and yet, every general staff in Europe had a plan for it for decades). It's true we couldn't predict the details of it, but history tells us that Great Power wars happen a lot, and technology and economics could tell us they would get bigger, and meaner. In the big scale of things, who fought, who won and who lost, the "Battles and Kings" school of history, don't matter so much. What was important to predict was that there would be battles and kings.

Number 5, Wishful thinking, is about the only one I'm confident of avoiding. Point 2 (rates of change) and point 6 (scale of prediction) are always going to be tricky, and the others all rely on having the right spread and depth of expertise - knowing enough about enough things to get the big picture right but not miss sneaky details.