Sep 162014

Someone posted in an instructional design forum the following question: If learning is not trackable is it still learning?
Below is my somewhat modified answer. I thought I would post it here too, as I have never seen this topic discussed in an ID context and people seem to find it useful.

If it is (in principle, for nobody, ever!) not possible to detect any traces that some sort of learning have occurred, then – of course – no learning in any practical sense – has occurred.

The question “If learning is not trackable is it still learning?” is a great example of what some call an ill-defined question/problem. Ill-defined questions are surprisingly common. They usually occur as a result of a good idea and intentions, but sloppy or rushed phrasing (it happens to all of us!). Often this can lead to wasted time and resources when people get lost in the hopeless search for correct – but impossible or meaningless – answers.

However, sometimes it is fairly obvious what the asker wanted to know and the inaccuracies in the phrasing don’t matter or even go unnoticed.
Chess players know this phenomenon as a “double blunder”. Both sides messup (one made a mistake, but the other one did not see it) but in practice it doesn’t matter since no-one noticed.

I think this may be the case with the above question too. However, it is not quite that clear to me.I think the OP probably meant something like “If learning occurs but could not be detected by the usual methods (often only automated summative assessment), can it still be called learning?” Perhaps she also meant “… can it still be a successful learning program?”

“Successful”, would also need clarification in this context. The learner may have learned something for “herself” only. Or perhaps the learner could have learned something that is useful for “the company” that sponsored the implementation of the learning program. Or both.

The great thing with these types of questions is that it can be easy to find the answers simply by realizing it is an ill-defined question and trying to formulate the question more precisely. It is not always so easy to spot these nasties though – or find the right phrasing. This is where good critical thinking skills are helpful.

Identifying and avoiding ill-defined questions is something I found extremely effective when designing and improving learning programs.

I have been chasing these tricky guys for years and still create or overlook them more often than I should. Nobody is safe from them and they seem to like hanging around me a lot.

The term “ill-defined” is also not my choice but a common term in areas like critical thinking and problem solving. We could also call them – perhaps more respectfully – “inaccurately formulated questions” or perhaps “misleading questions”.

Occasionally these questions do have useful (side) effects or can be used purposefully. For example, in order to trigger higher level learning experiences by trying to mislead the learner first, then let him discover his own mistakes and flaws in his original method.

The ill-defined question in this example could have such side effect. I certainly learned and refreshed a few things already while reading, writing and reflecting during this post. Thank you, ill-defined question.

Happy learning,


Keywords: exposing ill-defined questions, instructional design tips

Sep 162014

Ask George, an average person, what questions he has about quantum mechanics. I would guess he could only formulate one question – probably something like:

  1. What is quantum mechanics?

He may soon learn that quantum mechanics is the science of the very small: the body of scientific principles that explains the behavior of matter and its interactions with energy on the scale of atoms and subatomic particles.

After learning the answer to his single question, George will look for answers to questions such as:

  1. What are the scientific principles?
  2. What are examples of subatomic particles?
  3. When do things get small enough?
  4. How small are atoms?

While some answers may be answered to his full satisfaction without generating more immediate questions, answers to the more complex questions will most likely create new questions Рoften more than one. In other words, just a a few answers tend  to create a lot of questions.

I had many questions when I began to study physics. At the end of my studies I realized I had even more questions than at the beginning.¬† I did not expect that. First, I was disappointed. I even feared I had failed to learn things well enough. It was the moment when I became aware of my interest in “questions”.

I was fortunate to be surrounded by many knowledgeable people for many years. Both Cambridge, UK but also Silicon valley, USA were hotspots in that respect. One thing I noticed was that people I liked to learn from where those who also had questions. They often said that they do not know something – or they would love to know more about x.

Those who did not express their questions typically also did not share much of their knowledge. Perhaps it is harder for reluctant communicators to accumulate and create reliable knowledge? I could not help but suspect they did not know as much as their question loving peers. Occasionally I could confirm these when they could not answer the questions I had.

Later I found that question-lovers tend to be more knowledgeable in practically any area (where they ask questions..) – not just in academia.

Recently I asked my kids who they think knows more: a person who is not afraid of asking questions or one who does? It got them to think. I am pretty sure there are exceptions. However, as a rule, I rather ask if I feel I can learn something.


Keywords: asking questions, good questions, fear of asking a question