If you truly believe you have a far better way to teach physics, you really should let the world see how you do it. Make Youtube videos and point us to them. We're looking for other great teachers that are consistent with our mission and resonate with students.

Given some of the criticism of your YouTube videos, I agree with your simple advice: Let them make something better and let the viewers decide.

The Khan Academy does not have some kind of monopoly on providing education content to help people of any age learn. Rather, the Khan Academy was simply the first to resonate with so many people around the world in such a short period of time. You clearly have many people's attention.

The Internet is an open medium to provide content, and to those who think they can do better, they know that they don't need much to get something up, as has been made very clear with Sal and his use of a simple drawing tablet, screen recorder, and headset. So make it happen -- show how your content is vastly superior to Sal's that both educates and makes it fun. Oh yeah, and free.

Sal, as you have noted a few times now, you are actively looking for other people that posses your talents to educate people. Have you thought about creating an open competition on your site and let the public vote on what they think is good content and fits the "Khan Academy mold"? Just a thought :-).

I think torque2's point is that the way they teach physics cannot be encapsulated in a YouTube video. Physics teaching based on the last 20-years of PER requires engagement of prior conceptions and inquiry-based activities (to name a few key findings). This cannot be done in the type of teacher-centric videos you're giving. I'll admit that your lectures are awesome... but they're based on a learning traditional that has been proven to be ineffective relative to student-centric environments by a couple decades of education research.

I had a serious question that I hope you'll answer: "When you learn the physics required to make a video, do you just watch MIT lectures or do you go out and actively search for and build your own understanding of the concepts? Do you talk to other people about the ideas you have and what they think about the subject? In short, do you have a conversation with the material, or is the material hand-fed to you from a video?"

Thanks for all your hard work Sal, it definitely has a place in a learner's experience... it just might not be the place it's currently being touted as in the media.

And because something is popular doesn't mean it's good. Take modern music. A lot of popular singers attract crowds of people and make tons of money. But very often the music just sucks. And people forget these pop idols as fast as they make new ones.

Page hits isn't a very good metric if you're trying to get feedback about what viewers actually thought of the videos. A lot of people have heard of Khan Academy through the news or may land there via a search engine. How many of them watch the video all the way through? How many stick around and watch parts II, III, etc.? I could be totally wrong about this (please correct me if that is the case), but my suspicion is that YouTube's stats probably aren't sufficiently granular to distinguish between a page visit in which the viewer watches the video in its entirety (suggesting that the video was perceived as worthwhile) and a visit in which the viewer watches for a little while and then bails (indicating that it was not perceived as a valuable experience). User comments are subject to a great deal of self-selection bias. Viewers who thought the video was the best thing since sliced bread are way more likely to take the time to login (if already registered) or register and login (if not already a registered YouTuber) than those who thought it was a waste of time.

There's also a more fundamentally important question to consider: Did the viewer actually learn the material or were they simply duped into a false belief in their own competency? HN regularly features posts about interviewing techniques aimed at sorting out real programmers who can write code proficiently from people who simply possess a false belief that they are programmers. Sound pedagogical methods require some means of verifying that the student has, in fact, built a valid mental model of the subject matter and, of particular importance in the sciences, is capable of applying their knowledge to reason about novel situations. How do you know that your students actually understand the material? Do you have any way of distinguishing between students who have had a previous formal learning experience with the subject matter and are simply looking for a refresher course, students who are concurrently receiving other instruction (the teacher is using Khan Academy as a supplement), and students who have had no prior exposure to the material?

Or at the very least, please watch the whole video because it explains one massive problem with video science lessons and how to partially fix the problem.

http://youtu.be/eVtCO84MDj8

The irony is also thick with him using a video to try to explain that videos can't explain things.

First of all, he says all kinds of positive things about the work you're doing. "Impugn Khan Academy"? Really?

Second, he is explaining physics education research that he himself did for his Ph.D thesis. He's using examples from his own work to make comparisons in technique to what worked and what didn't work in his own, actually objectively tested research. This is not "a small interview of a handful of people".

Third, he's not saying videos can't explain things - he's saying that they often fail to make students question their own pre-existing misconceptions.

I hope you actually watch the whole video and catch the research context? I mean I am handing you a research-tested science education technique that you could easily incorporate into your own videos if you actually wanted.

The study was, as best I can tell after reading through much of it (a lot not relevant to this discussion), quite limited. The largest number of students considered for any of the experiments: 370. All from one school. Most of the other experiments included far fewer students (fewer than 100).

Chapter 9 (additional experiments in chapter 10), and the section on Participants and Design has most of the relevant information. I was particularly fascinated by the decision to remove students from the final result for "watching more than one multimedia treatment." I mean I understand why it's important to control for this, but part of the point of KA is to provide additional treatments of information.

I am not saying that this in any way invalidates the research, but I also don't think it's fair to say that this is somehow a surefire tested solution that is being handed over. In addition, because the research hasn't been repeated outside of the original setting -- at least I couldn't find it, please correct me here if I'm wrong -- it makes the claim that "because introducing misconceptions was effective in this study means it must be effective in all science videos" a pretty serious stretch.

I'm all for research. We're doing a lot of it right now with our pilot students/classrooms and existing users. But just because something has been research-tested doesn't mean it's automatically the right thing to implement for KA or its students.

On a more serious note, however, as a science educator, I appreciate the Khan Academy's work to make these very clear educational resources available to all. They definitely can be useful in many ways. Now if only my district would un-block YouTube...

However, I have to agree with many points Frank and others bring up. I have taught for 5 years--my first 4, I worked relentlessly to provide thorough and clear explanations for all the science concepts I taught. And I was baffled when I graded my final exams and found that my students learned little to nothing from 9 months of clear explanations and practice. Last summer, I learned about Modeling Physics (http://modeling.asu.edu/) and implemented it in my physics and chemistry classes this year. Now, my students have rocked their final exams and shown huge improvements from pre- to post-test on standardized instruments. This is simply my experience, but there really is significant physics education research which points to the same conclusions.

Educators are not trying to belittle or denigrate anyone. However, we have to protest when so many resources and media attention are being given to a method of instruction that has been shown to be rather ineffective, particularly in science, when incredible alternatives are available but are, unfortunately, largely being ignored and unfunded.

With respect to the Khan Academy, it would be interesting to see tools that provide a kind of interactive exchange of preconceived ideas and then helping to break them down. Having tests after the exchange would then help provide great metrics on the retention and understanding of the subject.

In any case, 1veritasium has at least provided an thoughtful response to the Khan Academy. He recognizes a good thing but provide some nice constructive criticism that could certainly improve what the Khan Academy already could offer. So as someone who enjoys Sal's videos, this idea gets a +1 from me... You know, as just some random person on the Internet ;-).

The whole point is not who can make better video lectures. The point is why lecture at all?

I wish the world would see my YouTube video about Modeling Physics (and resources) here: http://bit.ly/ModelingPhysics

I wish the world would see my 13 other videos showing Modeling Physics in action: http://vimeo.com/channels/modelingphysics

I wish the world would read my post on pseudoteaching: http://bit.ly/MITpseudoteaching

I wish the world would read my posts on standards-based grading, which allows my students to show growth and mastery in physics: http://bit.ly/SBGposts

One of my pseudoteaching posts talks about how forward thinking universities like MIT (your alma mater) are switching from lecture-based physics to a more interactive model like Modeling. The research on Modeling is there, please don't ignore it: http://modeling.asu.edu/modeling/Mod_Instr-effective.htm

More research --> The 3 key principles from "How Students Learn: History, Mathematics, and Science in the Classroom" are:

#1. Engaging Prior Understandings - "If students’ preconceptions are not addressed directly, they often memorize content (e.g., formulas in physics), yet still use their experience-based preconceptions to act in the world.

#2. The Essential Role of Factual Knowledge and Conceptual Frameworks in Understanding - "What novices see as separate pieces of information, experts see as organized sets of ideas."

#3. The Importance of Self-Monitoring - "Appropriate kinds of self-monitoring and reflection have been demonstrated to support learning with understanding in a variety of areas. In one study,15 for example, students who were directed to engage in self-explanation as they solved mathematics problems developed deeper conceptual understanding than did students who solved those same problems but did not engage in self-explanation."

You can read the entire book for free: http://www.nap.edu/catalog.php?record_id=10126

Modeling Physics does all 3. Your videos are 0 for 3. Badges do not count as "self-monitoring" in any meaningful sense.

You also have no hard data (as of yet) as to the effectiveness of your videos. Just anecdotal comments from students. You have no controlled studies. In fact, controlled studies show that video lectures are rather ineffective. See Derek Muller's video abstract and research: http://bit.ly/KhanEffectiveness

I'm not trying to be mean. I think your videos can be a resource for some teachers/students. But using video as the primary method for content delivery is ineffective.

We should not have to "flip" lectures and HW in order for students to be more interactive in class. When inquiry is done right, the interaction is already built in.

I'll stop there, before this gets too long.

Thanks for your time, Frank Noschese (aka torque2)

I would assume that studies comparing a competent physics teacher to a video instructor would show the former being more effective.

However, there is a shortage of competent physics instructors. There is a (possibly bigger) shortage of competent maths teachers. I am speaking from my more extreme experience in South Africa, where most of the teachers are abysmal. Expensive video-based instruction helped me achieve a respectable mark in physics/chemistry, most of my peers lacked that privilege and did very poorly. Even at university, things like differential equations, linear algebra and calculus were not particularly well taught - most professors are not really good at teaching, and in my case, they were teaching in a second language (Afrikaans speakers teaching English speaking students). The Khan Academy would have helped me even at the undergraduate level.

I have posted to HN in a related thread about video instruction in South Africa [ http://news.ycombinator.com/item?id=2351100 ]. People have been doing it for years here. I would recommend a controlled study in South Africa using KA, or The Learning Channel videos by William Smith (NOT run in an elite private school, but at an average township school) before we write off video based instruction as "ineffective".

One item in particular that hasn't been much discussed here is the self-paced exercises. In here, students can try a problem, to see if they understand the content; and they keep trying until they get 10 in a row. The Khan Academy logs data about how many tries and how long it takes, so they have data on what they consider to be "mastery" (this may be an oversimplification...) There is also a certain game-play aspect which I think is kinda cool, and I like the map of linked concepts/exercise sets. As far as I see, they only have math problems up there as of now.

At face value, it looks like plug-and-chug, drill-and-kill methodology that irks any teacher of merit. But: there are opportunities to take this infrastructure and make it so much stronger ("formative" vs "summative").

Diagnoser.com is an example of an online exercise bank that includes aspects of several decades of physics education research (also some chem and life science) but the concept could be generalized for other disciplines as well. Each question includes "distractors" (wrong answers) that appeal to certain common misconceptions (what they call "facets" of understanding). If you get it right, great! you go on to another problem that might pose a similar question with a twist to challenge you. If you get it wrong, it recognizes from your answer what your misconception might be, gives you a quick mini-lesson to challenge your thinking on that, and gives you a related (though not the same) problem to see if you get it (for physics teachers: like an interactive FCI). The data is reported to the teacher, including right/wrong answers, and the "facets" that correspond to certain ideas and misconceptions. The teacher can see where their teaching (or in this case, Khan videos) have been effective, and where they could use more work (perhaps suggesting the student watch or review a particular video). NOTE: this is only one aspect of the Diagnoser Project's larger program, so I have taken it out of context, but it seems appropriate for this discussion.

This formative approach is effective especially for the conceptual problems, not just the plug-and-chug problems that I saw on the KhanAcademy.org site. Yes, this is much more difficult to program than "right/wrong, next!" problems, and requires research to write good facet-based "distractors" (take advantage of years or decades of education research out there!) but it (a) gets more useful data than just "right/wrong" and (b) allows students to act on their wrong answers immediately by challenging them to think about why it was wrong. I think incorporating something like this with Khan's gameplay/scoring/data collection methods would make the exercise feature much stronger.

What's curious about this critique is that it sounds almost exactly like what Khan advocates in his recent TED talk (here:http://www.ted.com/talks/salman_khan_let_s_use_video_to_rein...)

In it, he advocates flipping the traditional school model of passive lecturing in the classroom, followed by problem solving during homework, and instead use videos like his to allow the students to view the lectures at home as homework and then spend their class time interactive with the teach and other students in active learning and problem solving.

My take on the assumptions: 1- The best (and research proven - google physics education researcy) methods are not direct lecture (aka traditional lecture / sage on the stage etc...). The best methods involve a process commonly referred to as Interactive Engagement (IE). Using these methods, students interact with each other, the instructor, and the appropriate materials for the class. Me discussing things, working some samples, asking the students to work some samples... that is the traditional approach. It is only minimally effective at changing the way students think. There are 2 links below that go more fully into the interactive education approach (backed up by Physics Education Research - PER).

2 - The assumption that videos can't work is also problematic. They can be useful. However, if i create a video in which i: discuss things, work some samples, ask the students to work some samples... well, since that is only minimally effective when I am there and able to respond to questions, it is even less effective if I am not there and the student can only watch. If the video is made in a way that addresses common prior-conceptions, then you've got a shot at changing the way people think. However, it is not effective to simply state the common prior-conception is wrong. A common example, beginners think light bulbs use up current. The first bulb uses some, the next uses what’s left. Tell them that is not the case and they will parrot it back. When they test on it, or when it is phrased or needed in a different way, beginners go back to their initial conception. The approach of, “they just need to listen” doesn’t work. A common way in which we address prior conceptions (I think originated by McDermott at UW) is to show a couple of student reasoning examples that attempt to explain a phenomena. The current student chooses if student A, B, or neither is correct. Then that same prior-conception is hit over and over in a variety of ways in an effort to truly overcome the prior-conception and establish / replace / adjust it so that the student’s concept of what is going on agrees with the accepted evidence. Again, if I make a video of traditional teaching, it will (at best) be as bad as traditional teaching. Putting it on video doesn’t make it better. This would be, essentially video Cliff notes.

3 – Feedback… This can be messy. There have been an odd number of studies that check to see if student feedback is helpful. An MIT study showed that students gave very poor reviews to a new teaching method for E&M. These are MIT kids that should be pretty bright. Pretty self-aware of their learning (one would think). The overwhelming response on reviews of the new approach was that students preferred the more traditional course structure and felt they learned little. The results were that the students in the new course (that students didn’t rate highly) did significantly better than the students in the traditional course.

Two sources of information that will help a truly concerned person evaluate these ideas in more depth. <http://www.phys.washington.edu/groups/peg/rl.htm>; <http://webcache.googleusercontent.com/search?q=cache:http://...; The 2nd link is a cached copy.