Kurzweil makes some good points here. He doesn't address each and every criticism with his view of AI progress, but he does a good job calling out Paul Allen on not doing the homework.
There are some good points in Kurzweil's response, but the ones about Paul Allen are definitely not them.
I think his best point was about extrapolating function from individual cells or structures, without needing to understand every single cell or structure individually.
I disagree. When one "addresses" a long-standing, well-thought out argument with off-the-cuff snap statement or something only slightly more thought-out, it is perfectly permissible to call them out as not being serious. It's like going into a serious religion debate with "Evil exists, therefore God does not. Ah-ha, you are defeated!", or "God must exist because something must have started it all, so there!" as if in the thousands of years of years the debate has been raging on nobody has ever thought of those things, or addressed them at length, in both directions.
If you're going to debate the singularity here, maybe you can get by with just stating "I don't believe it's possible" without citing any logic, which as of this writing there's at least two people in this comment set already who have simply stated that without defense, but if you're going to debate one of the leaders of the field it would help if you would at least grant your opponent the courtesy of thinking that just maybe over the course of the decades he's been thinking about this, the obvious objections that you thought up in five seconds just might have been addressed at some point. You may not think they've been adequately or correctly addressed, but don't pretend they haven't been addressed at all.
Personally I'm not completely sold on the matter for a variety of reasons myself, but the usual logic given for why you should be skeptical about it is terrible. The interesting questions are a great deal more complicated than something that can be dismissed with something that generally boils down to "Look, I just can't imagine the world changing that much, so it won't".
I think your comparison to arguments about religion is apt. Many of the opponents to Kurzweil's ideas remind me of those whose opposition to the possibility of a godless universe amounts to, "I can't imagine it, so I don't believe it."
On the other hand, Kurzweil (at least in his essays and articles) often ignores the question of what a fair null hypothesis is for the possibility of the singularity. I think his gift for creating a compelling vision tends to make people forget that the null hypothesis for a scientific assertion is doubt.
Kurzweil provides both high level general evidence (like improvements in computation) and low level, domain-specific evidence (like the discussion about the pancreas) to support his claims, but none of that justifies the use of the word "law" in "law of accelerating returns". He attempts an analogy with thermodynamic laws and how they are derived from underlying statistical principles, but there are no underlying fundamental principles of human innovation and progress that are in any way comparable to the certainty and universality of physical laws. This, I think, is why a lot of people (myself included) have a hard time taking him seriously. He tries to apply the same kind of formal analysis that works well in science to human beings and the complex, highly non-scientific processes that underly innovation today. The bottom line is that, until the singularity occurs, human beings will still be needed to build ever more complex and powerful systems, but human beings do not progress at anything close to an exponential rate.
I guess my point was just that calling him out, while strictly correct, is doing nothing to refute whatever statements he may have made on the singularity, nor is it doing anything to promote the idea of the singularity.
Replying to "evil exists, therefore God does not," with, "that argument has been made before," doesn't say anything about whether God does or doesn't exist, and it doesn't do anything to refute the argument.
The point of PG's How To Disagree is that the point of disagreeing is to address the truth or falsehood of a proposition, not the way the proposition is presented.
The DH# levels (at least DH0-2) can be restated something like this:
DH0: I don't like the author, therefore his argument is false.
DH1: The author has a relevant fault or weakness, therefore his argument is false.
DH2: The author didn't present his argument well, therefore his argument is false.
While it's nothing against him personally, the first three paragraphs of Kurzweil's reply read like "the author didn't present his argument well, therefore it is false." He recovers later, sure. He easily reaches 5-6 on the DH scale, but the rebuttal would be stronger without the first three paragraphs, and they certainly aren't the best, or most praiseworthy, part of it.
Allen attacks Kurzweil rather directly in his essay and the essay title is even "The Singularity Isn't Near". So in that case I think it's fair game if Kurzweil calls him out for not even regarding the arguments he made in "The Singularity is near".
