Journal of Artificial General Intelligence

Thank you for the opportunity to read the subject paper. A comparative study should be accurate and fair about the alternatives it compares with its author’s proposed approach, here, “NARS.” My chief concerns about the paper are its false depiction of Bayesian reasoning, and how its adverse comparison between Bayes and the author’s system contradicts its favorable comparison between that system and a formalism that barely differs from Bayesian practice, if it differs at all.

In this paper, the author discusses several approaches to handle and measure evidence - classical, one-number and two-number approaches, and finally shows how evidence is handled in his system NARS (Non-Axiomatic Reasoning System). His major point in the paper is that two-numbers are necessary to measure evidence adequately.

There are two general problems with this paper, and what its title suggests: First, this is not a paper on questions like what evidence is, how it should be treated properly (e.g., as opposed to facts and generic knowledge), how to treat uncertain evidence, how evidence is obtained (e.g., by observation), how to be sure about the independence of pieces of evidence, and a lot more. This paper deals primarily with the problem how evidence should be measured. Second, large parts of this paper deal with problems that arise when conditional statements are taken as material implications, and that have long been discussed in conditional logics and nonmonotonic reasoning. So, the author's insight (p.6) that "we have seen that in the classical logic, the concept of ... inconclusive evidence is hard to introduce" is not a new one. He then turns to default logic, but the proper logical concept to be turned to here would be conditional logic. For instance, the three-valued approach to conditionals by de Finetti (first published in the 1930's) would already resolve some of the paradoxes investigated in this section. In this respect, some of the analyses of the paper are a bit fallacious and out of date. The topic of evidence should be discussed in the context of uncertain reasoning, but not with a classical background. And truth should not be confused with belief.

Having understood this, the question is what contribution this paper makes to the area of general artificial inteliigence. Its support for two-number approaches and the presentation of NARS are indeed interesting, but the author should be more precise and informative here. It's clear that two numbers can express more than one, but one has to be sure what kind of information the numbers express. At the end of section 4, the author uses the terms "stability" and "direction" here, but I don't see how exactly the approaches dealt with in this section fit into this categorization. For instance, if d(B,E) = {m,n} (p. 14), what is stability and what is direction? Numbers should be given a clear meaning, or even semantics, in a logical environment. For instance, on p. 14, a "combination rule" is derived; here, the author should explain, how it is derived, and what it means.

The author's system NARS is claimed to be a logic, but its syntax and semantics are only vaguely introduced. What exactly is the meaning of the numbers (e.g., as compared to probabilities or Fuzzy values), and what formal properties can be shown for reasoning via NARS? Examples would help here a lot. NARS seems to be quite ad hoc, and the lacking coherence is irritating. It seems to be more like an argumentation system (that might be incoherent, but has some strategies to resolve conflicts). And, indeed, a comparison to argumentation would be very interesting.

Overall, I recommend another major revision. The author should mention conditional logics as a proper way out of some of the presented paradoxes, and should give examples for reasoning with NARS. Moreover, he should be more precise on the points (see also below) that are addressed in this review.

 Some minor comments:

Define conclusive vw. inconclusive evidence, perfect vs. imperfect inheritance relation

On p. 11, the author claims that "It is not enough to use a single probability distribution for representation." What exactly supports this claim? Section 3 is more on ignoring the difference between evidence and background knowledge, and on the confusion between material and conditional implication, than on single probability values.

On p.13^4f, I don't see why "given the definition of weight of evidence and belief function, the additivity of the former does not correspond to the Dempster combination of the latter." By the way, the Dempser combination rule should be recalled here.

Review of Pei Wang’s “Formalization of Evidence: A Comparative Study”
Revision 3

Compared with the original, I see little improvement in the current draft. In a “major revision,” a submission is fundamentally rethought, so that it may be substantially rewritten, not merely cosmetically altered with pro forma accomodations to the particular examples that referees may have used to illustrate the existence of shortcomings. I recommend that this paper be rejected, without prejudice to some future submission de novo

Reply to the second review of Paul Snow

The following are replies to the major points in the review, cited as quotations.

1 Whether there is anything new in NARS

My decision to accept the paper of Pei Wang is based on the following criteria: Already the first version of the manuscript did meet many criteria for a high quality paper. The paper was well-written, had a clear organization, was original and related to the state of research in reasoning. But, although not an expert, I felt unsure about some statements concerning Bayesian reasoning. Therefore, I selected two experts in this domain as reviewers. Both reviews were very detailed and commented on a lot of technical points as well as some points of general perspective on Baysian approaches. Pei Wang addressed all technical points raised in the succeeding versions of the manuscript. In my opinion, now the paper is technically sound and the remaining comments of the reviewers which concern the evaluation of the Bayesian approach in relation to NARS are addressed in such a way that the perspective of the author becomes clear. I want to thank the reviewers very much for their detailled comments on earlier versions of this paper.