FM75, on 2014-July-03, 16:46, said:
1) They do not supply their data for independent review.
2) Their experiments are not "repeatable" (exactly) - can't reproduce the exact deals that they used.
3) Their experiments are not "repeatable" (even approximately), because they do not specify the constraints imposed on the deals.
4) They do not compare their results properly against a "null hypothesis". Was it better than the alternative - with respect to both correct predictions as well as false positives and negatives? By how much?
I laud the approach. I have criticized elsewhere the poor proof-reading - and obviously the errors missed in the second publication.
Kudo's for a scientific approach to the problems.
1. There is no set of established journals or conferences on bridge theory. We, the bridge community, are the independent review.
2. Their experiments are repeatable; running double dummy sims is easy to do on your own. This is often the case with processes which are somewhat random -- you won't get the exact same result, but the long-term trends should be the same. Further, I don't think an exhaustive list of their coin flips (deals) is particularly useful in examining an analysis of a stochastic process, nor do I find that such listings are typically provided in academic papers.
3. They do specify the constraints on the deals in the books, at least in most cases.
4. I am not sure what "alternative" you want; it is not like opening leads are an exact science or follow established rules. At various points they do state null hypothesis (lead 4th from longest and strongest, lead top from three touching honors but low from two touching honors) and show that these are incorrect.