Please consider supporting this podcast:

Transcript:

[00:00] Michelle: All right. I am so excited to be here with Professor John Kidd from UVU. I was so thankful, I sent out lots of requests to professors to be willing to talk to me. And John Kidd responded and he has been fabulous to work with just so kind and willing to generous with his time and his expertise and I think, I think he’s just brilliant. So let me introduce you. This is Professor John Kidd from UVU. He did correct me if I get any details wrong, but he did his undergrad and his masters at Utah State, my husband’s alma mater. He will be very happy to hear that in statistics. And then he went on to do a phd in biostatistics from the University of Carolina at Chapel Hill. And he is now up

[00:49] John Kidd: North Carolina, just North Carolina, North Carolina. I say it’s you got to be careful out there because if you, you know, we’ll talk about the Carolinas, but there are some divisions. And unless you’re talking about the Carolina Panthers, those divisions tend to be pretty important.

[01:07] Michelle: Ok. I’m ashamed. I can’t believe I did that. My son is in North Carolina. On a mission right now. So I should have gotten that right. So, at Chapel Hill in North Carolina, thank you for the correction. Calling me out on that. And I believe you’ve been at UVU since 2021 tormenting your students with the same mathematical joy that you got to experience.

[01:30] John Kidd: I don’t think I’m tormenting all of them. But yeah, I’m fairly certain, quite a few, especially those that just took an exam from me today. They’re, they’re probably feeling a bit tormented.

[01:39] Michelle: I, I think you should show everyone your water bottle. You just showed me. It was very fun.

[01:47] John Kidd: This was a wonderful gift from my wife to replace the original one that my mother bought me. That was stolen. So,

[01:54] Michelle: oh, that’s great. So anyway, John and his wife have four Children and I know one keeps you very busy with that. You have a special needs child, which just my hat’s off to parents of special needs kids. It’s a very different thing. So that makes me even more appreciative for the time that John has given to me because I know he does not have much of it. And then he just told me he also is a runner and a marathoner. I believe you’ve run four marathons, which is so awesome. I’m a runner but never a marathon. So that’s something I admire.

[02:28] John Kidd: It’s definitely something you choose and, and I cannot, I cannot honestly say I’ve ever finished one and said, I feel great. I, I typically burst into tears but that’s for, you know, I’ve accomplished this but then I get finished and go. Why did I do that? Why did I? But then I talked to my wife and I’m like, yeah, I want to do at least one more. And she looks at me like, are you crazy? I’m like, hey, you said, I said the same thing to you after we, each of our Children when you went. Oh, why did I do this?

[02:55] Michelle: That’s good. Four kids and four marathons. So yeah, we’ll see who, who runs more in the end, right? Who comes out on top. So

[03:03] John Kidd: I got her into running. So we’ll just both switch that one. No, she’s done a couple of half marathons but uh I think she’s good with that.

[03:10] Michelle: That’s great. Yeah, that, that would be about my limit, I think for my knees. We’re like, why are you doing this? But um anyway, ok, so John, I have to again, thank you. This has been awesome. We’ve been able to talk, well, we’ve emailed a couple of times and then have one con previous conversation. So I asked John to come and talk to me basically about the claim that land deeds are. Well, I know that this is a um a word that’s antithetical to your profession, but the land deeds prove that Joseph Smith was a polygamist and, and so I wanted him to kind of give us an overview of what would be considered a valid or compelling statistical analysis, what information you would need and maybe why we shouldn’t look at this quite that way quite as the smoking gun that I know that Bill real wants to claim that it is. So thank you John for being here.

[04:07] John Kidd: You’re welcome. Thanks for having me.

[04:10] Michelle: So do you want to take it away or? Yeah, you go ahead.

[04:13] John Kidd: So I say, yeah, I think even first and foremost for statisticians proving anything is going to be a difficult process in general. Um We, we do science in order to uh try to find truth, but the field of statistics takes into account and is based around the idea that anything you look at, you know, there is variability, there’s error and measurements that we take, that things are complicated. Um Every model, you know, I can write out math formulas for things we do. We always account for the fact that we’re probably not understanding everything that’s happening. And then to get to the end, we do make conclusions and say, OK, this is this is the approach or this is the conclusion that we make. But in all of those cases, we also recognize from the get go or at least should recognize that there’s a chance that we make a mistake. So even the field of statistics in general, we talk about uh different errors that we can make and the moment that we start doing a test, there is always a chance we’re going to make an error and we recognize that we accept it, we tried to balance things out. And so being able to say, prove from the beginning, that’s going to be difficult. Uh Additionally, statistical approaches uh in truth, some of the claims that are made of saying uh this is, you know, because we see this type of thing happening, that’s going to prove something happens or the probabilities of this happening are incredibly small. I think it’s difficult to make claims based on that in the end to if you know, to set kind of the baseline and the standard for this, if there are no circumstances at all, then you know the scenario, if you believe that nothing can occur, that there’s no possibilities, then observing even one event is theoretically in that case, proof enough, you know, if it is impossible for the sun to have crashed into the earth, then observing the sun crashing into the earth is enough to disprove that immediately and you can prove, you know the opposite, but that, you know, in the end, that’s not really a statistical point. And so to try to approach some of these things from a statistical point are going to make are going to require a lot more assumptions and materials and things along those lines. So kind of

[06:47] Michelle: so if I’m understanding you, so so just so maybe statistics isn’t the correct tool for this analysis because there are so many unknowns that

[06:59] John Kidd: and I mean, what, what might be the correct tool I think is one that’s going to be a difficult thing to put on, you know, to, to define in the end. Uh anyway, um One thing also to consider in statistics, the basis behind most of what we do is the idea that we’re taking what we call a sample And from that, you know, a sample being a smaller group or a smaller subset of a larger population. So examples I give for my students, imagine the population of the entire United States 300 you know, 385 plus million. And I used the 385 million. That was a couple of years ago. So it’s probably a bit different from that. You can’t go out and measure absolutely everybody from that population, the population is too large. I’d like to give the examples of, imagine you wanted to know the average height and I asked my students, can you go out and measure everybody? Well, no, uh why? Well, you can’t ever actually talk to everybody like, ok, other problems that you run into your population may change things along those lines. So what we do in statistics instead is let’s go look at a sample, a smaller group. And from that smaller group, we want to make a conclusion about a larger group. That’s one of the main aspects of statistics and statistical studies. From that, we’ll talk about what is the probability of observing this type of thing happening given some base assumption. And so any time you want to use statistics, you’ve got to have those assumptions very well laid out, very well determined. This is what we’re assuming. This is I tell my students our prior belief and like a lot of things in science. Um A saying I’ve become partial to recently, science is constantly in, in the act of trying to prove itself wrong. You can observe gravity and we talk about the laws of gravity. But all it would take is one day when gravity stopped working for us to go. Oh, I guess that wasn’t a law. It’s not always constant. So statistics, we, we can’t ever prove a pre held belief. We can disprove, we can, we can find great evidence against it though.

[09:21] Michelle: OK. So here’s an example that comes to mind. I just was listening to an interview last month and um the Harvard professor was saying that she had this deep down knowledge that giraffes don’t eat meat until one day she saw in the wild in Africa, a giraffe eating meat. And that that’s kind of what you’re talking about, right? Like there’s something that could happen that’s completely unexpected from all of our prior knowledge, but we can’t use our prior prior knowledge to rule out any possibility of something unexpected happening am I understanding that correctly?

[09:53] John Kidd: So for example, if I had the belief, you know, something like that works perfectly or if I say uh all students at UVU have a favorite color of green, I I can very easily disprove that. But if I go out and take a s and even if I were to go out, take a sample, say I talked to 100 students, it’s possible that I talk to 100 students who all have a favorite color of green. I can’t get to the end and say I have proved that every student has a favorite color of green because all it takes is one student who doesn’t to make my prior belief incorrect.

[10:29] Michelle: OK. Ok. So basically right now, what I’m hearing is that we just need to be humble in our like there’s, there’s the humility required in science in general, but also in statistics and probably life in general as well.

[10:44] John Kidd: Yeah, probably a little more humility and in science in general than we typically see.

[10:49] Michelle: Ok. So the overall claim as I understand it is that in Navoo Joseph Smith was first of all, well, he, he became the trustee in trust and so the property was in basically his control as the trustee and trust. And so land deeds were granted by him usually in exchange for many, there’s a, there’s a price listed on the land deeds. There are a few that were given charitably and that’s listed. But anyway, so the, the claim is that between 1842 and 1844 which is I, I believe somewhat arbitrarily limited to those years because that’s where they believe they will find polygamy. So first of all, you, you um affecting the sample size based on your outcomes, which I think is also already a little bit of a problem. But in those years, Bill Real claims that 34 women purchased land deeds. Well, he doesn’t claim they were purchased. I will show that they were purchased. 34 women were given land deeds with a purchase price included where there is no husband on the deed, even though there’s also, there’s often other siblings or Children with, with the woman, just no husband. And so based on that assumption that Joseph had, based on the assumption that Joseph had 33 to 34 wives, what he says is if the LDS female population of Navoo is 3000 and that’s just based on an assumption that the population was 12,000, then an assumption that half of those were Mormon than an assumption that half of those were women. So there are, this is as you were speaking about all of the assumptions built in. So taking the number of that there, the assumption that there might have been 3000 Mormon women in Navoo and Joseph had 34 wives. Then the chance of a single deed being given to a single wife would be 34 and 3000 or 1.1%. So that’s how he’s setting up the statistical analysis, which I, I find highly problematic to begin with. And then he said, so he compares it to like if you have 3000 marbles in a jar and 34 of them are red, I think he says, or 34 of them are black, whatever color, the chance of getting one of those 34 colored marbles is very low. And then each time you get another one, the chance goes down. So the fact that 12 deeds he claims were purchased by were given to, he says, supposed wives would be statistically impossible. So therefore, ergo, we have proven that Joseph Smith was a polygamist giving land deeds to his wife. So that’s the basic claim that I find very, not compelling. But I wanted a stati statistician to explain to me why I find it so not compelling.

[13:35] John Kidd: So far, there are quite a few things to consider in this, in truth, the basic idea. And if I recall correctly, the math that was used to find the, the proposed probability was to say, all right, let’s take the 34 out of 3000 and then raise it to the 12th power to say it couldn’t, it wouldn’t be likely to happen once it’s not going to happen 12 different times. One. This limits things quite a bit. Uh, in truth, there’s a fairly simple way to model the approach. Uh because in one case, we have to consider how many were actually drawn. If I’m talking about 3000 women or three, let’s let’s go back to the marble example if I’ve got a jar with 3000 marbles and all, but 34 of them are white and those 34 are black. If I reach in and grab 1500 marbles, it’s not unlikely that I’m going to pick a large number of those black marbles because I’m replacing. So I’m picking so many. So we have to account for that. In part, we have to account for the number that were selected. If anybody wants to look it up. There’s something called the hypergeometric distribution and it models this scenario perfectly. And in fact, you know, we could, we could say, oh, out of this many, these are the number in each group. What’s the probability we observe exactly this many being picked or less or more. So, in statistics, a lot of times with these types of uh we call them discrete examples where there’s a set number of possible outcomes, we very rarely are interested in what’s the probability of exactly this happening? Because there’s a lot of possible ways that things could happen. And there’s a lot of possibilities for orders that things can happen, not a perfect relationship because this one follows a slightly different example. But if I go to a basketball court and somebody is shooting free throws and I say, all right, shoot 10 of them. And we’ll count the number you make if they make five. I mean, I need to account for how many different ways could they make five? They could make five in the first five, miss the last five, they could make the first four, miss the fifth, make the sixth. There’s a lot of possible ways it could happen. And so we don’t look for just, oh, this is the one possible way this could occur. We like to account for others. And in that end, with the basketball example, I might be more interested in what’s the probability you make five or more, we often consider ranges. So that’s one thing to consider one way to and you know, one way to be or one reason to be skeptical uh another and honestly, bigger example to consider our assumptions are often mistaken. When we do, you know, statistical analysis, we, we do our best and across all statistical analyses, we have to make assumptions. We have to assume something, we have to approach it from somewhere because we aren’t all knowing. There are those, you know, like the number of people on this earth who have ever been all knowing very, very limited, but you know, so assumptions have to happen and I’m perfectly fine with those, but we need to make valid assumptions to assume that, you know, and even if the assumption of 3000 women is perfectly valid. We have to consider what are the chances of those women receiving land deeds? The assumption to say, oh, we’re picking marbles out of a jar. That relationship assumes that every single one had an equally likely chance. That’s not, that’s probably not valid at the time. I am certain that some that rec you know, some that were eligible to receive land deeds were probably more likely to due to various circumstances. These circumstances could include their family situation, their financial situations could include the land deeds or the land plots that were available. The locations, there are a lot of different things that could go into it. And so to say that everybody had an equally likely chance is very, there is a and I don’t say this to, I don’t use this in the term to be rude, but it’s a very naive assumption and very rarely are fully naive assumptions valid. So we, and honestly, that would be my main, my main reason to be skeptical of claims of this nature because any of these assumptions and old picking marbles and these probabilities are based on that idea for me. And this could be purely a um purely my opinion. I have no idea of what, you know, this doesn’t represent much, but I spend a lot of time trying to think of reasons, things could happen. If you’ve got 3000 women, there’s going, you know, at the time, I would expect that it is a rather small proportion that are single women or those that do not have support from, you know, other places. And so the deed would be given to them if I’m thinking of my marble jar, that probably means that those marbles are quite a lot, quite a lot bigger. And whether we think of it or not, if you reach into a jar and you feel something larger that stands out. And in this case, I would think they are more likely to be selected. And so any, any woman who was single or more in need is likely to have been given that scenario. And again, I’m not certain on a lot of things and I can’t make claims for certain areas. But is it more likely that those that were in need are more likely to have been included in other groups, you know, to have been listed as a possible life? Um, so again, I, I’ll be honest for those watching, this is not something I’ve studied. So I don’t, I don’t know one way or another, but you mean

[19:46] Michelle: Joseph’s polygamy? Like

[19:50] John Kidd: I have enough other things. I have to study that this is not one that I’ve been able to read up on. Um, I, I’ve spent too much of my time cleaning up after my son.

[20:01] Michelle: I see

[20:01] John Kidd: the scenario.

[20:03] Michelle: Yeah. But what you’re saying is you’re just coming up with another possible assumption that you could build into this as an example of why you can’t just say why the marbles in a jar is not a good analogy. You have a limited number of exact marbles with an exact boundary that are all exactly the same size. And you’re pretending that that accurately represents the the situation in NAVOO. Is that, am I understanding that correctly? Yes,

[20:32] John Kidd: that would be a thought. And then so a possible better approach again. So, you know, is there an ideal situation now, what would be the ideal situation? Somebody build a time machine. Let’s go back and talk to people that, that, you know, we’ll get quantum physics if you want the field, that’s most ideal to to figure this out. I’m making the assumption that quantum physics is going to be able to be the field that figures out a time machine. Yeah, you know, Marty mcfly is gonna come in, he’s the one we need. Another possible approach would be to say, let’s narrow down the three, you know, the assumption of 3000 women. And then let’s consider those instead that are more likely to receive a deed at all, whether purch you know, to have the opportunity to purchase or to be given a charitable, whichever approach who is more likely to receive this assuming a charitable environment, uh different approaches that the saints were making in NAU at the time. It’s pro you know, you could narrow down a little bit and you still might find even amongst that subgroup, some that are more likely than others to have received a deed for, you know, various reasons. But in general, if you want to try a simple approach, narrow down to the group you’re interested in consider. All right, these are the groups. Um I think perhaps the original assumption said there were remind me how many deeds in total were, were given to single or I believe it said 30 votes were given for a woman without a husband.

[22:09] Michelle: He said 34. Well, yeah, but, but funny thing is a lot of those women had husbands, which is funny because a lot of Joseph Smith’s supposed wives also had husbands. So you can’t rule women out that way. Which is, which is amusing to me that we claim that women who had other husbands, we’re also married to Joseph Smith and we claim that, um, women who had husbands were given land deeds on their own. That’s evidence that they were married to Joseph Smith. So, anyway, that’s, that’s a big, another big wild card just to throw in there.

[22:41] John Kidd: I say my wife is the, uh, the primary person listed on our, uh, our van title. So I say, because I was living on the other side of the country when we purchased it. But so there are, in those cases, there’s lots of arguments in the end to do. Yeah, I guess my general thought is that a statistical analysis for this is probably an oversimplification. We have to make a lot of assumptions. But once we try and we get down, even if we were to narrow it down to just those that we know for sure we were eligible for this criteria then to go in and, and try to assign an even possibility to all of them. I just don’t feel is a valid approach. I don’t think that that’s acceptable under any of these methods because things are complicated. Life is complicated and life is going to make things more, more difficult even if the assumption is, well, of course, it’s not going to be even. That’s what we’re saying is more were given to a certain type of woman than others and like, but is it possible that those that are listed as possible wives had other similar characteristics that would be likely to link them to receiving a deed more than another. So to give, I would say to give examples or to give an example away from this. Yeah, we talk in statistics a lot about obs observational trials and what we call experiments and an experiment, we control everything. So here’s the controlling aspect of of the field. I want to be hands on control everything. There’s a reason for that to justify us all because the alternative is an observational study. Now to give an example that a lot of people are familiar with, imagine that we want to show that smoking causes cancer. Now, for many, many years, we couldn’t do this. Now, I’ll, I’ll tell the fun story at the end. New scientific thing. Purely statistical. Sorry for. If I bore people, you can’t, you couldn’t, for many, many years show that smoking causes cancer because you can’t do an experiment where you make somebody do something potentially harmful to them.

[24:59] Michelle: Ok. Ok. So,

[25:00] John Kidd: so we can’t control anything. But what we can do is observe and we can go out and observe a sample of people say we do select them randomly and we’re going to observe that some of them smoke, some of them don’t. And in each group, we can observe the number that get lung cancer. Now we did this for many years. Court cases you now if you walk past any store that sells cigarettes, big old signs that say these types of things in truth, you can’t make the conclusion that smoking causes cancer even if you observe an increased rate of lung cancer. Because is there some other unknown characteristic back here that makes somebody more likely to smoke and also makes them more likely to develop cancer to fill in this story even more? And the stat to get to the statistical aspect, it turns out that there is a genetic predisposition to becoming addicted to cigarettes, smoke, to nicotine to cigarette smoking. Oh, so imagine the theoretical idea and why we can’t say this causes something else. Why we say things are complicated if that pre genetic or that genetic predisposition also increased your chances of developing lung cancer, right? What causes the change or what causes the cancer? Is it the smoking or is it that genetic predisposition now to fill in the end of the story turns out no, it has no influence. And because of that, we now can say smoking causes cancer but to simply observe and try to make these conclusions in that case is not perfectly valid. And another scenario to consider similar to this, is there something else in the background? We’re not accounting for that affects both things. If I come and tell you that ice cream sales are highly, highly correlated with boating accidents as ice cream sales increase. So do boating accidents. Well, I can’t come up and say, well that proves that ice cream sales cause boating accidents unless the amount of rum ice cream being sold has increased drastically over the last millennia, which I don’t think it has what instead is a more likely possibility.

[27:31] Michelle: They’re both in the summer,

[27:32] John Kidd: both in the summer as temperatures rise, people buy more ice cream. I worked at an ice cream place, famous ice cream place in Salt Lake. I saw this in my high school years as it got warmer, it got busier. People don’t tend to go boating in the middle of the winter. Much, much more likely that you’re gonna find people out on the lake or if you’ve got a nice boat out in the ocean during the summer. So a main point and perhaps one to focus primarily on is thing we can’t always claim, oh, this means this for sure, because there are a lot of things going on. And if there’s one thing I, I try to share with everyone, things are complicated.

[28:21] Michelle: Yes. OK. So can I ask you a couple of, so what I’m hearing in that description is the well known correlation does not equal causation. First of all, right? And um and, and the funny thing here that I keep saying is we don’t even have correlation like with, with this claim of Joseph’s wives and land deeds, most of Joseph’s wives weren’t given land deeds, right? Didn’t purchase land deeds and most of the land deeds, even the land deeds to women didn’t go to Joseph Smith’s supposed wives. So there’s not even correlation that’s very strong. And then in addition, even if you did have correlation, which you definitely don’t, you would have to show some element to claim causation just as you said, you can’t say that ice cream causes boating accidents, right? Like it could be oh, a high percentage of women who had land eats, became followers of Brigham Young and a high percentage of followers of Brigham Young claimed to be like almost all of the women who claimed to be Joseph Smith’s wives were followers of Brigham Young. There’s one thing we could, we could consider. Right. And so that’s the first point I’m hearing and then I have one other point that I, oh, and then the second point I’m hearing and what you’re explaining is that it might be important to consider what these land deeds might be about and to do some actual exploration rather than just laying over the assumption that if they were land deeds to women, then they had something to do with polygamy, right? You like more effort is required to actually do some research to understand more about these land deeds. And the situation that was going on with Joseph, which is what I’ve been doing. So am I understanding correctly? Like because I have no case summary?

[30:02] John Kidd: Yeah. And I think even just to put it in general statistical terms, uh one of my favorite professors in my undergrad talked about, you know, the statistical process or any study involves quite a few stages. The one that gets the most attention is the analysis portion that coming up with the number and making a conclusion at the end, he also said that should be the smallest portion of everything we do. If you gather poor data, every conclusion afterwards is going to be incorrect. The less polite term is garbage in garbage out. If you don’t take care of that data, you don’t catch errors that enter in and errors always enter in everywhere. I worked for a company where at one point we were doing analysis on relationships between, you know, involving different cancer types. And we had a female patient that developed prostate cancer. This was, and this was biologically female, you know, assumed female patient had, had prostate cancer. Well, sometimes the

[31:12] Michelle: easiest

[31:14] John Kidd: this was, this was before any like it was, this wasn’t a case of transgender. Uh There was no cleaner, there was no, no relationship with that. It was the easiest. It was as often the simple answer is often the most correct somebody entered in gender incorrect. It was a male patient. OK? But imagine and imagine the results. You know, this was a, this, this ended up affecting, you know, a lot of things and it had effects on statistical analysis until we figured out what happened and we had to correct it. So original data collection, data entry, keeping track of things and even the type of analysis that you do if you do them incorrectly are not going to give you valid results. If you to give a simple example, we often do what we call linear regression where we say I’ve got two variables. I want to fit a straight line to represent them. Well, if instead of having, you know, if you’re fitting a straight line and say, OK, this is the pattern. We see if your points instead come along this way and then swing up. If you fit a straight line to this, it will look like this. But any type of prediction you try to make say, oh, this is the relationship. Well, you’re right here and you’re correct here, you’re always overestimating in this region and you’re always underestimating everywhere else. So, so the numbers are the nice part and that’s the part that gets sent out and people get to view and talk about and think about if they’re not done, you know, we need to spend a lot more time doing everything else to jump in naively and make claims. Doesn’t there is, is potentially misleading. Um Hopefully not just sometimes if we don’t think about things and we approach it in too simplistic a form we can come to false conclusions.

[33:19] Michelle: Ok? Which is what can sometimes give statistics a bad name. I’m assuming where it’s, is it Mark Twain that there lies damn lies and statistics he’s

[33:29] John Kidd: quoted with it. I, I’ve heard that the quote is, is much older somewhere from the early, you know, 17 hundreds and back in England. But yes, that’s, that’s my favorite. I should have worn that shirt. In fact, it’s green.

[33:44] Michelle: Ok. But that’s where it comes from is because so, so I like what you’re saying, if we are applying it to this analysis that um Bill Real did like he, he’s, he did the marble analogy and then spent all of his time proving how statistically impossible it is to claim that Joseph wasn’t a polygamist once he created the jars and the marbles in a jar analogy. What you’re seeing as I’m understanding it is where all of the effort should be is to decide whether that’s an accurate analogy, which we can assume that because of the vast complexity of the situation and all of the massive assumptions, and I would say just very false assumptions built into it, we can say no, that’s not a proper model. And so a lot more time needs to be spent to set up a proper model if we wanted to do anything statistically with this.

[34:40] John Kidd: Yeah. And I will say statisticians will never commit to anything 100%. That’s why I say the probabilities that the assumed model are correct are very, very low.

[34:54] Michelle: I like that. Thank you. OK. Is there any like, tell me what else we need to talk about in this? And then I’ll,

[35:02] John Kidd: I think in general that’s, that’s the vast majority of it. Um Again, and one other thing to consider, uh perhaps the final, going back to the idea of statistics accounts for the possibility of making errors. If we do, we can do absolutely everything correct in what we’re trying to do and things can happen by chance. Now, whether this in truth, whether this fully applies to the situation, because we’re assuming that we know all of the values within, you know, we know all of the deeds and everything there. So in truth, trying to account for unknown variability is a little difficult or impossible. Because if we know everything, then we understand, we understand fully in an academic setting. I say we are the oracle, we know all. So if we tru truly know everything about the land deeds, then, you know, unknown variability is, is a little less of a point. But in general, there’s always a chance that things can happen while it may be unlikely if I take a coin and flip it 10 times in a row, the chances that all 10 pop up heads is very, very small, but it’s not zero, things can occur. And we need to also be willing to acknowledge that, that I think to bring it somewhat full circle wraps into this idea of, we can’t prove things for certain. There’s never 100% certainty in what we’re doing. In truth, I should backtrack on my statement earlier that we can prove that smoking causes cancer. We are very, very confident now that smoking causes cancer. But it could be that we’ve collected samples on individuals that have been, you know, on that. There were errors there. If I flip a coin 10 times, it’s very unlikely that it’s going that I’m going to see 10 heads in a row. But if I repeated that exercise, a million times, a few of those are going to have 10 heads in a row. And so if we imagine, you know, a vast universe of possibilities occurring, seeing something happen is you know, everything is possible. It’s very possible that all 34 of the proposed women could have received a land deed and not to be contrarian or negative. If that is the case, if we’re trying to make this argument, one might ask, then why didn’t all 34 receive a land? I think that’s a

[37:44] Michelle: 30 four wives, right? Is that what you mean? Ok. Like we would have to answer the question of why some wives and why not other wives, if, if we wanted to prove correlation, even between Joseph’s wives and vans.

[37:58] John Kidd: Yeah. And I think, and again, there are lots of places to go. I mean, what is the best approach? Uh I’ll leave that up to those that are more historically informed than I am. Um But there are to be to be transparent, to be fair, stati statistics gets its reputation for a lot of reasons. You know, one is, yes, there’s a lot of ways there’s variability and, and things can be complicated in that way. Two, it’s, it is far too easy to manipulate results. And so I’m one that feels that whenever possible, we should approach things from every viewpoint, we can and be able to answer things perfectly a scenario I like to consider for life in general, but it works for statistics as well. Is this wonderful image here? Slightly cut out? There we go. Depending upon the viewpoint, you look at things you may make one conclusion, but somebody else may see something entirely different. But what if instead we walk around to the other side, you know, either in cases of disagreements with people, let’s walk around to the other side and be willing to see their viewpoint and whether we are, we don’t necessarily have to be convinced that they’re right, but we understand where they’re coming from. But I also like the idea of look at it, not just from their side, but look at it from the cross angle too. Because if we look at it, not just, oh, it has to be a six or a nine, I can show it as it is a shape on the ground. And so from that, you know, we can get a full understanding of what’s happening rather than trying to make an overly simplistic conclusion.

[39:42] Michelle: So we need to be careful of bringing it. It’s um the the description of an elephant. I know um President UF talked about it. One of his conference talks that every blind man was holding a different part of the elephant telling, telling everyone fighting about what the elephant was like. One had its tail, one had its trunk, one has it ear, one had its body, right? And it’s that same, that same idea. And so I guess this is a question because I think if, if an, if an analysis is done well, somebody with no bias should be able to come to the same conclusion. Right. It should be reproducible in some way. So you shouldn’t have to come to it. I, I guess that’s what I like. I don’t think anybody would come to this list of land deeds and go, oh my gosh, these 10 land deeds were given to supposed wives. Therefore, there is this not only correlation but causality between land deeds and what? Right. Or there’s, I can show I can use land deeds to as evidence of Joseph’s polygamy. I don’t think that would ever ever happen. And so is it that kind of important?

[40:54] John Kidd: I would say again, on the statistic, we can’t commit to anything, anything can happen by chance to, you know, we should in general in any research, be able to reproduce our results. And again, we’ll, we’ll jump back to statistics. That’s what I know in statistics, we have uh you know what we call hypothesis testing and for those that have taken a stats class or are studying it, no, what we tend to do is not what was ever intended by the creators of the process. OK. What we end up doing is a little bit of a mix between processes. But what one of the original people said was Aaron a man by the name of Ronald Fisher, who was actually one of the biggest proponents of, you can’t prove that smoking causes cancer, which is why he smoked a pipe multiple times a day for his entire life. Brilliant man

[41:46] Michelle: going on as well.

[41:48] John Kidd: He might have been a little biased, biased everywhere. His general approach was if you can do something multiple times and you know, and you don’t see something more than one in every 20 tries, you know, the idea that something can happen by chance one in 20 tries. And that doesn’t tell you anything really. But if we can reproduce something and get the same results over and over again, then that’s pretty good evidence that, you know, some original assumption was incorrect or that things follow a certain model. So in this case, I think trying to apply, it’s going to be a little bit difficult um trying to introduce things to individuals because depending upon the viewpoint of the individual who’s looking at the land deeds, they’re going to look for different things. And all of us have certain biases. And I try very hard in my life to acknowledge mine. But I’ll fully admit if somebody gives me a data set and says here, here are some things that we measured. I’m probably going to look at it a different way than somebody else. And if I recall hearing recently about a study where some number of different statistician groups were given the exact same data set, asked to analyze it. And almost all of them came back with a different conclusion based on that data. OK, we have to be careful of our, of our biases. We also should be careful of other people’s, I’m accused and I fully accept the fact that I am a very cynical person because we should take things that are presented to us. And again, remember, things are complicated. There may be other explanations and whether those are convincing or not, sometimes we need to consider those and be very, very careful of conclusions that are made.

[43:47] Michelle: Ok? I, I’m guessing that you as a professional of statistics watching this presentation didn’t find, I find it compelling or, um, you know, admirable even statistically,

[44:01] John Kidd: I find the approach interesting and I always find somebody who’s willing to try something statistical, at least a little bit admirable. I’ll give them that anybody who’s willing to approach a subject most people hate, I’ll give you that most don’t want to, but I didn’t find the reasonings as convincing as they were portrayed to be.

[44:21] Michelle: Ok? Oh, what a, what a beautifully diplomatic complimentary way to say that. Thank you so much. I appreciate that.

[44:28] John Kidd: I, I don’t have tenure yet. I still have to be very, I will continue to remain that way because it’s much nicer to interact with people, but I don’t have tenure yet. So there’s, there’s extra motivation to be very diplomatic and careful. I think in a lot of cases you’re right about the complexity. There’s a lot of different ways that things can be looked at and be considered. Um, I’d say for myself, I, in, I say a lot for people and getting very, very, you know, religious and considering the only thing in the world that I consider as simple is the atonement that the savior died for us. Um, and even with that, I mean, we talk about the most incomprehensible thing that’s ever happened. But beyond that, I think there’s a lot of things that we just, you know, the, they’re complicated enough that it’s, there’s a lot of possibilities and whether we ever get a true straight answer about some of them in this life may or may not be seen. And so for all cases, I think there’s, there’s all there’s evidence to consider, think about and make a determination of on both sides.

[45:42] Michelle: Ok. That’s, that was again, very diplomatically done. Thank you. I appreciate it. And, and a good assessment. Let me just kind of been wrapping up. Let me ask you if I were to ask you to do some sort of a statistical analysis of land deeds and Joseph’s wives. Can you give me just some idea of the information you would need to be willing to accept that project to uh

[46:08] John Kidd: about a week’s worth of time to think about what would be needed. Uh So I think in part, the, the previous analysis approach, I think takes at least to give a compliment is a good approach in trying to say what is the probability of seeing something like this, given certain assumptions to better understand though we would need to have a very clear, precise idea of who actually was eligible. So, first thing to consider is what, what is this denominator? How many marbles would there be in the bowl? Next, we would need to know additional information about how likely, how likely is each marble to be pulled. Because I, I find it a very difficult assumption to believe that everything is equally likely. That would be the idea that Joseph had a hat with everybody’s name in it. And he was just, and today you get a deed, this is not Oprah, we’re not handing them out just pulling names from a hat in. Anyway, if you go and see Oprah, it was everybody in the audience. So we would need to have the idea of who was possible or who was eligible, I should say, rather than possible who is eligible to receive and who or what chances were, if there were other things that were being included. And if that’s the scenario, we need, we would, you know, to be scientifically rigorous, we could find probabilities of, you know, chances of seeing an event given some initial assumption. However, even what that initial assumption should be probably needs to be considered because again, if, if we’re assuming, you know, trying to state this proportion of a certain group received it and that proves that certain things were happening, I don’t think is, is a valid approach um in trying to do these things with a statistical analysis is going to be complicated because we would need to say, what are the, we could say at any point, what’s the probability that these specific 34 women received a deed? But I mean, you’ve got other claims that have to be in there. An underlying claim that you make is that these are not accurate representations. And so to do a statistical analysis as at the end, uh so to break away from things that are needed, just another thought to consider. Uh there’s a phrase called the Texas sharpshooter, how there’s a specific name for it. But Texas Sharpshooter is the one that pops up most the idea that forgive the slightly violent reference. But if I go out to a barn side of a barn and shoot it a bunch of times and then walk over and draw the target around it, I look really good. But that doesn’t mean that I actually hit the point. So

[49:08] Michelle: that’s a great analogy. I feel like that’s exactly what happened in this analysis. OK. I like that.

[49:14] John Kidd: So that’s, that’s something to consider. If we’re going to try to say what’s the probability of certain things happening? We have to be very, very clear on what it is. We’re trying to show if we do want to say what’s the prob I mean, the values that were calculated are what’s the probability this many of the specific women were selected? Then you found that, but is that we would need to know the numbers for sure. And I think certain assumptions would need to be made that yes, this is an underlying proportion. And if we’re assuming equal distribution, then this is the probability of exactly this happening, given that scenario that is very small or it’s not as small as is claimed. Um I think in general a statistical analysis or even just a probability calculation for this wouldn’t quite be sufficient. I don’t feel that it can grasp the complexity of the situation we’re looking at or give a fulfilling answer to the question that could, would need to be formulated.

[50:23] Michelle: OK. That’s excellent. So, OK, so, so there would, as you were saying before, there would have to be a me like infinitely more attention given to how many marbles should be in the jar, what size each marble should be, right? How much? And, and that’s another probability, you know, like, like problem built into that. And so, um so even the idea that, well, first of all, the idea that the population of NAVOO was 12,000 between 42 and four and 44 is hugely problematic. Like I’ll just tell you a couple more things if, if that’s, you know, interesting. But um a paper I found that I, I think I probably shared with you, but that measured the, the population of NAVOO said um Navoo grew from 100 in 1839 100 people in 1839 to about 4000 in 1844 and rose to the height of 12,000 in 1844. So it went from 4000 and in 42 to 12,000 in 44 and stood at around 11,000 in, in 45. So anyway, so this even the population of NAVOO was extremely complex and getting the population from year to year is a huge problem. And then another thing built in there, I just looked up average family size at that period of time. And I don’t know if NAVOO would have had the average family size or maybe larger than average for members of the church, smaller than average for the criminal element coming in because that was also flooding into NAVOO again, so much complexity, but the average family had six Children at that time. And so to even to say that half of the, the the the population was 12,000, that half of those were Mormon and half of those were eligible women, all of that are such wild assumptions that are very easily disproven. I would, you know, like I can say those numbers right? There are all faults. So why should we assume any of the other assumptions could be valid? Anyway, go ahead. Sorry,

[52:30] John Kidd: I would say in, in something in that case, what I always prefer and tell people to do is if you’re going to do an analysis. Now, part of this is to be safe. But also if you find something at the end, you can actually make your argument more convincing. Anytime you do something like this, finding the probabilities, you should make every assumption on the conservative side, meaning the side, not what you are biased towards. And so I would be more willing to give credence to the initial claims that were made if using that assumption. So if we based it off that and said 12,000, if the family has six Children, then that means the mom represents 1/8 of the possibilities, maybe we scale that down a little bit because not everybody is part of a family. But even then if you set a four,

[53:22] Michelle: that’s the average, that’s the average population. Yeah. Yeah.

[53:26] John Kidd: And so, and in that case, if it’s the average, then you’ve got that, I would also expect that or expect or hope that not many deeds were given to the criminal element there probably was. And whether it’s just discernment of what was happening or likely less of a need for a for a deeded, you know, deeded land or combinations in there. Um I think the initial assumptions are, are definitely concerning and if you’re going to do things of this nature, it’s always better to assume on the conservative side, something that does not fall into what you expect to see or what you are trying to prove. So your quotes aren’t perfectly inappropriate there but what you’re trying, what you’re trying to show it’s much better to take the, to give benefit to the other side. And again, I even hate using that because there’s too much division in the world. Can we stop with that?

[54:28] Michelle: But right, we should assume that our bias is going to sway us toward our desired conclusion. So we should do our best to counteract that the best that we can. Rather than what I think you, I think the best analogy for the statistical analysis that Bill did was the Texas sharpshooter was shooting the barn and then going and drawing the bull’s eye around it. That’s exactly what I feel like happened here and it sounds like you tend to agree with that.

[54:55] John Kidd: I feel, I feel that, yeah, I feel that’s accurate. I will put the warning out for people in general of make sure you also don’t fall into that same scenario for anything that we do. Let’s make sure it is. May, may I reference pop culture? I remember my wife watching an episode of I believe it was C SI where at the beginning the person who was, you know, the the victim of the show because there’s always a victim per episode had just been to see a fortune teller who said all of these specific things and that person walks out and everything immediately happened afterwards. They did a very good job in that show at the end showing that the, the the sayings that were given to the individual were visual things that the fortune teller saw immediately outside. It was just visual cues. But we tend to see things more when you know, if they’re presented to us, I would ask you what car do you drive? How often do you see that car on the road? Right. At least, at least for me, I notice every single time that I see the car that matches the make and even more so the model when I served my mission, we drove Toyota Corolla. Oh my gosh, there are so many corollas in the world. I don’t notice them anymore. I don’t see them because it’s not on my mind. But for all of us, we can confirm our own biases or we can see aspects of things. If it’s on our mind, we’re gonna see it, we can find evidence of anything that we want to if we’re constantly looking for it. And it’s a lot easier when we’re doing that to ignore evidence against the things that we feel.

[56:42] Michelle: Absolutely. Ok. Which is why this method of bring your evidence and then have someone bring their evidence and pull each other’s evidence apart and see what evidence is left standing is so valuable, right? Like that’s exactly what the scientific method is supposed to be is to let everyone kind let all the ideas and all the evidence battle it out. So that, that way we get a better chance of having good conclusions rather than just motivated reasoning and bias. So, ok, so, so the last question I have, um, I have had people tell me, oh, I wasn’t sure. But now that I’ve seen the, the land deeds and the statistics, like now I really am convinced that Joseph was, was doing this, you know, was using the land deeds to give the wives. Do you think that, um, would you advise people that that’s a good assumption to make based on this presentation or would you advise against that? And maybe doing some more thinking,

[57:44] John Kidd: I would advise doing more thinking. And I think in any case, whatever situation you’re in be very critical. Uh If you have seen evidence on one side, you’ve seen one side. If you’ve seen evidence on one side, you’ve seen one side of it. Um So to, to relate to something a little more personal with my son, my son is um I to say whichever your preferred way of description is he is autistic is on the autism spectrum. One of my favorite sayings and it definitely chooses one of those options is if you’ve met one person with autism, you’ve met one person with autism. Um That’s a beautiful community, all very, very unique and should be treated as such to adapt that a little bit here. If you’ve heard one argument, you’ve heard one argument or if you’ve seen one set of data, you’ve seen one set of data personally in my life right now, I’m part of a community going through a somewhat contentious local election and there’s a lot of evidence and claims being thrown around and I’m trying to do my best to hear from all sides so that I can try to make a less biased decision. So for this case, I would advise anyone if you’ve seen one side, take time to see another. Um If you are spiritually inclined, turn to a higher power. Um So for me, my feeling is that, you know, we my feeling and also at least the branch of statistics that I follow. There’s, there’s two main divisions, not quite as bad as the blood and the crypts, but in statistics, there’s two main divisions, we talk about bays and frequent. I fall into the latter category and one somewhat principled belief is there is truth out there, but we can’t know it adding in a religious side. You know, my feeling is that somebody knows it, we may not be able to figure it out personally ourselves. Um We can do our best for research. But remember, as people present things to you, they definitely have their belief and whether we mean to or not anytime we believe in something, we will portray our side in a better light than the competition, the the opposite of what we’re looking at. And so take every opportunity you can for anything to research. If there’s only two sides, both sides. If there’s more than two sides, I find those situations even better research every side and hear from as many people as you can before making a decision so that you are very well informed.

[1:00:40] Michelle: Absolutely. Ok. That’s awesome. Thank you. That’s, that’s very helpful. I think that also it’s, that’s a good reminder that it’s important to let people speak on their own behalf so that someone doesn’t repre like there are a lot of people um interviewing other people about the claims that people like me make, which is I think amusing because then you’re hearing someone’s strawman version instead of their own version. So just like you said, it’s important to go to every other, every, every side and let them make the best arguments for themselves. That’s a really important thing. And then also something you said earlier that it’s good to kind of fight against our assumptions. So I do think that when you have someone who was strongly in one place, but then evidence brought them to another place that tends to have a little bit more credibility than someone who just is in their place and fighting against the evidence that might move them to another perspective. And so, um that’s, that seems to be another valuable thing to consider is that evidence does have the ability to change mindset if people will honestly consider it. And, and I, I think that that’s, that, that’s my story, right. I, I have changed direction on this topic massively two separate times about two se like I used to think polygamy was a true principle of God. It totally changed direction on that. Then I used to think, ok, it’s wrong and Joseph Smith was wrong about it. Then made a huge directional change on that based on evidence. So it’s good to look at the evidence. So

[1:02:17] John Kidd: I think the, the key I’d ask is honestly consider things and it’s there’s there is too. So in general, my soapbox for the world, there’s too much division in the world. Be willing to listen to sides, be willing to, to have an open mind anytime we go into a situation saying, oh I’m that there’s no possible way that anybody could convince me of anything otherwise I think we do ourselves and the world a disservice. Um Yes, there are some things in my life that I, I probably have that opinion towards. Um Yeah, and, and somewhere I, I feel that that’s a little bit justified. Um But if we can approach, say some, you know, things in our life, be willing to hear. If you can ask yourself the question, what would I need to see? What would I need to observe? What evidence do I need to see in order to, to change my mind? And if the answer is nothing could possibly change my mind. Consider what topic you’re talking about and whether that is the best situation to be in.

[1:03:28] Michelle: That’s great. Ok, I really appreciate that. I’ve answered that question many times. If I, I saw a child from any other wife it game over. I’m wrong Joseph was. And there are many other things that could change my mind too. So John, I can’t thank you enough, Professor Kidd. I can’t thank you enough for coming and talking to me. Thank you. And um really, as I said, appreciate your time and your expertise and just you’re just an all around good guy, guys. If you have kids that are smart enough to go study statistics,

[1:04:02] John Kidd: I teach, I teach everyone the truth. Be told the vast majority of my students are biology majors because that’s the class they, I I love them in my class.

[1:04:15] Michelle: That’s great. Ok, so, so, so thank you again. I really appreciate your time and have a fantastic day. Thank

[1:04:23] John Kidd: you. You as well.