Benford's law (with Vi Hart, 2 of 2)

SAL: So where we left off in the last video, Vi and myself had posed a mystery to you. We had talked about Benford's law. VI: And we asked, what is up with Benford's law? SAL: This idea that, if you took just random countries and took their population and took the most significant digit in their population and plotted the numbers of countries that their most significant digit is a 1, versus a 2, versus a 3, you just had it was much more likely that it would be a 1. Or that, if you took physical constants of the universe, that they're most likely to have 1 as their most significant digit. VI: I wish we had more graphs, because graphs are fun. SAL: Yes. VI: But if you look at information from the stock market or anything, what's up? SAL: Yes. And it seems to all follow this curve. And what was extremely mysterious-- and this is where we finished off the last video-- was if you look at pure, I would say, compounding phenomenon, like for example, the Fibonacci sequence, or powers of 2, that exactly fits the Benford distribution. It exactly fits this. If you take all the powers of 2, a little bit over 30% of those powers of 2, all of the powers of 2 have 1 as their most significant digit. What is this? 17? Roughly 17% of all of them have 2 as their most significant digit. VI: Yeah. Although in this case, there's an infinite number in every set, so it's harder to graph. SAL: But if you wanted to try it out, you could take the first million powers of 2 and then find the percentage. And that will probably give you a pretty good approximation of things. VI: Yeah. So to me, that's like less mysterious. On the one hand, wow, this fits exactly with mathematics. But that also gives you a really good handle, because you realize, alright, there's something here I can actually take a look at. SAL: You could take a look at it and it starts to become something you can dig deeper in. And we said, in the last video, we wanted you to pause it and think about why this is happening because, frankly, we had to do the same thing. And a big clue for us was when we looked at a logarithmic scale. And we're looking at one right over here. And just to be clear, what's going on in this logarithmic scale is you see equal spaces on this scale are powers of 10. So on a linear scale, this would be a 1. And maybe this would be a 2 and then a 3. Or if we wanted to say that this is a 2, you would say this is a 1, this is a 10, this would be a 20, then would be a 30, so on and so forth. But in a logarithmic scale, equal distances are times 10 or, in this case, if we're taking powers of 10. So this is 1:10, then 10:100, then 100:1,000. And you see how the numbers in between fall out, that the space between 1 and 2 is pretty big. And then 2 and 3 is still pretty big, but a little bit smaller. And then 3 and 4 gets smaller and smaller and smaller, until you get to 10. And that's a pretty big clue about what's going on with Benford's law. VI: Yeah. It seems to match up somehow. So there's a connection here. SAL: And it actually turns out-- and this actually a very big clue-- that this, if you take this area right here as a percentage of this entire area, it's exactly this percentage. It's exactly that percentage there. And if you take this area as a percentage of that entire area, it's exactly this percentage, that roughly 17%, or whatever that number is right over there. So that's a huge clue. VI: Yeah, or at least for powers of 2 or a Fibonacci sequence thing-- for powers, it definitely makes sense. SAL: Yes, for any powers. And so the logic is-- and this is now our biggest clue-- is to actually plot the powers of 2 on a logarithmic scale like this. VI: All right, let's see where they fall. SAL: All right, let's try it out. So 2 to the zeroth power is 1. 2 to the first power is 2. Then you get to 4. Then you get to 8. Then you get to 16, which is going to be someplace around here. Then you want to go to 32, which is going to be someplace around there. That's 30, so this is 32. Then you want to go to 64. And so this is 40, 50, 60. 64 is going to be right over there. And so what you see is, when you plot the powers of 2 on this logarithmic scale, they're equal distance apart. So you keep stepping along. If you were to plot on a linear scale, they'd get farther and farther apart. VI: Yeah. SAL: Actually, twice as far apart every time. But on this scale right over here they are equally spaced. So what's happening is you have something that's just equally stepping along here. You can imagine even just like walking along this. And if your sidewalk is shaped like this logarithmic scale, the probability on any given step, as you do many, many steps, or as you count all the steps, you're going to have many, many more steps that fall into the block that's between 1 and 2, or between 10 and 20, than you will, for example, the block that's between 9 and 10. VI: Yeah, if you just take a random point along here, you're more likely to fall in a area starting with 1. SAL: Right, one of these areas. Exactly, starting with 1, so between 1 and 2, or 10 and 20, or 100 and-- and that's exactly-- VI: So taking equal steps is going to give you that distribution, unless your steps happen to-- because there's special cases, right? So if you're getting-- SAL: Or people walk logarithmically. [LAUGHS] VI: Yeah, if you walk from 1 to 10, if your steps are 10 long-- SAL: Yeah. In special cases, yes. VI: So that's what happens there. SAL: If your steps are 10 long-- VI: You happen to exactly on [INAUDIBLE]. SAL: Right. But if you're anything-- any slight variation away from that exact thing, and then you will get the distribution. VI: Yeah. You're going to end up stepping all over the place. SAL: The Benford's distribution. VI: Benford's distribution. SAL: Even though I think we now understand why, it's still fascinating. VI: Yeah. Well, this explains it for these number series. SAL: Yes. VI: So now we have to somehow figure out how to connect that to the real world information. SAL: The general idea, well, so for populations. And we read up a little about it. And Benford's distribution tends to work for things that grow exponentially. VI: Yes. SAL: Like powers of 2. VI: Like powers of 2. SAL: Like powers of 2. And populations grow exponentially. VI: Yeah. And in finance, a lot of things also grow exponentially. SAL: Yes. Or decline exponentially. Either way. [LAUGHTER] SAL: But it tends to operate exponentially. You keep growing by 10% every year. That's an exponential path. What's fascinating is physical constants. And we actually aren't 100% sure why this is happening. VI: No. This is still crazy to me. SAL: We only have theories here. And the general idea-- because, you know, physical constants is sort of dependent on the units you're dealing with. They're depending on a whole bunch of things. Actually, I have a few very loose theories. But I'll let you all think about that more. VI: OK. SAL: All right? And so, hopefully, you all enjoyed this.