If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

Main content
Current time:0:00Total duration:43:27

Scott Cook - Founder and Chairman of the Executive Committee, Intuit

Video transcript

all right I think we're ready to start anyone who wants to anyone else wants to join us for the talk with Scott cook founder of Intuit so I'll just start you know for everyone here at Khan Academy who doesn't know both Scott and Cigna cook or you know some of the earliest kind of believers in Khan Academy and supporters and you know to a large degree you know Khan Academy wouldn't be where it is right now without a lot of their help and so you know I just feel like I owe you guys but with that out of the way we're just fans we cheer from the stands the players who win the game are in the room very nice thank you the but the point of having you here and we know whenever we have people of your stature come to the office we think it's just a valuable learning experience to have a conversation with you and then to see what what what's going on in your in your mind and and and the place I like to start before we open it up to the rest of the team is you know you see a lot of folks who've done big things like start into it and you know when I was a kid I always wondered how does that happen how do you go from being you know having a normal job and one day you start a company that company becomes a really big company so tell us a little bit about that story how did Intuit get started well I think for you it was your niece was it my cousin cousin cousin yes for me it was my wife very good she was struggling with her math problem no she had to do she does our joint checkbook and she was complaining about doing it she was very good at it she's diligent and mathematical but it was a you know a hassle because she had to pay the bills and he had to rewrite into the check register reconcile and she complained I thought hah this was in 1982 when pcs were just starting to explode I said Weymouth this sounds like the kind of problem everybody might have because everyone's got to have a checkbook everyone has to pay bills and PCs are now going into homes and this is very solvable on a computer this is the kind of task that computers are very good at and so though maybe I'll write this software just for fun but I hadn't programmed a decade so I figured maybe I should get somebody who actually knew what they were doing and so I found a student at Stanford Tom Perot and he I co-founded the company and with the goal of building a simple piece of software that allowed people to eliminate the hassles of managing their daily monthly finances and how do you go from that because what you see a lot of in Silicon Valley in other places is you saw a need you were able to build a prototype to do it but then there's a big jump between that and actually getting traction actually people using it how did you make that that leap yeah for for us that leap between launching it and actually getting a lot of users was not a leap it was a chasm and it was a crevasse software the time was sold in stores in boxes and you took money and we try to get VCS to invest two million dollars to give us the marketing money to convince stores to carry it and convince people to buy it and no VC said yes they all said no including VC firms run by classmates shows you how unpopular Sai dia was so that was a big problem the we borrowed some money we got a little bit of seed money from some relatives we stumble along we spent that we ran out of money we had to give back to rented furniture and rented computers we stopped paying salaries we it was pretty ugly and why didn't you just I mean you know it was stressful yeah our marriages this broke up and and I mean given that why didn't you just cut you know cut your losses and I think cutting losses would have been the rational thing actually it did look so hopeless but I felt I was kind of boxed in because we borrowed this money from some people we'd I'd taken my father's retirement money most of it we had the two relatives who invested equity it was kind of angel money but I felt I had to pay that back and he looked at the amount of money I'd have to pay back through honest work if I had quit and the business and so I felt trapped that I can't give up because then I got to pay all the stuff back so we just kept working we just kept working at it and at some point how are you living I mean how are you paying your bills and forcing signe had a very good job in fact in either this building or in one of the adjacent ones she yeah she worked in a software company software publishing Corp that had a bunch of these buildings so she and VP marking so she had a very good job and so it was her income that paid the bills and then what so how long did this I guess this this crevasse how long was this period and what was the what was the the moment where actually obviously you were you saw the the light at the end of the tunnel well and it's actually even worse than one crevasse there were two something we tried started to work and then it crapped out and so that's even worse when you get left down let's see we launched the product with great hope in October of 84 and it was the time when I finally felt the thing was doing well enough that I could buy something that wasn't a necessity something optional that could splurge on which turned out to be a CD player when I finally could afford a CD player was in late 87 and if that still was but that was and so this was a story I was still you know it wasn't buy it wasn't like a hockey stick at this point it was still a know slow is dead and so the question you ask let me answer directly the question Haskell so what actually did turn the business around it turns out it was word of mouth because the way we built the product was different than others had nobody up to that point software is an industrial product sold to corporations you know the big software success was visicalc which was sold to people finance people in corporations and the UI sucked you couldn't figure it out without training or reading a book but that's why software was we were the first people to do consumer testing usability tech we didn't call it back but usability testing so we brought housewives in from the Junior League because at the time there were all these women who didn't work during the day and they didn't use computers we put them in front of computer so I gave them our stuff and said okay use it to pay a bill and they'd fumble and goof and we look at it oh [ __ ] oh oh yeah now I see we go back and redesign it and we try it again and refine it and have people who didn't know the product no manual try it by hand so we had done more cycles of usability refinement and nobody had ever done that before so our product was actually usable and fast compared to thee and we're selling to people in homes no no executives telling them to use it they gotta want it and so people started using it from our earlier and feeble marketing and then they'd start telling their friends and start telling your friends and they started to be demand-pull and it was the word of mouth that ultimately took off and then it just is the rest is and we started tripling every year we triple in size and went into the most kind of obvious adjacent spaces tax taxes and well it was you know I've learned something about surprise in fact the first question we talked about today was what's been surprising I told him about our burglary situation when yes I laid out yeah there's learning and surprises so we had when we ran our first so we launched Quicken and even before Turk Hoffman was just you know failing and bumbling along we did a little survey of the few users we had and you ask all these questions and one question made no sense it was a we you know at the end of the questions where we were asking about how do they use it whether they like not like etc at the end there was a democratic set so you asked age location income and we asked a question where are you using the product home office or both and half the people claimed to using it in office now this is a home product so we thought maybe there they don't have a home computer I mean this was 1984 1985 maybe they're taking their their stuff to work and doing it there so we ignored it every subsequent survey the same answer half the people we're using it in an office now we thought it didn't make any sense finally it started bugging me why are people answering this question wrong so I said well let's let's go ask those people what they're doing instead of ignoring it this was years later so we went in actually Colson and asked him interview them went to visit them see what they were doing and by God they a lot of them weren't doing their home stuff they were running a business on now we built this as a checkbook manager not business accounting and the mindset everyone had is at business accounting talk to any who's took accounting who's an accountant here any accountants okay now it's very good now and what's the system by which you keep books for a business in accounting want this double entry double entry accounting with debits and credits journals and Ledger's but any expert will tell you that is the only way and we assume that so here we found these businesses using a tool that had none of that no debits no credits no double entry of anything and what but they're using a throw of it they are running their business on it it's where all the expenses in the income flows they're it's their accounting system and then we thought what why why are they doing this unexpected thing and it now it's obvious in hindsight but of those of you who took accounting in school raise your hand if you took accounting at school okay how many of you liked accounting in school yeah you and I were both weird but the rest of you are normal most people hate accounting and the people who keep the books in a small business tend to be the owner the owner spouse an unlucky clerk or assistant office manager nine times out of ten they do not know accounting and they do not want to learn to them general ledger which is a feature of all counting systems is a world war ii hero so they that's why they were using art product cuz we were in english and they didn't have to learn anything where all the others was a massive learning curve so i finally said well what would happen if we actually built a product for businesses but made it on the same promise of no debe snow credits no journals no i just put it in plain english you see an invoice you fill it in you see a check you fill it in so we worked for that on that launch that so all came because of a surprise today that business which is quickbooks is until about ten times larger than the quicken business so that's surprised which we ignored for years has created a business and if you add all of our small business stuff we've added payroll and payments in all that it's about 15 times the size of the quicken business so I didn't recognize it at the time but there's a pattern about when there are surprises big upside or downside savor the surprise what's really going what explains that that's not what we expected stop and savor it's too easy to keep going all right well that's a learn multiple times so that's the story on some of our subsequent businesses so now I know when you ask me questions that there's like a underpinning philosophy they're not random they may seem random very good and and and now before I open it up you know where do you see Intuit going I mean it's a much larger company now you know how do you as a the leader or the manager how do you try to navigate that and where do you see is the big opportunities and the pitfalls now the pitfalls are clear it is amazing how success and scale make you dumb in an organization it is just stunning the degree to which the creative inspiring innovation invention suddenly starts disappearing as you grow you add layers decision makers managers try to get rigorous and do all the stuff that you have to do to be both efficient and to be reliable well you know people don't want their I mean we did launch version of QuickBooks where your data disappeared yeah randomly after you all your data disappeared yeah this is a problem you don't really want to do this don't try this at home kids so you got to get rigorous and but something happens to you know without continued innovation where I've continued how do we make the current thing better how do we find the next problem to solve you eventually get stale and die so how do you be both rigorous and inventive and most importantly how do you free the your most inventive people who are often quite new to the company could be quite young likely our renegades how do you free them to invent without the board the organization boarding them and so we're redoing how we work along the lines of lean experimentation the thing I've learned is if you allow normal hierarchal decisions you have the problem all the problems happen well meaning middle managers it just slows down but what if you tell the middle managers your job isn't to make that decision well let the experience experiments make the decision middle your goal is to put in systems allow your teams and your brand-new employees to be able to take their best idea to solve your mission and run it as an experiment fast and cheap the internet now allows us to run these fast cheap experiments so there's this whole wave of change called the Lean Startup that Eric Ries has founded I've not seen any idea enter the common parlance of business as fast as the Lean Startup his concept of pivot went from novel to trite in two years I've never seen it's been absorbed at least as language so fast but he wrote the book about startups I believe there's two groups who need the Lean Startup even more than startups this concept of lean experimentation and those are large companies and nonprofits will benefit even more than startups so we're trying to operate as a network of startups inside where people can try their ideas and we've got coaches teams systems we put people through a two-day experience where people just sign up and by the end of the first day they have their idea up and running in some sort of test to test their leap of faith assumption all to uncork that inventive power that is what has created all great enterprises in South and Ally so that's kind of and if we do that right then I won't have to pick a direction our people will invent the directions far better than I can yeah perhaps absolutely I mean on that note I'd love to take questions and you won't have questions I have a bunch more anyone we started the company with a bunch of surveys waiting on I described one that helped us get the surprise on that led to QuickBooks but I have become a skeptic on surveys so I actually don't guide teams to run surveys I tell them avoid the survey we found there's not a high correlation particularly if you're looking at inventing new ideas and new businesses new products there's not much of a correlation between what people say they will do and what they actually do what they say they will do they'll do about half the time well so that means it's a coin flip we've had we did a product in India which we have which we surveyed users and we're trying to figure out how to monetize it so we surveyed we expected 20% would pay for it was we've been giving it away for free we surveyed users and 40% said they'd pay and then we actually did the test and took existing users and added a charge and 2% paid just there's another example a team working in payroll division had a new wave for a business to start up payroll they mocked it up in screens showed it to 20 payroll customers these are employers someone here runs a system to write your paychecks so it'd do that person zero of the twenty said they'd use it zero amazingly they didn't stop to thank God they didn't stop there but because of our system of experimentation they then got coached we actually had Eric Ries in coaching that say and he figured out a way what you could run an experiment in 24 hours to actually see what people would do not what they say they would do they ran the experiment and 58 percent of real customers did it if they listened the survey we never would have done this thing because they could take it to a test a test of just a slice of it they then learned that customers would we fleshed out built the rest of it and we got the fastest customer growth in ten years in that business so what I've learned is trust behaviors don't trust surveys measure and monitor their behaviors and then right after behavior you can ask somebody why did you do that or why not and then you might get something close to truth so that and do it more as an in-depth interview of the people who just did something or in a test just didn't do something and then you might learn something useful from the interview but I so we when we go out and talk to customer in an interview and observe one thing we'll do is go and watch customers as I find it's much more much more reliable to watch people in what they do and you've got an opportunity you've got people at home studying or whatever we could go watch and see actual behaviors don't say anything just watch you'll see things they would never comment about the I'm on the board of the Procter & Gamble Company they make soap tide and pampers and other things and in trying to figure out how to improve tide they'd run surveys and one of the minions you've got many things you ask about one of which was the box the packaging and they asked is it easy and the answers in the survey where yeah it's easy to open what years later they did some in-home observation and they followed some homemakers around watching what they did and they then did observe a few people opening a brand new box of tide and they thought it was easy if you kept a knife or a screwdriver next year and and because women have nails and they they couldn't open those things so they were hacking at it but he didn't never come up in a survey but they could see it so the one the most the thing I'd recommend is to understand customers deeply go ahead and watch them we call it follow me home testing leads to follow customers home from the store not exactly we'd ask permission so we could watch them start up well let me answer that in two parts one the story of how we got into payments and then the crystal ball gazing there was actually one guy an engineer after African immigrant who looking at our QuickBooks users said ah they really could use payments and in fact what they needs different and what everyone else is selling so he got our product manager work with him and a two of them cooked up this scheme and inserted it into the product and by God it took off because they'd found a problem for businesses who accept payments when there's no card present because QuickBooks is typically used in the back office around invoices and some customers wanted to pay with a credit card like over the phone but there's no card present and the whole payments industry had been routed around that card being the key so they invented different economics and different approaching embedded in QuickBooks and so now we have a payments business that's four hundred million in revenue sizable all because of they found a problem nobody else had solved and figured out how we could solve it now on gazing the about the future I right now you're seeing a hotbed of innovation around payments and things are becoming dramatically easier to send money person person to send me for businesses or businesses to set up and accept payments it's a it's long band payments a kind of monopolize urquhart alized business with a few major payment utilities such as Visa and MasterCard or one central bank run system called the automatic Clearing House and now you're this rampant innovations happen around so it's so much innovation in most of it I can't tell you how it's going to turn out but it's going to get a lot easier and cheaper for everyone the place we're focused on is still in the back-office because everyone else tends to be focused heavily on the point-of-sale but the place that's still being ignored is that business that sends bills or invoices out and for example in rent payments here how many of you pay rent raise your hand okay how many of you pay that electronically most of the hands went right there especially small landlords are not set up to accept electronic payments and it's a royal pain for you and for them they gotta hang around the mailbox waiting for the money to come in because if it doesn't they gotta bug you so a small team how many people you me how many people is that team when they started two or three two or three folks that we can we can fix that problem and so they invented a system called spark rent which allows the merchants to have the merchants the landlord to sign up and then the tenants it can pay electronically really quickly the landlord can be anywhere on vacation and they know everything it sends reminders out cheap you know it's had some hiccups but it's now the most popular way that's that small landlords can get paid electronically and tenants can pay electronically so that's the that's just the beginning of the kind of innovation from that what we want to do is develop a way so that our system will know that you pay your rent on time so when you try to get that next department in San Francisco and there's five people trying to get that apartment you've got a track record that you can say that I'm you know you can trust me here's my spark rent track record so we think we can then take the data and make it useful to both sides you know I when we're at our best yes we can pull that off but that's a real bit we're far from perfect on that in our payments group again we've had a group that came at a much lower price point we've in our tax group group was working on right now you know you go to TurboTax and you got to pay your taxes on a on a computer but this group said why don't we use the phone and why don't we use the phone to snap pictures of those tax documents so you don't have to type it in well management thought this idea wouldn't work and outside experts said it wouldn't work but they kept at it and they've built something called snap tax that works it's an app on your phone and you snap pictures of your w-2s and it asks if your taxes are simple and a lot of you guys are it answers asks like nine more questions and you're done nothing we have done in and it's at a lower price point it's easier nothing we've done in our company's history has gotten as higher ratings from customers it's five stars in the app stores and we get reviews like you can't believe one person wrote in that she was in the recovery room after surgery doing her taxes and so happy about it another guy wrote in the review in the app store that it was febris in bed with his girlfriend and she's so stoked she might even wear the gift he just gave it so but again was a small team had the freedom to experiment and invent something which is just that the mothership wasn't working on so at our best we do that there's other times when I wish we were better this is the eternal priority-setting question no matter what size herein before we launched we had more features we wanted to put in that we could fit in after we launched and now as a four and a half billion dollar company there's much more stuff we want to get to and that people want to get to them we can I have two thoughts on that and they are kind of opposing thoughts one is make it fast and cheap to run experiments so you can try more things if it's big and expensive to try something then you've got to be choosy if you can make it fast and cheap to test the leap of faith assumption on which a decision depends then you can try a lot more things so this Eric Ries is Lean Startup Khan we were already running experimentation and then I discovered Eric Ries on an online video and said my god that's our idea he explains it a lot better and he took it a lot farther so I you know circulate the Lean Startup and look at some of these videos it's really it makes sense so that increases your capacity to try new things and to try new radical things because if it's fast and cheap it can be radical the other the principle is that the actual important things is a small subset of all the things you're working on what's truly important here's an example so I run I learned this technique from another CEO to run a thing called a town hall meeting so we'll get a group of important customers together let's say accountants because we sell a lot of things to accountants and we'll get a group of 12 them together in a room and I'll stand up to the white board and I'll ask them two main questions what do you love about dealing with intuit and each person write down private silent work write down a list what do you love about working with intuit second question what do you hate about working with intuit what's wrong what do we where would you piss you off what are we just missing the boat okay silently each person write your own list okay then once they have those lists I stand up in front say okay let's read off list number one and everyone read their list one and I'm up the whiteboard and fill out the whiteboard with all the great stuff and I said thank you that's great that's awesome now let's go to question two everyone tell me the lousy stuff and I write all those ideas down and people will have real passion about the stuff that's wrong that needs to be fixed needs to be changed and you'll get 30 items up there real passion from folks and then I'll say okay let's forget list one thank you for let's focus on the list two I'm going to give each of you five colored stickers go up to this white board and put those colored stickers by the things that you find most important you can put all five on one you can put one sticker on each of five you can split them up what amazingly happens out of the 30 or 35 items that will be on the board most get no votes at all not even from the person who was advocating it the votes a uniformly are incredibly lopsided now they're not collaborating to each other not comparing notes they're doing this as individual work the votes there's one always every time I've done this one big one and then there'll be a second that gets half as many votes and it may be a third that gets half as many and then it's noise and then what we do in the meeting is say okay let's dive into number one okay what's the problem what's the issue when does it happen how does it happen tell us more what's the root cause we'll go now how do we go about fixing it and I've been to number one so basically you should work with most of your effort on number one small effort on number two and forget the rest organizations can't do that organizations will work on twelve Oh Suzy wants to work on this and boy that's a good idea from they can't that customer was so passionate and we got so you'll get 12 and you'll underwhelm the important ones you'll spend much too much time on the stuff that just isn't so this is the best example I've ever seen how customers actually are much more focused on what's important then companies are and it's one big one you know I tell folks what's the one and a half things you should be working on and then run cheap experiments so you keep the idea creation because you'll get surprised out of the cheap experiments that's your safety valve to get surprised but so that's that that might be helpful let's do a little show of hands how many people want to understand all the words in the tax law raise your hand how many people just want to get their taxes filed and get their money yeah we're not in the education business that's your business we have found at least when it comes to finance most people are exactly like you they just want to get it over and get the work done if they occasionally want to learn more we've got a lot of what we call point of need help we just click there's a little blue underline things saying what is this or tell me more and people can click and learn more when they do that it's typically not because they want to learn it's because they want to make the decision right so even when they're quote learning more it's only to make the decision and then move on so we know what you're doing is Noble and really important and crucial to the world but it's not why people tend to buy or use our stuff yeah so why did we acquire mint given our passion for internal innovation keep mine we acquired mint before we had ramped up a lot of the experimentation work okay so we were we're not as good as we want now but we were much less good then but also we're I'm very eclectic we want innovation and innovative teams for both sources so we both acquire and we teach and coach our internal and enable our internal entrepreneurs so we do both we certainly don't I think was build joy of son who once said almost all the smart people in the world don't work for your company so we want to be where almost all the smart people are which is you know we're only one slice of the smart people and in this case with mint we had an internal effort to do quicken online and mint and some other competitors were pursuing similar things and mint just outrun the whole field they produced a better solution that had a different business model and was just better than what we were doing so we said you know why fight a losing battle let's go get the top team and the top approach since we require them their business is tripled or quadrupled in size their ideas have infused much of the rest of into it that was one thing we were hoping to have their ideas change the direction of other businesses we've had them run the quicken business so we could take their learning and we gave them the quicken business to run so we really tried to maximize the wisdom transfer and now we're building a mint product for small businesses so again they have such a good thing for consumers plus we're adding much more to the MIT data and what MIT can do for you but we're also saying why can we take that same magic and solve problems for small businesses as well so that's some of the story I'm in your premise is exactly right when you do your finances with any system it accumulates the data so we've got a very large repository of data on people's taxes investments pending business at a transactional level the first thing to recognize is in my view that's not our data it's our customers data they allow us to be the stewards of their data only if we earn that so we can only do things that the customer wants and gives us permission to do and if we break that trust people will run from us and they deserve to the next thing is mint with another piece of the mint DNA we wanted was they were making better use of the data than anything we've done we've largely done nothing with big data we've used the data to solve your immediate problem but we have not then the way mint has said Oh based on the your data we can see that you be much better off if you did X or Y so that was another piece that we wanted to do to cross pollinate from the mint acquisition we now have a data team with a new leader we took one of our leaders who's very talented and added this challenge to her so there the data team was focused on how they can help into it with data and we said that's nice but what we really want to is how do we help customers with data how can we change their lives so we re oriented the data team on the mission that you described so that hopefully in a year or two we'll have more ways that we can help people here's just one that's already in in action it turns out most small businesses apply for a loan at some point seventy percent of our QuickBooks customers have applied for a business loan and most small businesses get turned down in a rather slow long ugly process where they wind up giving a lot of data information and paper to some bank that ultimately turns them down plus there's a lot of lenders who'd like to lend the customers they don't yet know we think we can make that happen because of the rich data in QuickBooks we can help figure out who is a good lender for you and then make it easy for you to apply with a push button instead of filling out all this paper and then have a lender be able to rapidly improve and get higher confidence that they're lending to good people because we've got deeper data on your business than anybody we can only use it with your permission but this is an area where businesses really want someone to take advantage of their data to give them a better loan at a better rate so we've got a lending operation we don't do the lending we work with lenders and we're trying to pioneer this lending marketplace using QuickBooks data as the kind of unique asset to make it happen so that's an example and we're getting some number of lenders are playing with it and ultimate goal is to build a better credit score based on richer data so that more deserving businesses can get better loans so I'll I'll just finish up you know obviously you are known as one of the you know top managers and not only your industry but in Silicon Valley or more generally and you know you've you've had you've known Khan Academy since really the beginning you know what thoughts do you have about what we're doing where do you see the opportunities where do you see the pitfalls and and which kind of your general thoughts and advice for us as a team I I bet some of you are better aware of the things to be afraid of I'm not sure I'm close enough to to give you a thoughtful assessment of that but I think one thing that would I can't say is an issue here but has been an issue in other fast growth operation so you grew from about five to six million monthly actives to about ten million over the last year a near doubling and even you know you're expecting a future like that your superstars celebrities in the press there is the curse of success where things are on a roll what everyone wants to talk to you which somehow is a drug that causes people in other organizations to take their eye off the ball to stop focusing on two things one what really solves the customer problem yeah a customers using things are great we don't to worry about that but companies can persist in doing things that ultimately are second rate just because the second rate stuff was better than what was before and they don't find out the breaker and we think of overstock.com who then just got surpassed and crushed by google robots by eBay or think of Yahoo which gosh they were the darlings huge the Internet company and then Google just went to buy them because they didn't they stopped focusing okay what's the major problem to solve and how can we innovate and solve it much hugely better and Yahoo always was a place people came to to go somewhere else and their search was not good so stick to the Knitting of what's the funnel problem we're trying to solve what do people really want us to solve and how are we the best in the world of that from what I see you're working on that so with the dashboard in that and the other curse of success and can be the curse that Friendster had so for in-store was the social network but for Facebook and even before it's the music one out of LA myspace and they were had taken off as a social network kleiner perkins major VC had invested they asked me to go down and take a look talk to the entrepreneur and that was good but he talked about the end of the talk about some technical problems and douve in to learn more and he wasn't gonna long with this technical founder or technical guy at all and the system was crashing it just couldn't keep up with demand it was collapsing and ultimately that killed Fenster one of the things that mark will say about why Facebook succeeded is he had seen the collapse of Friendster because of technical bums they just couldn't scale with the volume and he and Dustin Moskowitz would just did German that was not going to stand in their way so they limited growth early on and then they chest piled the resources to make sure they were up in reliable so those are some of the things I've seen other cases of companies running afoul I'm seeing good stuff here I don't know don't know don't suspect that's happening here but those would be some watch outs from the history of some companies who like you have been shooting upward oh great no thank you that's let me just cover one more thing which is the other half of your questions yeah which is about how do I see the future maybe I'm not very good seeing the future but I can see today and there's a something you may have seen this as an article was in Harvard magazine about a physics teacher there a guy named Eric Mazur anyones yep very familiar the article was entitled something like the twilight of the lecture so Eric Mazur was a well respected teacher on the physics staff there he'd been teaching physics for a decade I believe the tests that he gave showed the students learn subject matter he's all happy then he heard about the University of Arizona that had invented kind of a capstone test that tested kind of like the Common Core did people really get this stuff could they kind of apply it and and the results on University of Arizona was after taking the semester physics that the physics students there just failed this can you actually apply it in real situations so he looked and so now it's interesting that those are University of Arizona students his Harvard students almost all of his students virtually all got our number one score one or two in their AP Physics so he just as a lark took the test and gave it to his heart students after they had finished his semester of physics and by God they failed it he said it was monkey level if you had guessed red monkeys guesting randomly you would do as well as history now Erik is an unusual professor at a research institution because he actually cared he he didn't say well screw that and back to the research he actually said we what does it say about me what am I doing wrong as a as a teacher I'm failing these students Hey so he dove in and tried to figure out what was happening he has evolved from that once I when I guys passed me the article and then we reached out to him and we actually spent some time with him learning and he basically found out that students don't learn from lectures even from highly rated professors I just don't seem to learn from lectures and these are Harvard students they don't learn to I mean they learn to pass the basic test but they don't learn to apply in a deep way so he started trying alternative ideas he tried a very different he's basically flipped the classroom and the students work on problems and then he can figure out through a system which students got it and which didn't and then he have the students who got that section explain it to the students who didn't and he said they're much better at it because the student has just tried it didn't do it and they're now they're hearing from someone who just learned it who just is walking in their shoes and he says he's now now his students Ace this test I think there's a revolution in learning from something that was done to you education was done to people lectures are done to people too turning education on its head learning on its head so people now it's up to me it's now become something you do and you learn at your speed and you learn by doing yourself not by being reliant on someone else to pour learning into your head you learn by using you learn by doing that's the way real humans really learn fast you guys are in the vanguard of leading that globally with the internet with the raw inventiveness of the dashboard the on-demand the lectures to understand how the students progress creating a platform so that math physics and other subjects can be taught by learn by doing this will change learning and lifelong learning this is the this is the invention of printing by Gutenberg applied to a field that desperately needs it and more than that this is the sum total of IQ points in the world is fixed it's what genes have delivered to us but how you turn those IQ points into productive human potential into skills to be able to solve problems to be able to see problems to be that is entirely a function of people's learning and can be dialed up or down dramatically by the learning method compared when you've been in some class which was boring badly taught or the great research professor didn't really know English and you just your learning rate is so slow your love of learning just goes into the toilet and then compare that to a class where you the teacher was fascinating where you could move and learn and try things and they worked and he got this feeling of accomplishment it's night and day igniting the fire of learning in billions of people is a force of power far greater than the printing press and I believe you guys are the world leader in doing that well that gives us a little bit of pressure but but it's exciting at the same time and so I'll just on behalf of the whole team I just really thank you that was really a special conversation thank you you