Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are by Seth Stephens-Davidowitz (PDF)

Description

An Economist Best Book of the YearA PBS NewsHour Book of the YearAn Entrepeneur Top Business Book An Amazon Best Book of the Year in Business and LeadershipNew York Times BestsellerForeword by Steven Pinker, author of The Better Angels of our Nature Blending the informed analysis of The Signal and the Noise with the instructive iconoclasm of Think Like a Freak, a fascinating, illuminating, and witty look at what the vast amounts of information now instantly available to us reveals about ourselves and our world—provided we ask the right questions.By the end of an average day in the early twenty-first century, human beings searching the internet will amass eight trillion gigabytes of data. This staggering amount of information—unprecedented in history—can tell us a great deal about who we are—the fears, desires, and behaviors that drive us, and the conscious and unconscious decisions we make. From the profound to the mundane, we can gain astonishing knowledge about the human psyche that less than twenty years ago, seemed unfathomable. Everybody Lies offers fascinating, surprising, and sometimes laugh-out-loud insights into everything from economics to ethics to sports to race to sex, gender and more, all drawn from the world of big data. What percentage of white voters didn’t vote for Barack Obama because he’s black? Does where you go to school effect how successful you are in life? Do parents secretly favor boy children over girls? Do violent films affect the crime rate? Can you beat the stock market? How regularly do we lie about our sex lives and who’s more self-conscious about sex, men or women? Investigating these questions and a host of others, Seth Stephens-Davidowitz offers revelations that can help us understand ourselves and our lives better. Drawing on studies and experiments on how we really live and think, he demonstrates in fascinating and often funny ways the extent to which all the world is indeed a lab. With conclusions ranging from strange-but-true to thought-provoking to disturbing, he explores the power of this digital truth serum and its deeper potential—revealing biases deeply embedded within us, information we can use to change our culture, and the questions we’re afraid to ask that might be essential to our health—both emotional and physical. All of us are touched by big data everyday, and its influence is multiplying. Everybody Lies challenges us to think differently about how we see it and the world.

User’s Reviews

Editorial Reviews: Review “This book is about a whole new way of studying the mind . . . an unprecedented peek into people’s psyches . . . Time and again my preconceptions about my country and my species were turned upside-down by Stephens-Davidowitz’s discoveries . . . endlessly fascinating.” — Steven Pinker, author of The Better Angels of Our Nature“Move over Freakonomics. Move over Moneyball. This brilliant book is the best demonstration yet of how big data plus cleverness can illuminate and then move the world. Read it and you’ll see life in a new way.” — Lawrence Summers, President Emeritus and Charles W. Eliot University Professor of Harvard University“Everybody Lies relies on big data to rip the veneer of what we like to think of as our civilized selves. A book that is fascinating, shocking, sometimes horrifying, but above all, revealing.” — Tim Wu, author of The Attention Merchants“Brimming with intriguing anecdotes and counterintuitive facts, Stephens-Davidowitz does his level best to help usher in a new age of human understanding, one digital data point at a time.” — Fortune, Best New Business Books“Freakonomics on steroids―this book shows how big data can give us surprising new answers to important and interesting questions. Seth Stephens-Davidowitz brings data analysis alive in a crisp, witty manner, providing a terrific introduction to how big data is shaping social science.” — Raj Chetty, Professor of Economics at Stanford University“Everybody Lies is a spirited and enthralling examination of the data of our lives. Drawing on a wide variety of revelatory sources, Seth Stephens-Davidowitz will make you cringe, chuckle, and wince at the people you thought we were.” — Christian Rudder, author of Dataclysm“A tour de force―a well-written and entertaining journey through big data that, along the way, happens to put forward an important new perspective on human behavior itself. If you want to understand what’s going on in the world, or even with your friends, this is one book you should read cover to cover.” — Peter Orszag, Managing Director, Lazard and former Director of the Office of Management and Budget“Stephens-Davidowitz, a former data scientist at Google, has spent the last four years poring over Internet search data . . . What he found is that Internet search data might be the Holy Grail when it comes to understanding the true nature of humanity.” — New York Post“Everybody Lies is an astoundingly clever and mischievous exploration of what big data tells us about everyday life. Seth Stephens-Davidowitz is as good a data storyteller as I have ever met.” — Steven Levitt, co-author, Freakonomics “A whirlwind tour of the modern human psyche using search data as its guide. . . . The empirical findings in Everybody Lies are so intriguing that the book would be a page-turner even if it were structured as a mere laundry list.” — The Economist“Pivotal . . . A book for those who are intensely curious about human nature, informational analysis, and amusing anecdotes to the tune of Steven Levitt and Stephen Dubner’s Freakanomics.” — Library Journal From the Back Cover How much sex are people really having? How many Americans are actually racist? Is America experiencing a hidden back-alley abortion crisis? Can you game the stock market? Does violent entertainment increase the rate of violent crime? Do parents treat sons differently from daughters? How many people actually read the books they buy?In this groundbreaking work, Seth Stephens-Davidowitz, a Harvard-trained economist, former Google data scientist, and New York Times writer, argues that much of what we thought about people has been dead wrong. The reason? People lie, to friends, lovers, doctors, surveys—and themselves.However, we no longer need to rely on what people tell us. New data from the internet—the traces of information that billions of people leave on Google, social media, dating, and even pornography sites—finally reveals the truth. By analyzing this digital goldmine, we can now learn what people really think, what they really want, and what they really do. Sometimes the new data will make you laugh out loud. Sometimes the new data will shock you. Sometimes the new data will deeply disturb you. But, always, this new data will make you think. Everybody Lies combines the informed analysis of Nate Silver’s The Signal and the Noise, the storytelling of Malcolm Gladwell’s Outliers, and the wit and fun of Steven Levitt and Stephen Dubner’s Freakonomics in a book that will change the way you view the world. There is almost no limit to what can be learned about human nature from Big Data—provided, that is, you ask the right questions. About the Author Seth Stephens-Davidowitz is a contributing op-ed writer for the New York Times, a lecturer at The Wharton School, and a former Google data scientist. He received a BA from Stanford and a PhD from Harvard. His research has appeared in the Journal of Public Economics and other prestigious publications. He lives in New York City. Read more

Reviews from Amazon users which were colected at the time this book was published on the website:

⭐The basic thesis of “Everybody Lies” is that online data on human behavior, including Google searches and data from Facebook, shopping and pornographic sites, can reveal much about what we really think than data from surveys in which people might be too embarrassed to tell the truth. In our unguarded moments, when we are alone and searching Google in the privacy of our homes, we are much more likely to divulge our innermost desires. The premise is that truly understanding human behavior by way of psychology or neuroscience is too complicated right now, so it’s much better to simply bypass that kind of understanding and look at what the numbers are telling us in terms of what people’s online behavior. In doing this the author looks at a remarkable variety of online sources and studies by leading researchers, and one must congratulate him for the diversity and depth of material he has plumbed.What has allowed us to access this pool of unguarded opinions and truckloads of data concerning human behavior is the Internet and the tools of “big” data. As the author puts it, this data is not just “big” but also “new”, which means that the kind of data we can access is also quite different from what we are used to; in his words, we live in a world where every sneeze, cough, internet purchase, political opinion, and evening run can be considered “data”. This makes it possible to test hypotheses that we could not have tested before. For instance, the author gives the example of testing Freud’s Oedipus Complex through accessing pornographic data which indicates a measurable interest in incest. Generally speaking there is quite an emphasis on exploring human sexuality in the book, partly because sexuality is one of those aspects of our life that we wish to hide the most and are also pruriently interested in, and partly because investigating this data through Google searches and pornographic sites reveals some rather bizarre sexual preference that are also sometimes specific to one country or another. This is a somewhat fun use of data mining.Data exploration can both reveal the obvious as well as throw up unexpected observations. A more serious use of data tools concerns political opinions. Based on Google searches in particular states, the author shows how racism (as indicated by racist Google searches) was a primary indicator of which states voted for Obama in the 2008 election and Trump in the 2016 election. That’s possibly an obvious conclusion, at least in retrospect. A more counterintuitive conclusion is that the racism divide does not seem to map neatly on the urban-rural divide or the North-South divide, but rather on the East-West divide; people seem to be searching much more for explicitly racist things in the East compared to the West. There is also an interesting survey of gay people in more and less tolerant states which concludes that you are as likely to find gay people in both parts of the country. Another interesting section of the book talked about how calls for peace by politicians after terrorist attacks actually lead to more rather than less xenophobic Google searches; this is accompanied by a section that hints at how the trends can be potentially reversed if different words are used in political speeches. There is also an interesting discussion of how the belief that newspaper political leanings drive customer political preferences gets it exactly backward; the data shows that customer political preferences shape what newspapers print, so effectively they are doing nothing different from any other customer-focused, profit making organization.The primary tool for doing all this data analysis is correlation or regression analysis, where you look at online searches and try to find correlations between certain terms and factors like geographic location, gender, ethnicity. One hopes that one has separated the most important correlated variable and has eliminated other potentially important ones.There are tons of other amusing and informative studies – sometimes the author’s own but more often other people’s – that reveal human desires and behavior across a wide swathe of fields, including politics, dating, sports, education, shopping and sexuality. There’s plenty of potentially useful material in these studies. For instance, some of the data that indicates gaps in educational or social attainment in different parts of the country are immediately actionable in principle. Google searches have also been used to keep track of flu and other disease epidemics. Sometimes finding correlations is financially lucrative; there is a story about how a horse expert found that success in horse races seems to correlate with one factor more than any other: the size of the left ventricle. Another study isolated the impact of the early growing season on the quality of wines. There is no doubt that financial firms, supermarkets, newspapers, hospitals and online purveyors of everything from pornography to peanuts are going to keep a close eye on this data to maximize their reach and profits.Generally speaking I enjoyed “Everybody Lies”; for the scope of the material, the easy-going style and some of the counterintuitive observations it reveals. My main reservation about the book is that I think the author overstates his case and sometimes sounds a little too breathless about the great changes these tools are going to bring. More than once he uses the term “revolutionary” in describing these data tools, but I am much more suspicious of their ultimate utility. Firstly, data does not equal knowledge; rather, it is the raw material for knowledge. As the author himself acknowledges, understanding correlation is not the same as understanding causation, and it’s in very few cases that a true causal relationship between people’s Google searches and their true nature can be established. Part of the reason I think this way is because I don’t believe that a person’s Google search is as reflective of their innermost desires as the book seems to think, so what a person truly believes may go way beyond their online behavior. Consider the studies revealing people’s sexual preferences for instance; how many of them point to trivial idiosyncrasies and how many are indicative of some deeper truth about human brains? The tools alone cannot draw this distinction. At the end of the day you could thus end up with a lot of data (including a lot of noise), but teasing apart the useful data points from the red herrings is a completely different matter. In this sense, looking at Google searches and other information can be a reductionist and simplistic approach.Secondly, it’s usually quite hard to control for all possible variables that may reflect a Google search; for instance in concluding that racism contributes the most to a particular political behavior, it’s very hard to tease out all other factors that also may do so, especially when you are talking about a heterogeneous collection of human beings. How can you know that you have corrected for every possible factor? Thirdly and finally, the “science” part of “data science” still lacks rigor in my opinion. For instance, a lot of the conclusions the book talks about are based on single studies which don’t seem to be repeated. In some cases the sample sizes are large, but in other cases they are small. Plus, people’s opinions can change over time, so it’s important to pick the right time window in which to do the study. All this points to great responsibility on the part of data scientists to make sure that their results are rigorous and not too simplistic, before they are taken up by both politicians and the general public as blunt instruments to change social policies. This responsibility increases especially as these approaches become more widespread and cheaper to use, especially in the hands of non-specialists. When you are in possession of a hammer, everything starts looking like a nail.Considering all these caveats, I thus find tools like those described in this volume to be the starting points for understanding human behavior, rather than direct determinants of human behavior. The tools themselves can tell you what they can be used for, not necessarily what problems would benefit the most from their application. The many interesting studies in this book certainly answer the “what” quite well, but most of them are still quite far from answering the “how” and especially the “why”. They point out the path to the door, but don’t necessarily tell us which door to open. And they can be especially impoverished in illuminating what lies beyond; for that only a true understanding of the human mind will pave the way.

⭐Author Stephens-Davidowitz is a data scientist, someone employed to analyse and interpret complex digital data. He has worked at Google, who hired him after learning about the strength and accuracy of his data research into racism.His exploration of data has led to fascinating revelations about mental illness, human sexuality, child abuse, abortion, advertising, religion and health. The datasets enabled by the digital explosion, offered new perspectives on all manner of issues that didn’t exist a couple of decades ago.The microscope made it possible to see that there is more to a drop of pond water than we thought, and the telescope showed us there is so much more to the night sky than we imagined. Digital data similarly reveals that there is more to human behaviour and society than we thought, and often very different to what we thought.“One of the primary goals of this book… is to provide the missing evidence of what can be done with Big Data—how we can find the needles, if you will, in those larger and larger haystacks,” the author explains.In the past we might have suspected something, now using Big Data, we can prove it, or show that the world works in precisely the opposite manner.The author’s grandmother frequently emphasized the importance of couples having common friends as a key factor for their marital success, as it was in hers. Is this sound advice?A team of computer scientists recently analysed the biggest dataset ever assembled on human relationships—Facebook, to answer this question. What the data showed was that having a common core group of friends, is a strong predictor that a relationship will not last. Having separate social circles may actually make relationships stronger.So why did grandmother believe just the opposite of what is true? People tend to exaggerate the relevance of their own experience. We give far too much to weight to certain data points – ourselves. Similarly, we tend to overestimate the prevalence of anything that makes for a memorable story. Consider, for example, whether more people in the OECD countries die from terrorist attacks, or from drowning in bath tubs? (The answer is bath tubs!)The author claims four unique powers of Big Data.The first power of Big Data is, obviously, new data – data that could not be understood in small quantities.The second power is being able to provide honest data. In the digital age, people still hide their thoughts, prejudices and desires from themselves and from other people. This is the origin of the book’s title, “Everybody Lies”. However, through people’s searches on the internet for example, even with their anonymity protected, people’s aggregated views are accurate and honest reflections of their thoughts.We can also zoom in on small subsets of people – the third power of Big Data. For example, are people sick with the flu more likely to make flu-related searches? Which searches most closely track housing prices? If for example, searches for schools in a district increase, we can expect housing price changes.We can also do many causal experiments with Big Data. What types of crucial information will make the stock market move? In the U.S. one answer is the monthly unemployment rate. Financial institutions do whatever they can to maximize the speed with which they receive, analyse, and act on this information, and make buy or sell decisions. Today, once the labour statistics are released, the market will move in less time than it takes you to blink your eyes.By analysing Big Data, we are also able to identify information of real value even if it is not explained. The size of a horse’s heart, and particularly the size of the left ventricle, is the single most important predictor of a horse’s success. In the same vein, horses with small spleens earned virtually nothing. And the horse’s pedigree is a far less reliable predictor of success that we used to believe. This realization will eventually affect the price of pedigreed horses.Based on their Big Data analysis, Walmart identified a strong positive correlation between the sale of strawberry Pop-Tarts, and impending hurricanes. In like manner, the quality of a wine can be explained simply by the weather during the growing season, and less by a host of other factors we have become used to considering.If your goal is to predict which wine will excel, what products will sell, which horses will win, you don’t need to be concerned with why your model works. “Just get the numbers right,” Stephens-Davidowitz recommendsBig Data comes in many forms – not only numbers, but text and even images. Traditionally, when academics or businesspeople want data, they conduct surveys.Do newspapers influence readers’ left or right political leanings, or do readers’ leanings influence the newspaper? Using Big Data researchers can prove that just as supermarkets identify what ice cream people want, and then fill their shelves with it, newspapers identify the viewpoints people want to read, and fill their pages with it. The influence relationship is in the opposite direction to what many thought. But the two big data sets, how people vote in a district, and which papers they read, don’t lie.Pictures are also data, as we see from the changing ways people have posed. Researchers studied 949 scanned yearbooks from American high schools from 1905–2013. From these they were able to create an “average” face out of the pictures from every decade. The image data showed how Americans, particularly women, started smiling in photos.People originally thought of photographs as paintings for which you posed for hours. Holding a smile would have been impossible. When Kodak began associating photos with happiness, being photographed smiling was how people want to show others what a good time they were having.This is the stuff of science, not pseudoscience. In the past, the world’s most famous linguists analysed individual texts – today they can reveal patterns across billions of books. The methodologies taught to graduate students in psychology, political science, and sociology and business, have been virtually untouched by the digital revolution.This book demonstrates how much they have missed.Readability Light –+– SeriousInsights High -+— LowPractical High –+– Low*Ian Mann of Gateways consults internationally on leadership and strategy, and is the author of the recently released ‘Executive Update.

⭐This is a really good book. It is funny, reasonably well-written and with enough amusing anecdotes and vignettes to break up the more serious thrust of the argument, which is simple enough: internet searches provide the large datasets which avoid many of the difficulties and expenses in traditional face-to-face and survey / questionnaire work. The second part of the argument is that people lie when presenting themselves on the Internet (e.g. Facebook) – they are presenting their “ideal selves”; but they tell the truth when typing into search engines for their own searches. This is especially true of the most sensitive areas of human emotional life: sex, relationships, racial hatred, homosexuality, abortion and so on. It also allows certain other questions to be tackled which go against received wisdom: is the Internet creating “echo chambers” where politically-engaged people only read / listen to their own side (spoiler: no). And it allows the verification of certain stereotypes (do more men who search for gay porn also search for information about Judy Garland? spoiler: yes). It allows one to find proper evidence-based examples of cultural differences. And so on. One of my favourites was showing the effect on countries of political leaders being assassinated…So this book is full of interest for those believing – or who are open to being persuaded – that the march of big data into the social sciences is continuing. And on the flip side, it shows such techniques are being used in the corporate and political world as well, to sell us more stuff or get us to donate more; primarily by using these big data techniques to leverage natural and quick feedback experiments to find out “what works”. Although it also does show why it won’t work for the stock market, as part of an overall section showing the limitations of these techniques.Highly recommended for those interested in the uses, actual and potential (and abuses), of big data in the modern world, particularly using internet searches as the dataset.

⭐An excellent discussion. Given the focus on Google search, this could be described as a book on Googleology. But it takes the reader into the rethinking that comes from understanding how with access to more and different data, new understanding of human behaviour is possible.Disturbing as well, but useful to understand how idiots get elected, and how people learn to be thieves and commit tax fraud, for instance and what makes us happy and sad.

⭐Listening to an audience or publics is a critical of element of marketing and public relations practice. It’s rightly a growing area of significant investment.Identifying an audience and understanding its motivation is critical to planning and engagement. It informs strategy, media and content.But what do you do if people don’t tell the truth? They typically don’t. People tell pollsters and market researchers what they think they want to hear.I’ve started dry January this month and signed up to the Try Dry app from Alcohol Change UK. It coaches you to reduce your alcohol consumption with daily nudges and rewards.The app asked me to submit data about my drinking habits. When I entered my data, I reduced my actual alcohol consumption by approximately a quarter.Even when asked for data anonymously in the privacy of our own home we modify our response to present the best version of ourselves, or at least the version we think that others expect.This effect is called response bias. It goes someway to explaining why pollsters call elections wrong, notably the Brexit vote in the UK in 2016, and President Donald Trump’s US election success late the same year.It’s the basis of Seth Stephens-Davidowitz’s book Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are. If you’re interested in the growing use of data in business, science, or politics you should read this book.Stephens-Davidowitz suggests that Google provides a means to obtain an accurate insight into consumer behaviour.People share their deepest fears and secrets with the Google search box. This behavioural trait combined with the sheer volume of data, so called big data, creates a useful source of insight into the human condition. It’s a growing area of psychology and social science.The Google dataset is accessible to marketing and PR practitioners via autocomplete data using tools such as AnswerThePublic, and search volumes using the Google Trends and Google Ads Keyword Planner.It has led to an emerging marketing and PR discipline called search listening. Indeed, it’s the subject of a book by Sophie Coley called Consumer Insight in the Age of Google and the basis of her Search Listening consulting and training business.Stephens-Davidowitz uses Google data to explore attitudes to mental health, parenting, race, reproduction, sex and more. The book is written for an American audience but it’s application and insight are global.The book isn’t limited to the Google dataset. Stephens-Davidowitz describes how insights can be discovered in any large data set. He explores datasets related to the economy, population, educational attainment, sports and more.In the final chapter Stephens-Davidowitz shares a study of Amazon Kindle comments that provide insight into the number of readers that complete non-fiction books. You’ll have to read the book for yourself to discover the answer.

⭐It is an interesting book and easy read. The author has used Google Trends as a primary source to explain his views on this book. It is not a book on the subject of big data from an industrial sense rather it is more on the social side of things. I was very surprised to know some terms people put in Google Search and why would they do such things. Another interesting thing is if we are all looking for some information and even what information we are looking for becomes information for someone.On the downside, after reading 50-100 pages you sort of get the idea the author trying to convey, after which it becomes highly repetitive and also since the author US based, 9/10 things are about things in america.

⭐Its not surprising that the author throws out all the stops to show us how much people reveal the truth about themselves when they search on the internet – but despite some good stories he overplays big data. Correlation does not equal causation but despite warning us – the author falls into the trap himself. Typing I love my girlfriend into a search engine is not a simple search phrase so be careful how you interpret what the typist is ‘looking for’ Racists and Trump voters live in the same county but they may live next door to one another they don’t have to be the same people. Its actually quite stimulating to read and pull the hyperbole back into something a little less overblown.

Keywords

Free Download Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are in PDF format
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are PDF Free Download
Download Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are 2017 PDF Free
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are 2017 PDF Free Download
Download Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are PDF
Free Download Ebook Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are