White House Warns 'Big Data' Can Lead To Discrimination

Personalized ads and content can offer benefits to consumers, but also can reflect discrimination that causes “real harm to individuals,” the White House said Thursday in its widely anticipated Big Data report.

“The fusion of many different kinds of data, processed in real time, has the power to deliver exactly the right message, product or service to consumers before they even ask,” the report says. “Unfortunately, 'perfect personalization' also leaves room for subtle and not-so-subtle forms of discrimination in pricing, services and opportunities.”

The report is the product of a study commissioned earlier this year by President Obama. The study examines the use of data in a variety of contexts, ranging from how scientists draw on data to study diseases to the use of clickstream data by marketers.

Overall, the authors offer a mixed verdict on Big Data. “An important finding of this review is that while big data can be used for great social good, it can also be used in ways that perpetrate social harms or render outcomes that have inequitable impacts, even when discrimination is not intended,” the study states. “Society must take steps to guard against these potential harms by ensuring power is appropriately balanced between individuals and institutions, whether between citizen and government, consumer and firm, or employee and business.”

The report specifically mentions research by Latanya Sweeney, who has studied online discrimination. Sweeney, who now serves as chief technologist at the Federal Trade Commission, reported last year that Google searches for black-identifying names, like “DeShawn and Darnell,” were more likely to generate ads that contain the word “arrest” than searches for white-identity names, like “Geoffrey and Jill.”

“It’s clear that outcomes like these, by serving up different kinds of information to different groups, have the potential to cause real harm to individuals, whether they are pursuing a job, purchasing a home, or simply searching for information,” the report says.

The authors offer several policy recommendations, including that the Commerce Department move forward on developing a privacy “bill of rights” by seeking public comment on how to “support the innovations of big data while at the same time responding to its risks.”

The report also says that data brokers should develop programs to allow consumers to wield more control over how information about them is used. “The data services industry should follow the lead of the online advertising and credit industries and build a common Web site or online portal that lists companies, describes their data practices, and provides methods for consumers to better control how their information is collected and used or to opt-out of certain marketing uses,” the report states.

Sens. Jay Rockefeller (D-W.Va.) and Ed Markey (D-Mass.) introduced legislation to that effect earlier this year. The ad trade group Direct Marketing Association said it opposed the bill, arguing that legislation isn't necessary, and would ultimately “hurt consumers by limiting choices and raising prices.”

The Obama administration's report also backs “do not track” mechanisms that can help consumers prevent some data collection. “Strengthening these tools is especially important because there is now a growing array of technologies available for recording individual actions, behavior and location data across a range of services and devices,” the report says.

The report drew mixed reactions from privacy experts.

Jeff Chester, executive director of the privacy-advocacy organization Center for Digital Democracy, praised the White House for acknowledging the “discriminatory role” that Big Data can play, but also said the report should have explored the “commercial surveillance complex” created by data-driven businesses.

“It should have called for a national debate on whether the pervasive data collection, tracking and targeting system should be tolerated in the first place,” Cheser said in an emailed statement. “We are concerned that the report may give a green light to expanded data collection, where the principle is collect first and worry about privacy and consumer protection later.”

But Jules Polonetsky, co-chair and executive director of the industry-funded think tank Future of Privacy Forum, called the report “a full-barrel effort to move the dialogue forward in a way that demonstrates a deep commitment to the benefits of Big Data, while at the same time balancing appropriate concerns about privacy.”

The Direct Marketing Association's Peggy Hudson, senior vice president of government affairs, said in a statement that the group “continues to believe that self-regulation is the appropriate approach to address complex, dynamic data policy issues.”

 

 

Recommend (3)
6 comments about "White House Warns 'Big Data' Can Lead To Discrimination".
  1. Michael E. Keenan from Keenan & Company , May 1, 2014 at 5:17 p.m.
    This is the tip of the iceberg needing careful thought and review spanning issues of privacy rights, different-answers-for-different-people based on demo profiles in searches, downward pressure on emoloyment levels among others.
  2. Rob Frydlewicz from RAF Consulting , May 2, 2014 at 11:01 a.m.
    This was the case 30 years ago with basic targeting using data from MRI and Simmons. Black groups occasionally protested companies selling liquor and cigarettes because of their tendency to target black neighborhoods. Not that it resulted in companies changing their practice.
  3. David Cearley from self employed , May 2, 2014 at 6:25 p.m.
    Everything we do will now be judged by the government through the lens of social justice, and is in essence, discrimination that can only be measured by those claiming to be discriminated against. Because it's incapable of objective measurement, trial lawyers are going to love it.
  4. Domenico Tassone from Viewthrough Measurement Consortium , May 5, 2014 at 8:34 p.m.
    Gotta love the irony that the biggest privacy violating organization ever (the US Federal government: NSA, CIA, IRS, FBI, DOD, DHS, DOJ) is pointing out the splinter in private industry's eye while ignoring the plank in it's own.
  5. John Casey from Carmichael Industries , May 9, 2014 at 11:41 a.m.
    Pa-the-tic. People must accept responsibility for their own lives and success/failure. Of course perpetuating a victim culture keeps collectivist politicians in power and the myriad victim groups for which they shill.
  6. Domenico Tassone from Viewthrough Measurement Consortium , May 10, 2014 at 11:21 a.m.
    Don't forget ambulance chasers.