Big data = Big problems + Simple solutions?
Big data is a big trend in today’s corporate world. The general assumption about big data is that it “allows companies to make meaningful, strategic adjustments that minimize costs and maximize results.”(1).The headlines are full of stories about the use of big data by global businesses such as Google, TripAdvisor, Uber, eBay, Netflix, and Amazon. Most of them use the large junks of data to predict and through that influence customers’ buying behavior (2). In other words, big data is assumed to help large companies in industrialized economies. Despite its potential, many express worries about the validity and reliability of sources, applications and analysis of big data. This is especially relevant for companies overwhelmed with the data availability (see e.g. Lazer et al., 2014 for details about the “Google Flu” case).
However, despite the “corporate” image of big data, its potential have also been noticed elsewhere. In this post, I will concentrate on the opposite to large companies in industrialized economies with unlimited resources context – the use of big data by small groups of scientists working with limited resources on big problems in world’s poorest countries. In the following, I will present the short descriptions of two journal articles that cover the topic of mapping and predicting the poverty in world’s poorest countries. The reason for choosing these two articles are twofold: first, the journal (Science (3) is of the highest impact factor (37.205 in 2016) indicating high validity and reliability of presented findings. Second, both articles are rather recent (2015&2016) providing me with assurance about the relevance of the presented information.
Predicting and reducing poverty
The first paper (Blumenstock et al., 2015) discusses the possibilities to predict poverty from mobile phone metadata. Describing these possibilities, authors indicate that despite the lack of big data sources in developing countries, the mobile data coverage of all of these countries is good enough to provide researchers with reliable data. Authors describe the method how the information derived from the call records of 1.5 million subscribers in Rwanda overlaid on the country’s map providing scientists and officials with reliable information about the level of poverty within each particular district of the entire country.
Being constrained by the need to rely on solely publically available data, the authors of the second paper (Jean et al., 2016) developed another solution. Mapping the variation in local-level economic outcomes of five African countries (Nigeria, Tanzania, Uganda, Malawi, and Rwanda) they relied on a combination of a publicly available data from the daytime satellite imagery and machine learning. Despite their inability to generate as precise and detailed map as in the first study, the method used by the researchers (all from Stanford) provided them with an opportunity to explain up to 75% of the variation in local-level economic outcomes in each of the selected countries.
What did I learn from these two stories?
- Big data does not always have to serve only big corporations,
- Big data does not always have to be complicated,
- Big data does not always have to require complex solutions
- Big data can help “small” people
- …
- …
Please feel free to continue this list with your own opinions and examples.
References:
- https://www.forbes.com/sites/danielnewman/2015/12/22/the-role-big-data-plays-in-digital-transformation/#24314bf975d3
- https://hbr.org/2016/08/use-big-data-to-create-value-for-customers-not-just-target-them
- http://www.sciencemag.org/
- Blumenstock, J., Cadamuro, G., & On, R. (2015). Predicting poverty and wealth from mobile phone metadata. Science, 350(6264), 1073-1076.
- Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016). Combining satellite imagery and machine learning to predict poverty. Science, 353(6301), 790-794.
- Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: traps in big data analysis. Science, 343(6176), 1203-1205.
5 comments on “Big data = Big problems + Simple solutions?”
Comments are closed.
(238A) Dear Natasha,
another really insightful article! I like the approach of trying to use Big Data, or new technologies in general, for developmental purposes. As the developing countries have to “skip” some stages of development with regard to the history of developed countries, it is most important in my eyes to give them all possibilities to use or foster innovation in their own countries.
But especially in poor countries, we are often facing unstable states with less developed law and regulative systems. In such an environment the usage of “poornes data”, for example, which is a pretty sensible field in my eyes, must be very restricted and controlled. If those mechanisms and technologies get into the wrong hands or are being used by large global corporations, the trend can easily have negative effects on those societies or can even manifest the problems (of inequality, poorness, etc.). So in a gist, my point is that, especially in such environments as in the developing world, protection of privacy, sound regulation and inclusion is most important.
Another thought, following Mr. Chan’s presentation from last week, is the question of how effective the people in poor circumstances can use Big Data. As you and the papers have indicated, there are indeed some problems. From all the “V’s”, I think that the Volume, Velocity and Variety of data might be a big problem. For velocity there must be infrastructure improvements on the ground. Volume is probably hard to achieve in the most cases. And a variety might also not be provided in less developed countries.
Hopefully the developments you described will go on and allow the poor ones to not just make better and more effective use of Big Data, but also to go their own way, without a dominant “guiding hand” from big corporations, in order to solve the problems on the ground.
I love this post! First, thank you for presenting two recent articles that are highly relevant to your topic. The idea from your post that inspires me most is that “Big data can be used on small targets”: not only can big data be exploited by big companies to maximize their profits in “industrialized economies”, it can also be helpful in developing areas.
The points you listed as summarizations convince people that big data are not as difficult to use as they might have imagined. So I hope not only the data resources obtained by researchers are boosted, but also the number of inspired people who dedicate to mining and utilizing big data to solve real-world problems.
Great article!
The papers mentioned how they were able to analyze poverty from cell phone usage, and that even with low amounts of data they were able to create an overall reliable picture of poverty in developing countries. How were they able to collect data, especially in countries where blackouts and other energy issues are very common and may intervene in the process of getting reliable results?
In addition, how can this be used in developed countries? Wouldn’t it be easier to get results, since mobile phone usage is much higher than in developing countries? Or is it easier to do through other methods? If that’s the case, wouldn’t other methods also be applicable to developing countries?
Data has been an enormous source of insights lately. As you pointed, more and more data are now turning int interesting insights shape out business, marketing and sales. In one of my company that I worked, security has huge gap with terabytes of data storage that occurs almost daily and there has been now recently more and more security analytics product are available to search and build reports for executive and finding root cause analysis. This has made more engineering resources to save time have almost a realtime data sources to analyze them than every before because of power of BIG DATA!
I agree that big data can solve problems associated with poverty, but I believe it has the potential to impact rich and poor alike. Even though big data may have a “corporate” image, I believe that applications of big data can have huge positive impact on industries which transcend the boundaries of wealth.
Big data has the potential to transform the healthcare industry (1). Breakthroughs in medicine, such as a cure for diabetes or the flu, will still reach people of small means, even if funded by large corporations. Whether through government welfare programs or private healthcare, medicine is something that reaches everyone.
Our prediction of natural disasters stands to improve by using big data, resulting in rich and poor lives saved (2). Even if the technology is perfected in an immaculate lab at Stanford, it will go on to benefit even those who live in rural areas but are still affected by natural disasters.
Big data also improves our life when it comes to commute. We spend years of our lives commuting, but big data such as that used by Google Maps to determine traffic can reduce that significantly. Google Maps is a resource as omnipresent in our world as cellphones, and is just one instance of how big data can affect our daily lifestyles.
1. http://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/the-role-of-big-data-in-medicine
2. https://datafloq.com/read/how-big-data-will-be-used-predict-next-disaster/2434