B ig data represents a transformational change for creating new value from data in all sorts of settings, including workforce management. At IQN Labs, we’re building out a big data system that will drive better hiring processes based on the most advanced data management and analytic techniques. We call it Big Talent.
Complementing human insight with machine intelligence
The idea of big data has been criticized as mostly marketing hype, promoted by vendors hoping to sell you expensive tools your company doesn’t really need. Some data scientists consider it worse than hype, suggesting that big data is stupid data. The claim there is that insights will be found in right size data sets (often small), not big data sets.
But big data doesn’t replace insight-oriented small data projects; it complements them and often builds upon their findings and approaches. And big data isn’t about spending more money or buying fancy proprietary tools – it’s about building out systems that can (relatively) effortlessly scale out as more and more data becomes available without huge expenditures of money.
Spotting an epidemic
So you can understand how big data accelerates our ability to extract value from data, let’s leave the contingent workforce management space for now and consider a different domain altogether: that of infectious disease tracking. Healthmap, a big data system developed by researchers at Boston Children’s Hospital, recently made the news by raising the alert about a mystery hemorrhagic fever in southeastern Guinea nine days before the World Health Organization announced the Ebola outbreak in West Africa.
Map of Ebola outbreak as of September 23, 2014
Healthmap scrapes the web to find local and international news reports, government reports, physicians’ social networks, and other freely available sources that might have information about outbreaks. This unstructured text data is classified and analyzed then combined with geographic data to generate near-real-time tracking and alerts of disease outbreaks. This provides important information to public health officials, NGO staff, travelers, and anyone else with access to a web browser.
Healthmap doesn’t replace the careful and detailed studies that public health epidemiologists and statisticians carry out; it augments human insight generated by such studies with machine intelligence that can be delivered to a much wider audience than a CDC report, for example, might reach.
Human insight vs machine intelligence
Small data analyses designed to generate insight typically require highly trained data analysts to make sense of the data including carefully considering questions of correlation vs causation, understanding the data set in detail, and producing careful reports with detailed charts to communicate complex results. Such analyses can be supported by traditional business intelligence systems that offer ad hoc reporting and charting, such as that built into the IQNavigator application.
Big data systems take a different approach. Instead of offering ad hoc reporting and charting of carefully cleaned and structured transactional data, big data systems such as Healthmap typically do the following:
- Store and analyze very granular, event-oriented data
- Integrate data from multiple structured and unstructured sources
- Leverage machine learning algorithms to generate machine intelligence
- Rely upon free, open-source software to achieve cheap scalability
- Generate right-time alerts, predictions, and recommendations
- Provide those alerts, predictions, and recommendations to frontline decision makers, not just high level researchers or executives
The following table compares typical characteristics of small data approaches to the big data paradigm.
|Small Data||Big Data|
|Output||Insight in reports generated by humans||Alerts, predictions, suggestions, automated decisions generated by machine intelligence|
|Data set||Samples of structured data extracted and carefully cleaned by hand||Integrated from multiple structured and unstructured data sources|
|Data management||Data warehouses often implemented using relational SQL databases||Hadoop, NoSQL, other open source options|
|Analytical tools||Proprietary statistical tools such as SAS, SPSS, or Stata||Open source languages such as R, Python, or Java|
|Results||Provided to high-level decision makers||Provided to rank and file decision makers|
Big data applied to contingent workforce management
Our Big Talent system will integrate data from our vendor management system with additional outside data to enrich our analyses and machine learning capabilities. It will support the quick and agile exploration of data in small data insight-oriented projects such as the analysis of supplier submission limits we have in process while providing an infrastructure for big data style alerts, predictions, and recommendations.
Big Talent will eventually raise alerts, for example, around supplier performance or rate noncompliance, make predictions around time to fill or rate trends for different job titles, and generate recommendations such as suggested job titles to use for a given job description or resources that match a particular requisition.
Big Talent is a system for IQN Labs’ internal use but IQNavigator users will benefit as machine intelligence generated by Big Talent improves the VMS and thereby the hiring processes on an ongoing basis.
IQN Labs brings data-driven innovation to hiring
IQN Labs has the mission of driving game changing innovation in workforce management through collaborative projects with clients and partners. We think Big Talent will be a key part of delivering on this mission.
Stay tuned for more about how we’re innovating with data!