In the years following India’s Independence, one of the primary challenges faced by the new nation was the lack of reliable statistical data, which were crucial for shaping sound economic and social policies. The newly formed government recognised that in the absence of comprehensive data, making informed decisions regarding economic growth, poverty alleviation, and social welfare would be exceedingly difficult. This gap in data led to the formation of the National Income Committee in 1949, chaired by P.C. Mahalanobis, the then Statistical Adviser to the Cabinet and the “Father of Indian Statistics”. Under Mahalanobis’s stewardship, the committee identified a glaring deficiency in statistical data, particularly regarding the unorganised and household sectors of the economy.
To address the pressing need for reliable data, the National Income Committee recommended the creation of a specialised statistical body dedicated to the systematic collection of comprehensive national data. Thus, the National Sample Survey (NSS) was established. The first round of the NSS was conducted from October 1950 to March 1951, with a primary focus on rural areas. The survey aimed to collect data on multi-subject enquiry in socio-economic variables, including household consumption expenditure, agriculture, livestock, household industries, and financial services. This initiative marked the beginning of a monumental effort to collect high-quality statistical data to better understand the complexities of India’s socio-economic landscape.
Policy formulation, planning
The formation of the NSS represented a watershed moment in India’s statistical and policymaking infrastructure. The comprehensive data collected by the NSS became an invaluable resource for policymakers, providing the insights needed to craft effective policies on issues spanning from economic development to poverty alleviation and social welfare. The NSS has, over the years, served as the principal statistical tool for informing India’s policy decisions, helping to shape the direction of national development. Its data have guided major initiatives and schemes aimed at reducing poverty, enhancing education, and improving healthcare across the country.
As the NSS celebrates its 75th year, its role in the nation-building process remains as critical as ever. The survey continues to be an indispensable instrument for the government in formulating evidence-based policies, particularly those that address the challenges faced by India’s diverse population. The insights derived from the NSS have been instrumental in shaping India’s national development agenda, positioning the country towards becoming a Viksit Bharat by 2047 — a vision for a developed, prosperous, and inclusive nation.
Significant milestones
Over its seven-decade history, the National Sample Survey has been a cornerstone of India’s socioeconomic development. One of the earliest and most impactful contributions of the NSS was its role in the mid-1970s when it revealed the high incidence of poverty in India. These findings, based on rigorous sampling, were a catalyst for several landmark social welfare programmes. Among the key initiatives that emerged in response to the NSS’s findings were:
Public Distribution System (PDS): The PDS, which existed in its nascent form, was strengthened significantly in the wake of the NSS’s reports on widespread poverty and food insecurity. The programme aimed to provide subsidised foodgrains to economically vulnerable sections of society.
Integrated Child Development Services (ICDS): Introduced in 1975, the ICDS sought to address the nutritional, health, and developmental needs of children, particularly in rural and underserved areas. The programme focused on improving child survival and overall well-being.
Food-for-Work (FFW) programme: Launched in 1977-78, this initiative aimed at providing employment opportunities while simultaneously addressing hunger through the provision of food in exchange for labour. The best part of this scheme was that it addressed two major problems at once — unemployment and hunger. Later on, in 2006, this scheme, with many changes in policy enactment, implementation, and control, was absorbed into the National Rural Employment Guarantee Act (now known as MGNREGA), which is now the most important right-based employment guarantee scheme in the country.
Technological advancements
As the years progressed, the NSS evolved in tandem with advancements in technology, particularly in the field of data collection and processing. One of the most significant shifts in the survey methodology was the transition from pen-and-paper personal interviews (PAPI) to computer-assisted personal interviews (CAPI), where data collection is carried out using tablets. This technological leap greatly enhanced the efficiency and accuracy of data collection, allowing the NSS to collect real-time data and submit it incrementally, thereby speeding up the data compilation process.
The adoption of CAPI technology has been transformative for the NSS. It allows for more efficient management of field data, resulting in the faster release of survey results. This, in turn, enables policymakers to access timely information, facilitating prompt decision-making that is crucial for the effective implementation of policies and programs.
Global recognition
The NSS’s influence extends beyond India’s borders. As India’s statistical capabilities have grown, the NSS has garnered recognition globally for its contributions to international statistical practices. In its 75th year, India is again elected as a member of the United Nations Statistics Division (UNSD). This membership allows India to contribute to the development of global statistical standards, ensuring that the data collected by the NSS aligns with international norms and methodologies.
Furthermore, India has recently become a member of the United Nations Committee of Experts on Big Data and Data Science for Official Statistics (UN-CEBD), which shows India’s growing influence in the global statistical community. As the world increasingly turns to big data to inform policymaking, India’s expertise in managing large datasets and its experience in conducting national sample surveys make it a valuable contributor to shaping the future of global statistical standards.
Challenges
While the NSS has achieved remarkable success throughout its history, it faces several challenges that could impact its continued effectiveness. One of the most pressing issues facing the NSS, and surveys in general, is the rising non-response rate. This challenge is not unique to India but is a global concern that affects the reliability and representativeness of survey data. Over the years, non-response rates have steadily increased exponentially, making it more difficult to achieve the desired accuracy in survey results.
This rising trend poses a significant challenge to the NSS’s ability to collect a representative sample of the population, thereby affecting the overall data quality.
To address this issue, several approaches can be explored:
Public education: educating the public about the importance of participating in NSS surveys could help to reduce the non-response rate. In fact, participation in NSS surveys could be considered a fundamental duty under the Constitution, which is not enforceable by law. But imparting the importance of responsibility through the lens of the constitution from childhood will foster responsible individuals who contribute to the betterment of society.
Incentivisation: offering small, targeted incentives, such as monetary rewards or discounts on essential goods, could encourage greater participation in NSS surveys. These incentives would provide tangible benefits to respondents, making participation more appealing and for wider publicity.
Role of AI
As technological innovations continue to shape the future of data collection and analysis, Artificial Intelligence (AI) is poised to play a significant role in the future of the NSS. AI has the potential to enhance the speed and accuracy of data analysis, automate report writing, and even identify patterns in large datasets that would otherwise go unnoticed. Additionally, AI could help in improving survey design by predicting trends and anomalies in the data.
However, despite these advancements, field data collection through human interaction will continue to be essential in the NSS methodology. While AI can assist with data analysis, it cannot replicate the nuanced understanding and contextual insights that can only be obtained through direct engagement with survey respondents. Therefore, while AI will play a supportive role in the analysis phase, it cannot replace the fundamental need for human-based data collection.
The National Sample Surveys have played an indispensable role in the socio-economic development of India over the past 75 years. The NSS has shaped national policy and guided India’s growth in providing accurate, representative data on a wide range of socio-economic indicators. However, as we move into the future, the NSS must continue to adapt to emerging challenges and technological advancements to remain relevant in an increasingly data-driven world.
Strengthening the NSS will require addressing challenges such as rising non-response rates, leveraging AI and other technological tools for data analysis, and maintaining the core value of direct human interaction in field data collection. By embracing these advancements while staying true to its roots, the NSS can continue to serve as a vital tool for national development and a key player in shaping India’s future.
As the NSS celebrates its 75th anniversary, it remains a cornerstone of India’s governance framework, continuing to provide invaluable insights that inform the country’s policies and development strategies. The next phase of the NSS’s evolution will undoubtedly be critical in ensuring that India’s data collection and analysis capabilities remain at the forefront of global standards, contributing to the achievement of Viksit Bharat by 2047.
(Dr. Chandrasekar is a Deputy Director in the Ministry of Statistics and Programme Implementation; Dr. Usha is an Assistant Professor t Sri Sarada College for Women, Salem; views expressed are personal)
