The Workshop
The NIC.br Annual Workshop on Survey Methodology was conceptualized with the objective of promoting capacity-building on quantitative and qualitative approaches used for the production and usage of ICT-related statistics, enhancing the importance of solid and rigorous methods for data collection and use among the ICT data community.
The IX edition of this 4-day workshop will address the “Data for public statistics: Data Science, Big Data & Artificial Intelligence”, considering the importance of new sources of data for producing public statistics.
Taking into account the current scenario where massive amounts of data are being produced at fast rates, the days 1 and 2 aims to address the following questions: how can we leverage the potential of big data to improve data production? How is the data ecosystem reconfigured? Which approaches can be used to generate value from this data? What is the role of Data Science in this context? How can big data and artificial intelligence be used for decision-making and which are the challenges and barriers associated with these?.
Days 3 and 4 will also deliver the Short Course: “Quantitative Survey Sampling and Qualitative Sampling” aimed at introducing participants to both survey sampling and qualitative sampling and looking at why good quality sampling is important and how to implement it.
The workshop is composed by lectures and short courses that include interactive activities, practical examples and case studies. No fees are charged and all the course material will be provided at the course.
The workshop is conducted in English and no simultaneous translation service is provided.
Programme
Welcome Remarks
Alexandre Barbosa (Regional Center for Studies on the Development of the Information Society under the auspices of UNESCO (Cetic.br))
Demi Getschko (Brazilian Network Information Center (NIC.br))
Pedro Luis Nascimento Silva (Escola Nacional de Ciências Estatísticas / Instituto Brasileiro de Geografia e Estatística)
Data Production for decision-making: experience-sharing
Moderator: Denise Britz do Nascimento Silva (Escola Nacional de Ciências Estatísticas / Instituto Brasileiro de Geografia e Estatística)
Bernardo Canedo (Instituto Brasileiro de Opinião Pública - Inteligência)
Fernanda Campagnucci (São Paulo City Hall)
Iñigo Herguera (Universidad Complutense de Madrid)
Presentations
ix-nicbr-annual-workshop-campagnucci.pdf
Coffee Break
An Introduction to Data Science
Pedro Luis Nascimento Silva (Escola Nacional de Ciências Estatísticas / Instituto Brasileiro de Geografia e Estatística)
Presentations
Big data in official statistics
Jan van den Brakel (Statistics Netherlands/Maastricht University)
-
National statistical institutes are under increasing pressure to reduce administration costs and response burden for the production of official statistics. This could potentially be accomplished by using large data sets - so called big data. However, there are problems that must be addressed when using such data source for the production of official statistics.
In these sessions, two different research lines will be presented on how big data sources can be used in the production of official statistics. They will be illustrated with research results from projects conducted at Statistics Netherlands.
The first approach to be presented is to combine big data sources with sample data in a model-based inference approach. This implies that big-data sources are used as covariates in models used for small area estimation, for example in an area level model where cross-sectional correlation between areas are used to improve the effective sample size of the domains.
The second approach is to use big data sources as a primary data source for the compilations of official statistics. This can be considered if a big data source covers the intended target population and not suffer to much from under- and over-coverage, e.g. the use of satellite and areal images for deriving statistical information on land use. In most cases, however, adjustments for selection bias are required.
Presentations
ix-nicbr-annual-workshop-brakel-2.pdf
ix-nicbr-annual-workshop-brakel-3.pdf
ix-nicbr-annual-workshop-brakel-4.pdf
Lunch Break
Big data in official statistics
Jan van den Brakel (Statistics Netherlands/Maastricht University)
-
National statistical institutes are under increasing pressure to reduce administration costs and response burden for the production of official statistics. This could potentially be accomplished by using large data sets - so called big data. However, there are problems that must be addressed when using such data source for the production of official statistics.
In these sessions, two different research lines will be presented on how big data sources can be used in the production of official statistics. They will be illustrated with research results from projects conducted at Statistics Netherlands.
The first approach to be presented is to combine big data sources with sample data in a model-based inference approach. This implies that big-data sources are used as covariates in models used for small area estimation, for example in an area level model where cross-sectional correlation between areas are used to improve the effective sample size of the domains.
The second approach is to use big data sources as a primary data source for the compilations of official statistics. This can be considered if a big data source covers the intended target population and not suffer to much from under- and over-coverage, e.g. the use of satellite and areal images for deriving statistical information on land use. In most cases, however, adjustments for selection bias are required.
Presentations
ix-nicbr-annual-workshop-brakel-6.pdf
ix-nicbr-annual-workshop-brakel-7.pdf
Coffee Break
Challenges in the Production of Official Statistics with Different types of Data sources
Danny Pfeffermann (Central Bureau of Statistics of Israel)
Presentations
Readings
9-nicbr-annual-workshop_extramaterial-pfeffermann.pdfOECD Measuring the Digital Transformation and Going Digital Toolkit
Daniel Ker (Organisation for Economic Co-operation and Development)
-
"Measuring the Digital Transformation: A Roadmap for the Future" (https://oe.cd/mdt) was launched by the OECD at the Going Digital Summit in March 2019. It provides new insights into the state of the digital transformation by mapping indicators across a range of areas – from education and innovation, to trade and economic and social outcomes – against current digital policy issues, as presented in the accompanying publication "Going Digital: Shaping Policies, Improving Lives" (https://oe.cd/gdreport). In so doing, it identifies gaps in the current measurement framework, assesses progress made towards filling these gaps and sets-out a forward-looking measurement roadmap. The goal is to expand the evidence base, as a means to lay the ground for more robust policies for growth and well-being in the digital era.
Alongside these, the "Going Digital Toolkit" (https://www.oecd.org/going-digital-toolkit) helps countries assess their state of digital development and formulate policy strategies and approaches in response. Data exploration and visualisation are key features. The Going Digital Toolkit is structured along the 7 policy dimensions of the Going Digital Integrated Policy Framework, which cuts across policy areas to help ensure a whole-of-economy and society approach to realising the promises of digital transformation for all.
Presentations
Readings
9-nicbr-annual-workshop_extramaterial-ker.pdfMake Measurement Matter: Leveraging Big Data and AI for Monitoring and Promoting Sustainable Human Development.
Emmanuel Letouzé (DataPop Alliance)
-
A decade into the “Data Revolution”, many questions remain about the real potential of ‘Big Data’, and increasingly AI, to meaningfully contribute to sustainable human development, including the monitoring and promotion of the SDGs. Many papers and pilots have showed how analysis of those ‘digital breadcrumbs’, most of which collected and controlled by private companies, could shed light on human processes and outcomes at very fine levels of temporal and geographic granularities. But to date there are no systems nor standards developed and even less deployed to unlock this potential at scale, safely, and ethically. In a ‘post-truth’ age there is also a need to reconsider commonly held assumptions about the role and limitations of measurement in shaping decisions; often data and facts do not seem to matter very much. And yet making measurement matter for development still seems like a simple and powerful way to foster progress. How can this be done, in the age of Big Data and AI? What does and will it take in the next decade and beyond for the ‘data generations’—these children and teenagers growing in a dataified world? Based on the experience and perspectives of Data-Pop Alliance, OPAL ("Open Algorithms"), and others, this presentation will seek to provide context, suggest options, and foster discussions on how Big Data and AI may help monitor and promote sustainable human development.
Presentations
Readings
9-nicbr-annual-workshop_extramaterial-letouze.pdfCoffee Break
Make Measurement Matter: Leveraging Big Data and AI for Monitoring and Promoting Sustainable Human Development.
Emmanuel Letouzé (DataPop Alliance)
-
A decade into the “Data Revolution”, many questions remain about the real potential of ‘Big Data’, and increasingly AI, to meaningfully contribute to sustainable human development, including the monitoring and promotion of the SDGs. Many papers and pilots have showed how analysis of those ‘digital breadcrumbs’, most of which collected and controlled by private companies, could shed light on human processes and outcomes at very fine levels of temporal and geographic granularities. But to date there are no systems nor standards developed and even less deployed to unlock this potential at scale, safely, and ethically. In a ‘post-truth’ age there is also a need to reconsider commonly held assumptions about the role and limitations of measurement in shaping decisions; often data and facts do not seem to matter very much. And yet making measurement matter for development still seems like a simple and powerful way to foster progress. How can this be done, in the age of Big Data and AI? What does and will it take in the next decade and beyond for the ‘data generations’—these children and teenagers growing in a dataified world? Based on the experience and perspectives of Data-Pop Alliance, OPAL ("Open Algorithms"), and others, this presentation will seek to provide context, suggest options, and foster discussions on how Big Data and AI may help monitor and promote sustainable human development.
Presentations
Readings
9-nicbr-annual-workshop_extramaterial-letouze.pdfLunch Break
Use of Computational Tools to Support Planning and Policy
Johannes Bauer (Michigan State University)
-
The availability of rich and detailed data has greatly improved the ability of policy analysts and policy makers to develop better policy. Big data can also be utilized to improve the design and implementation of policies intended to advance fixed and wireless connectivity and to overcome second and third generation digital divides. One weakness of reliance on big data for purposes of policy design is that is inherently reflects past arrangements and relationships between players in the ICT industries. In as far as policy seeks to create a different future, it needs to augment the insights from big data analytics. One approach is computational modeling, which can help explore and evaluate different courses of action and their effects on social and economic outcomes. Examples that take advantage of increased data availability and computational power include scenario building, agent-based modeling, computer simulations. Designed for practitioners, the session will discuss these issues and illustrate the uses and limitations of computational methods for current policy issues (e.g., network neutrality, 5G deployment).
Presentations
Readings
9-nicbr-annual-workshop_extramaterial-bauer.pdfCoffee Break
AI & Ethical Implications
Diogo Cortiz (Web Technology Study Center (Ceweb.br)/ Pontifical Catholic University of São Paulo)
AI & Ethical Implications
Caitlin Sampaio Mulholland (Pontifical Catholic University of Rio de Janeiro)
Presentations
ix-nicbr-annual-workshop-mulholland-2.pdf
Readings
9-nicbr-annual-workshop_extramaterial-mulholland.pdfWelcome Cocktail
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Coffee-break
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Lunch
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Coffee-break
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Coffee-break
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Lunch
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.
Coffee-break
Quantitative Survey Sampling and Qualitative Sampling
Pamela Campanelli (The Survey Coach)
-
Do you have the right sampling design for your quantitative survey or for your qualitative focus groups or depth interviews? A poor sampling design can make or break a study. This course introduces participants to both survey sampling and qualitative sampling; looking at why good quality sampling is important and how to implement it. The course contains lots of practical advice and many workshops to put theory into practice. On the quantitative side, it does make use of mathematics and statistics, but it has been designed for all participants (those with little or no statistical background through to those who use statistics regularly). The topics covered include: types of samples (probability versus non-probability), how to construct a ‘sampling frame’, types of probability samples (e.g., simple random, systematic, stratified, multi-stage clustered, unequal probabilities of selection, probability proportional to size and flow sampling), what ‘sampling error’ is, the role of sampling error in confidence intervals, how to determine sample size, how a complex sampling design affects confidence intervals, issues with disproportionate designs and the impact on weighting and a brief introduction to weighting. On the qualitative side, the topics covered include: different types of samples (convenience, purposive, theoretical, heterogeneous and homogeneous), sampling units, sample size, and special tips for focus group samples.