Skip to content
Skip to navigation menu


Forecasting the Future with Big Social Data

21 March 2013

Social Media Network Visualisation

Social Media Network Visualisation

Internationally recognised social and computer scientists at the Cardiff Online Social Media Observatory (COSMOS) at Cardiff University will analyse tweets to help predict offline social behaviour such as crime and disorder.

The COSMOS Team has been awarded an ESRC National Centre for Research Methods Methodological Innovations grant to examine if Big Social Data can predict offline social phenomena.

There are now over 2.5 billion social media users and this new project will make use of user-generated naturally occurring data produced by online interactions. 

The pioneering study will harvest, store, analyse and interpret a portion of this vast amount of data via the Twitter Firehose to interrogate the potential statistical link between social media updates (in this case tweets) that relate to crime and disorder and official rates of crime as recorded by the police in six London boroughs. The Metropolitan Police Service and the Office for National Statistics are providing administrative and curated data for the project.

The inter-disciplinary project team includes academics from the Cardiff School of Social Sciences (Matthew Williams, William Housley, Adam Edwards, Luke Sloan and Jeff Morgan) from the Cardiff School of Computer Science and Informatics (Pete Burnap and Omer Rana), and from the Universities of Warwick (Rob Procter) and St. Andrews (Alex Voss).

Adam Edwards who is an expert on community safety and co-investigator on the project said: “The potential value added by social media data is that it is user-generated in real-time in voluminous amounts, and as such it can provide insight into the behaviour of populations on the move; the ‘pulse of the city’. This is in contrast to the necessarily retrospective snapshots of social trends and populations provided by conventional methods such as household surveys and officially recorded data.”

The research will draw upon existing mathematical techniques that have used social media data to predict voting patterns, the spread of disease, the revenue of Hollywood movies, and the estimates of the centres of earthquakes. It will also draw on the computational tools developed in the recently awarded COSMOS ESRC Google Data Analytics grant.

Dr Matthew Williams who leads the project said: “These studies illustrate how social media generates naturally occurring socially relevant data that can be used to complement and augment conventional curated data to predict offline phenomena.  In our project, we hypothesise that crime and disorder related tweets will be associated with actual crime rates. If proven correct, our statistical models based on naturally occurring social media data will provide an alternative to official constructions of the crime problem that are derived from curated and administrative data sources.”


The outcomes of the project will be of use to non-academics including the Home Office, the Ministry of Justice, police services, the Association of Chief Police Officers, the new College of Policing, Police and Crime Commissioners, the Office for National Statistics and various voluntary organisations.  The COSMOS team will also develop an advanced Doctoral Training Centre module on Social Media Predictive Analytics.

Dr Williams concluded: “The ESRC is driving the big data agenda forwards working in a range of collaborations with industry to enable the UK to become a world leader in the analysis of social media data. During this project we will be seeking to foster a more permanent relationship with Twitter by applying to their Certified Products Program, establishing COSMOS as the academic social media data repository in the UK.”

Follow COSMOS on Twitter @cosmos_cardiff or visit the COSMOS website for more information:



  1. The Economic and Social Research Council (ESRC) is the UK's largest organisation for funding research on economic and social issues. It supports independent high quality research which has an impact on business, the public sector and the third sector. The ESRC's total budget for 2012/13 is £205 million. At any one time the ESRC supports over 4,000 researchers and postgraduate students in academic institutions and independent research institutes.
  1. The ESRC-funded National Centre for Research Methods (NCRM) aims to stimulate creative and productive developments in social science research methods and to be responsive to new and emerging research and training needs and opportunities as they arise. The Centre’s overall mission is to provide a strategic focal point for the identification, development and delivery of an integrated national research and training programme aimed at promoting a step change in the quality and range of methodological skills and techniques used by the UK social science community.
  1. The aim of the NCRM Methodological Innovation projects is to develop and apply innovative methodological approaches in strategically important areas of social science.
  1. This is the sixth externally funded project awarded to the COSMOS team in the last 18 months.  COSMOS is an Economic and Social Research Council (ESRC) and Joint Information Systems Committee (JISC) half a million pound investment that brings together social, political, health, mathematical and computer scientists to study the methodological, theoretical, empirical and policy dimensions of Big ‘Social’ Data.   Their objective is to establish a coordinated international social science response to this new form of data in order to address next-generation research questions.