SPECIAL FEATURE
BIG DATA
Modernising the foundations of records and information systems:
Big Data BY LINDA SHAVE
Modernising the foundations of records and information systems in the era of Big Data 3.0 is the alignment of IT, business strategies and the vision within the organisation. Organisations need to identify and determine which improvements provide the best opportunities for transformation, productivity, governance and economic value.
WHAT ARE THE BIG DATA STAGES?
Table 1 – Three Stages of Big Data
The term ‘Big Data’ has been in use since the early 1990s, it could be said that this was the start of Big Data 1.0. Table 1 provides a brief overview of the Big Data stages.
STAGE
WHAT ARE THE BIG DATA TRENDS?
Big Data 1.0
Big data trends for capturing, storing and processing complex unstructured and semi-structured data is a fast-moving target for organisations. Big data trends are continuously evolving to meet government, citizen and economic drivers. The following outlines four potential big data trends:
Transactional – Early 1990s
1. Data Fabrics
Big Data 2.0
A big ‘data fabric’ is an augmented data management architecture that provides visibility of data and the ability to move, replicate and access data across multiple hybrid and cloud repositories. It is a metadata-driven approach to connecting disparate collections of data repositories, devices and tools to deliver capabilities in the areas of data access, discovery, transformation, integration, graph modelling, security and governance. For a business, having a data fabric means that data is free to move dynamically across all private and public cloud resources for greater efficiencies.
Networked – Early 2000s
Data Fabrics depend heavily on contextual information that integrate with pools of operational metadata, technical metadata and utilise machine learning (ML) to enhance data quality with learning models that can be trained and continuously learn from patterns to improve metadata analytics and business outcomes. 28 | iQ June 2022
Big Data 3.0 Intelligent – Early 2010 to date
DESCRIPTION Terms such as data analysis and data analytics for structured data originated from the field of database management. It relied heavily on the storage of data in Relational Database Management Systems (RDBMS). Techniques such as database queries, online analytical processing, dashboards, scorecards and standard reporting tools are common in data that is stored in RDBMS. Web/internet and the augmented creation and capture of digital data introduced an immense increase in the collection and storage of semi-structured and unstructured data types. Besides the standard structured data types, organisations now needed to find new approaches, tools, technologies and analytical techniques to deal with these data types in order to extract meaningful information.
Around 2010, we saw the increased capture and storage of mobile, real time and sensor-based data and the beginning of location awareness, person centred and context relevant analysis. This has required once again organisations to find new approaches, tools, analytical techniques and technologies. As an outcome we are now seeing the rise of artificial intelligence (AI), natural language processing (NL) and machine learning (ML) for data analytics. AI and ML algorithms are able to process the large volumes of data and produce detailed insights, highlight trends and provide valuable and actionable outcomes.
