In this we learn about Big Data, Slides of Information and Communications Technology (ICT)

Download In this we learn about Big Data and more Slides Information and Communications Technology (ICT) in PDF only on Docsity! BIG DATA Big data is a term used to refer to data sets that are too large or complex for traditional data-processing application software to adequately deal with . Data with many cases (rows) offer greater statistical power, while data with higher complexity (more attributes or columns ) may lead to a higher false discovery rate .Big data challenges include capturing data , data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy and data source. Big data was originally associated with three key concept: volume, variety and velocity. Current usage of the term “big data” tends to refer to the use of predictive analytics, or certain other advanced data analytics methods that extract value fro m data, and seldom to a particular size of data set. “there is little doubt that the quantities of data now available are indeed large, but that’s not the most relevant characteristic of this new data ecosystem ”. Analysis of data sets can find new correlations to “spot business trends, prevent diseases, combat crime and so on”. Data sets grow rapidly- in part because are increasingly gathered by cheap and numerous information- sensing internet of things devices such as mobile device, aerial (remote sensing) software logs, cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor networks. The world’s technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5exabytes (2.5*1018) of data are generated. Relational database management systems, desktop statistics[clarification needed] and software packages used to visualize data often have difficulty handling big data. The work may require “massively parallel software running on tens , hundreds, or even thousands of servers”. What is big data and why it matters? Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. But it’s not the amount of data that’s important. It’s what organizations do with the data that matters. Big data can be analyzed for insights that lead to batter decisions and strategic business moves. Who uses it and how it works? Big Data History and Current Considerations:- While the term “Big data” is relatively new, the act of gathering and storing large amount of information for eventual analysis is ages old. The concept of gained momentum in the early 2000s when industry analyst Doug Laney articulated the now-mainstream definition of big data as the three Vs. 1. Volume:- Organizations collect data from a variety of sources, including business transactions, social media and information from sensor or machine- to-machine data. In the past, storing it would’ve been a problem but new technologies have eased the burden. 2. Velocity:- Data streams in at an unprecedented speed and must be dealt with in a timely manner. RFID tags, sensors and smart metering and driving the need to deal with torrents of data in near-real time. 3. Variety:- Data comes in all types of formats from structured ,numeric data in traditional databases to unstructured text documents, email, videos, audio, stock ticker data and financial transactions. Why is big data important? The importance of big data doesn’t revolve around how much data you have, but what you do with it. You can take data from any source and analyze it to find answers that enable 1) cost reductions, 2) time reductions, 3) new product development and optimized offerings, and 4) smart decision making. When you combine big data with high-powered analytics, you can accomplish business-related tasks such as: Determining root causes of failures , issues and defects in near-real time. Big data in today’s world:- Big data – and the way organizations manage and derive insight from it – is changing the way the world uses business information. Who uses big data? Big data affects organizations across practically every industry . See how each industry can benefit from this onslaught of information. Banking: With large amounts of information streaming in from countless sources, banks are faced with finding new and innovative ways to manage big data. While it’s important to understand customers and boost their satisfaction, it’s equally important to minimize risk and fraud while maintaining regulatory compliance. Big data brings big insights, but it also requires financial institutions to stay one step ahead of the game with advanced analytics. Education: Educators armed with data-driven insight can make a significant impact on school system, students and curriculums. By analyzing big data, they can identify at-risk students, make sure students are making adequate progress, and can implement a better system for evolution and support of teachers and principals. Government: When government agencies are able to harness and apply analytics to their big data, they gain significant ground when it comes to managing utilities, running agencies, dealing with traffic congestion or preventing crime. But while there are many advantages to big data, governments must also address issues of transparency and privacy. Health Care: Patient records, treatment plans, prescription information. When it comes to health care, everything needs to be done quickly, accurately –and, in some cases, with enough transparency to satisfy stringent industry regulations. When big data is manage effectively, health care providers can uncover hidden insights that improve patient care. Manufacturing: Armed with insight that big data can provide, manufacturers can boost quality and output while minimizing waste – processes that are key in today’s highly competitive market. More and more manufacturers are working in an analytics-based culture, which means they can solve problems faster and make more agile business decisions. Retail: Customer relationship building is critical to the retail industry –and the way to manage that is to manage big data. Retailers need to know the best way to market to customers, the most effective way to handle transactions, and the most strategic way to bring back lapsed business. Big data remains at the heart or all those things. Data Exploration and Visualization: SAS makes it easy to understand what your data has to tell you. Interactively explore billions of rows of data in seconds. HOW IT WORKS? Before discovering how big data can work for your business, you should first understand where it comes from. The source for big data generally fall into one of three categories: Streaming data:- This category includes data that reaches your IT systems from a web of connected devices, often part of the IT. You can analyze this data as it arrives and make decision on what data to keep, what not to keep and what requires further analysis. Social media data:- The data on social interactions is an increasingly attractive set of information, particularly for marketing, sales and support functions. It’s often in unstructured forms, so it poses a unique challenge when it comes to consumption and analysis. Public available sources:- Massive amount of data are available through open data sources like the US government’s data.gov, the CIA world or the European Union Open Data Portal. How to store and manage it: Whereas storage would have been a problem several year ago, there are now low-cost options for storage data if that’s the best strategy for your business. Big data is an evolving term that describes a large volume of structured, semi-structured and unstructured data that has the potential to be mined for information and used in machine learning projects and other advanced analytics applications. Big data is often characterized by the 3Vs: the extreme volume of data, the wide variety of data types and the velocity at which the data must be processed. Those characteristics were first identified by Gartner analyst Doug Laney in a report published in 2001. More recently, several other have been added to descriptions of big data, including veracity, value and variability. Although big data doesn't equate to any specific volume of data, the term is often used to describe terabytes and petabytes of data captured over time. Big data is a collection of data from various sources ranging from well-defined to loosely defined, derived from human or machine sources. But as data collection and use has increased, so has data misuse. Concerned citizens who have experienced the mishandling of their data or been victims of a data breach are calling for laws around data collection transparency and consumer data privacy. How has your organization used big data to gain a competitive edge? Big data can be contrasted with small data, another evolving term that's often used to describe data whose volume and format can be easily used for self- service analytics. A commonly quoted axiom is that "big data is for machines; small data is for people.”