Methods and Applications for Data Analysis

The acquisition of data relates to the information process, with the major task to aggregate information in a digital form for further process. And the acquisition process consists of three divisions, that is, data collection, data transmission and preprocessing. First, data can come from a variety of sources, including websites text, images, videos-related data, thus some collection technology is needed to acquire raw data from specific data production environment. Secondly, once raw data collected, we need a super fast transmission mechanism to transfer the data to various types of executing applications as well as the targeted storage system, commonly a data center.

The transmission procedure can be classified into two phases, IP backbone transmission and data center transmission.

We should point out that, the harvested data sets could have various quality levels in terms of consistency, redundancy, noise and so on, due to their diverse reasons. Also, some methods and apps for data analysis may have corresponding yet different data quality requirements.

Therefore, data preprocessing techniques are designed and implemented to improve the quality of data in big data systems. Data storage involves the constant storage and management of large data sets.

Get quality help now

Prof. Finch

Verified

Proficient in: Information Technology

4.7 (346)

“ This writer never make an mistake for me always deliver long before due date. Am telling you man this writer is absolutely the best. ”

+84 relevant experts are online

Hire writer

There are two parts of a data storage system: hardware infrastructure and data management. Hardware infrastructure consists of pooled resources that are symmetrically organized to meet their urgent demand for various tasks. The hardware infrastructure should be able to scale up and out and be dynamically configured to address various types of applications and demands. In addition to the physical infrastructure, data management software is applied to maintain and manage large data sets.

It needs to mention that the storage infrastructure could be understood from different perspectives. First, storage devices can be subdivided based on the distinctive technology. Typical storage technologies include, but are not limited to: Random Access Memory, Magnetic Disks and Disk Arrays, and Storage Class Memory. In parallel, storage infrastructure could be demonstrated from a networking architecture perspective. In this category, the storage system can be organized in different ways, including, but not limited to: Direct Attached Storage (DAS), Network Attached Storage (NAS), and Storage Area Network (SAN).

Data analysis utilizes analytical methods or tools for the processing, transformation and modeling of value data, mostly to provide business insight. Many application fields use abundant data and field-specific analytical methods to achieve the intended impact. Although different fields have different requirements for data characteristics, similar underlying technologies have been developed and utilized in some of these fields. Current analytical industrial fields can be divided into several key areas, including text analysis, audio analysis, video analysis and social media analysis, which will be illustrated in details as below.

Text analysis, or text mining, refers to techniques that extract textual data information. Text is one of the most common forms of stored information and includes emails, documents, web pages, and social media content. Examples of textual data held by organization or corporations are feeds from emails, online forum, social networks, surveys, documents, news, etc. Text analysis could be conducted by statistical analysis, machine learning algorithms, and computer linguistic. Text analysis allows companies to transform large amount of human generated text into meaningful summaries, which support responsive decision-making based on information and evidences.

A good example is text-analysis-based prediction of stock market based on info from financial news. Audio analytics analyze data from raw audio data and extract useful summaries from them. Audio analytics also means speech analysis when applied to human natural language. Audio analytics and speech analysis are often used interchangeably, since these techniques are mainly used for spoken audio. Healthcare service providers and customer call centers are currently the primary audio analytics application regions.

Video analysis, involves a variety of techniques for monitoring, extracting and analyzing significant information from video streams. Video analytics are still lagging behind compared to other types of data mining, but different techniques for the processing of real-time videos have already been developed. The growing popularity of websites for video sharing are the two main driving forces to the growth of computational video analysis. However, the vast size of video data is a major challenge for this field. Instead of cost-intensive and error-prone manual processing, big data technologies are being leveraged to automatically implement intelligence from super big video streams.