08-16-2017, 11:56 PM
[attachment=3888]
Integration Of Data mining And Data warehousing Systems
Presented By:
- N.Nagaraju
3/3 M.C.A I Semester
Introduction:
Traditionally, organizations use data tactically - to manage operations. For a competitive edge, strong organizations use data strategically to expand the business, to improve profitability, to reduce costs, and to market more effectively. Data mining creates information assets that an organization can leverage to achieve these strategic objectives. Data Mining is the process of extracting knowledge hidden from large volumes of raw data. We define data mining as "the data-driven discovery and modeling of hidden patterns in large volumes of data."
Data mining: the extraction of hidden predictive information from large databases, is a powerful new technology with great potential to help companies focus on the most important information in their data warehouses. Data might be one of the most valuable assets of your corporation - but only if you know how to reveal valuable knowledge hidden in raw data. Data mining allows you to extract diamonds of knowledge from your historical data and predict outcomes of future situations.
Data warehousing: Integrating data from multiple sources into large warehouses and support on-line analytical processing and business decision making. The necessity of data warehousing is Data explosion problem-- automated data collection tools and mature database technology lead to tremendous amounts of data stored in databases.
The actual need of data warehouse is
To store wast and heterogeneous data for managerial decision purpose.
We can store data in various dimensions with in a data warehouse. So, it is easy to analyze the data and to take decisions.
A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of management s decision-making process.
-- W. H. Inmon
A data warehouse is architecture, a semantically consistent data store to fulfill different data access and reporting requirements, or an on-going process that blends data from multiple heterogeneous sources to support the continuing need for structured and /or ad hoc queries, analytical reporting, and decision support.
We have different types of methods to do modeling of data warehouses, they are
Star schema: A single object in the middle connected to a number of objects radially.
Snowflake schema: A refinement of star schema where the dimensional hierarchy is represented explicitly by normalizing the dimension tables.
Fact constellations: Multiple fact tables share dimension tables
OLAP: On-Line Analytical Processing:
A multidimensional, LOGICAL view of the data. We use OLAP operations to analytical processing of data stored in the form of data cubes in data warehouses. The
OLAP techniques contains interactive analysis of the data like drill, pivot, slice_dice, filter etc., Analytical modeling contains deriving ratios, variance, etc. and involving measurements or numerical data across many dimensions. Summarization and aggregations at every dimension intersection. OLAP methods are useful due to the following facilities,
Forecasting, trend analysis, and statistical analysis.
Retrieves and displays data in 2D or 3D cross tabs, charts, and graphs, with easy
pivoting of the axes.
Responds to queries quickly.
Integration of Data Mining and Data Warehousing:
Data warehouse provides clean, integrated data for fruitful mining.
Data mining provides powerful tools for analysis of data stored in data warehouses.
OLAP can be viewed as data summarization and simple data mining facility.
Data mining provides more analysis tools, e.g., association, classification, clustering, pattern-directed, and trend analysis.
Mining multi-level knowledge by integration with OLAP facilities: mining in multiple data cubes.
In data warehouses the data can be stored and operated by using data cube technology.
Data Cube:
.
Data Warehouse Operations:
Roll-up: Aggregates (summarizes) along a dimension
Drill-down: Increases detail of a dimension
Slice: Select a subset of the available dimensions
Dice: Group or partition on one or more dimensions
Pivot: Reorient a cube by swapping dimensions
Data Mining Functionality:
The following are different kinds of functionalities of data mining
Concept description: Characterization and Comparison:
Generalize, summarize, and possibly contrast data characteristics.
e.g., dry vs. wet regions.
Association:
From association, correlation, to causality.
inding rules like inside(x, city) --> near(x, highway).
Classification and Prediction:
Classify data based on the values in a classifying attribute, e.g., classify countries based on climate, or classify cars based on gas mileage.
Predict some unknown or missing attribute values based on other information
Clustering:
Group data to form new classes, e.g., cluster houses to find distribution patterns.
Trend and deviation analysis:
Find and characterize evolution trend, sequential patterns, similar Sequences, and deviation data, e.g., stock analysis.
Similarity-based pattern-directed analysis:
Find and characterize user-specified patterns in large databases.
Periodicity analysis:
Find segment-wise or total cycles or periodic behaviors in time-related data.
Data Mining Applications:
-> Numerous data mining applications.
Querying database knowledge
Multi-level data browsing
Performance prediction
Market analysis
Database design and query optimization
Intelligent query answering.
-> Intelligent Query Answering
Extended data model: Schemas, hierarchies, multi-layered databases, generalized relations/cubes, data mining tools.
Intelligent answering, Multi-level summaries & statistics, neighborhood info, roll-up & drill-down facilities.
Conclusion:
Data mining: A rich, promising, young field with broad applications and many challenging research issues.
Recent progress: Database-oriented, efficient data mining methods in relational and transaction DBs.
Tasks: Characterization, association, classification, clustering, sequence and pattern analysis, prediction, and many other tasks.
Domains: Data mining in extended-relational, transaction, object-oriented, spatial, temporal, document, multimedia, heterogeneous, and legacy databases, and WW.
Technology integration:
Database, data mining, & data warehousing technologies.
Other fields: machine learning, statistics, neural network, Information theory,
knowledge representation