Data Architecture, 1st Edition
Preface
Foreword
Section One: The Principles
1: Understanding Architectural Principles
1.1 Architecture Definition
1.2Design Problem Definition
1.3 Patterns and Pattern Usage
1.4 Concepts for Pattern Usage
1.5 Information Architecture
1.6 Structure works!
1.7 Architecture Problems
1.8 Architectural Solutions
1.9 The idea of ‘Form Follows Function’
1.10 Guideline: Composition and Environment
1.11 Guideline: Evolution
1.12 Guideline: Current and Future
1.13 Data Policies (Governance), the Foundation Building Codes
1.14 Data Policy Principles
1.15 References
2: Enterprise Architecture Frameworks and Methodologies
2.1 Architecture Frameworks
2.2 A brief history of Enterprise Architecture
2.3 The Zachman framework for Enterprise Architecture
2.4 The Open Group Architecture Framework (TOGAF)
2.5 The Federal Enterprise Architecture (FEA)
2.6 The Gartner Process
2.7 Enterprise Data Architectures
2.8 Enterprise Models
2.9 The Enterprise Data Model
2.10 Importance of the Enterprise Data Model
2.11 Object Concepts: Types and Structures within Databases
2.12 Inheritance
2.13 Object Life Cycles
2.14 Relationships and Collections
2.15 Object Frameworks:
2.16 Object Framework Programming
2.17 Pattern Based Frameworks
2.18 Architecture Patterns in Use
2.19 US Treasury Architecture Development Guidance (TADG)
2.20 TADG Pattern Content
2.21 TADG Architecture Patterns
2.22 IBM Patterns for e-Business
2.23 Enterprise Data Model Implementation Methods
3. Enterprise Level Data Architecture Practices
3.1 Enterprise Level Architectures
3.2 System Architectures
3.3 Enterprise Data Architectures
3.4 Enterprise Technology Architectures
3.5 Enterprise Architecture Terminology - Business terms
3.6 The Enterprise Model
3.7 The Enterprise Data Architecture from a Development Perspective
3.8 Subject Area Drivers
3.9 Naming and Object Standards
3.10 Data Sharing
3.11 Data Dictionary-Metadata Repository
3.12 Domain Constraints in Corporate and Non-Corporate Data
3.13 Organizational Control Components
3.14 Data Administration.
3.15 Database Administration.
3.16 Setting Up a Database Administration Group
3.17 Repository Management Area and Model Management.
3.18 References
4: Understanding Development Methodologies
4.1 Design Methods
4.2 Why do we need Development Methodologies?
4.3 The Beginnings
4.4 Structured methods
4.5 Structured Programming
4.6 Structured Design
4.7 Structured Analysis
4.8 Still Having Problems
4.9 Requirements Definitions
4.10 Problems with Structured Approaches
4.11 Personal Computers and the Age of Tools
4.12 Engineering Concepts Applied
4.13 Other Principles Utilized
4.14 The Birth of Information Engineering
4.15 The Four Tenets of Information Engineering
4.16 Information Engineering as a Design Methodology
4.17 The Synergy of Tools and Information Engineering.
4.18 Problems with Information Engineering
4.19 Implementing the best of IE while minimizing Expense
4.20 References:
Section Two: The Problem
5: Business Evolution
5.1 The Problem of Business Evolution
5.2 Expansion and Function Separation
5.3 Separate Function Communication.
5.4 Manual Data Redundancy
5.5 Management Organization and Data.
5.6 Data Planning and Process Planning
5.7 Corporate Architecture
5.8 Using Nolan’s Stages of Growth
5.9 Problems with Older Organizations
5.10 Business Today
5.11 When will it end
5.12 What can we do about it?
5.13 Generic Subject Areas for Corporate Architectures
5.14 Corporate Information Groupings or Functional Areas
5.15 Corporate Knowledge
5.16 References
6 Business Organizations
6.1 Purpose and Mission of the Organization.
6.2 Ideology, Mission and Purpose
6.3 Design with the Future of the Organization in Mind
6.4 Generalize for Future Potential Directions
6.5 Organizational Structure
6.6 What are the Basic Functions in an Organization?
6.7 The Information Needs of Management
6.8 Organizations Don’t Know What They Don’t Know
6.9 Information Strategy for Modern Business
6.10 Maximization of the Value of Information
6.11 Loci of forces in the Organization
6.12 References
7. Productivity inside the Data Organization
7.1 Information Technology
7.2 What Is Information Technology?
7.3 Trends in Information Technology
7.4 Vendor Software Development
7.5 The Other Option
7.6 Trends in Organizational Change
7.7 Productivity
7.8 Explanations for the Anomaly in Productivity
7.9 ‘Business has evolved and is using the benefits’ Theory
7.10 ‘Businesses are still evolving’ Theory
7.11 ‘It is just evolutionary change’ Theory
7.12 Information Technology and its impact on Organizations
7.13 Why Invest in Information Technology?
7.14 Ineffective Use of Information Technology
7.15 Other Impediments to Organizational Efficiency
7.16 Organizational Impediments to Information Technology
7.17 Technological Solutions for Information Technology
7.18 Human Resource issues in Information Technology
7.19 Quality of the Workforce
7.20 Summary of the Discussion of the Anomaly in Information Technology
7.21 Maximizing the Use of Information Technology
7.22 References:
8. Solutions That Cause Problems
8.1 Downsizing and Organizational Culture
8.2 Downsizing Defined
8.3 Culture Change
8.4 Organizational Level Analysis
8.5 Organizational/ Individual Level Analysis
8.6 Downswing’s Impact on Culture
8.7 A Different Approach to Culture Change and Downsizing
8.8 Conclusion for Downsizing
8.9 Outsourcing
8.10 Rapid Application Development
8.11 References
Section Three: The Process
9. Data Organization Practices
9.1 Fundamentals of all Data Organization Practices
9.2 Corporate Data Architecture
9.3 Corporate Data Policy
9.4 Architecture Team
9.5 Design Team
9.6 Develop the Project Structure
9.7 Scope Definition
9.8 Project Plan
9.9 Data Architecture and Strategic Requirements Planning
9.10 Data Gathering and Classification
9.11 Business Area Data Modeling
9.12 Current Data Inventory Analysis
9.13 Data and Function Integration
9.14 Procedure Definition via Functional Decomposition
9.15 Process Use Identification
9.16 New Function Creation
9.17 Utilization Analysis via Process Use Mapping
9.18 Access Path Mapping
9.19 Entity Cluster Development and Logical Residence Planning
9.20 Application Development Templates
9.21 Quality Assurance Metrics
9.22 Maintenance Control Process
9.23 The Software Development Methods
9.24 Architectural Development Methods
9.25 Atomic Process Models
9.26 Entity Process Models
9.27 The Unified Method
9.28 References:
10. Models and Model Repositories
10.1What are Models and How Did they come about?
10.2 Data Models Introduction
10.3 What does Modeling do for us?
10.4 Process Models Introduction.
10.5 Process Models, Why?
10.6 How are Automated Models developed?
10.7 How Are Models Retained?
10.8 Model Repository Policy and Approach
10.9 Shared Repository Objects
10.10 Model Driven Releases
10.11 Supporting an Application Release
10.12 Version Type
10.13 Participation
10.14 Seamless Development Control Process
10.15 Test Environments, Releases, and Databases
10.16 Release Stacking
10.17 Emergency Corrections
10.18 Emergency Correction Procedures
10.19 PTF Implementation for Shared Batch and On-line Objects
10.20 References:
11. Model Constructs and Model Types
11.1 Data Model Constructs
11.2 Application Audience and Services
11.3 Entities
11.4 Attributes
11.5 Relationships
11.6 Primary Identifiers
11.7 Entity Types
11.8 Entity Relationship Diagrams
11.9 Types of Relationships
11.10 Model Types
11.11 Physical-level design
11.12 Primary Keys
11.13 Normalization
11.14 Denormalization
11.15 Over-normalization
11.16 Domains
11.17 Domain Constraints
11.18 Reference Data
11.19 Generic Domain Constraint Constructs
11.20 References
12. Time as a Dimension of the Database
12.1 What is to be done with historical data?
12.2 Application History
12.3 Classes and Characteristics
12.4 Current Occurrence
12.5 Simple History
12.6 Bounded Simple History
12.7 Complex History
12.8 Logically Modeling History
12.9 Physical Design of History
12.10 Physical Implementation of History
12.11 Performance Tuning
12.12 Finding Patterns
12.13 Tips and Techniques for Implementing History
12.14 Types of Systems
12.15 Physical Structure
12.16 Dimensional History
12.17 References
12.18 Section Four: The Product
13. Concepts of Clustering, Indexing and Structures
13.1 Cluster Analysis
13.2 What is a Cluster?
13.3 Cluster Theory Applied
13.4 Inserts
13.5 Updates
13.6 Deletes
13.7 Physical Structure
13.8 Key History and Development
13.9 Primary Key
13.10 Foreign Keys
13.11 Foreign Key Propagation
13.12 Candidate Keys
13.13 Natural Keys
13.14 Engineered Keys
13.15 Surrogate Keys
13.16 High Water Key
13.17 One of a Kind Key
13.18 Other Specialized Keys
13.19 References
Section Four: The Product
14. Basic Requirements for Physical Design
14.1 Requirements for Physical Design
14.2 How Much Data?
14.3 History
14.4 Population Quantification of Application Data
14.5 Concurrency
14.6 Security / Audit
14.7 Audit
14.8 Archive/Purge
14.9 Recovery/Restart
14.10 Sort / Search Requirements
14.11 Reorganization and Restructuring
14.12 Data Integrity
14.13 Referential Integrity
14.14 Data Access
14.15 Privacy Requirements
14.16 Suggested Reading
15. Physical Database Considerations
15.1 Three Level Architecture
15.2 Data Independence
15.3 Data Base Languages
15.4 Classification of Data Base Management Systems
15.5 Factors Impacting Physical Database Design
15.6 Analysis of Queries, Reporting and Transactions
15.7 Queries, Reports and Transactions
15.8 Interpretation of the Functional Decomposition
15.9 Event Identification
15.10 Process Use Identification Reviewed
15.11 Utilization Analysis via Process Use Mapping
15.12 Analysis of expected frequency of Insert, Delete, Update
15.13 Other Physical Database Design Considerations
15.14 Population on the Database
15.15 References
15.16 Section Four: The Product
16. Interpretation of Models
16.1 Physical Design Philosophy
16.2 Objectives
16.3 The Entity Relationship Model
16.4 Interaction Analysis
16.5 The CRUD Matrix
16.6 Entity Life Cycle Analysis/ Entity State transition Diagrams
16.7 Process Dependency Scope and Process Dependency Diagram
16.8 Event Analysis
16.9 Process Logic Diagrams
16.10 Interaction Analysis Summary
16.11 Changes to ER Models
16.12 ERD Denormalization
16.13 Collapse of 1:1 relationships
16.14 Actions on Multiple Relationships
16.15 Resolution of Circular References
16.16 Resolution of Duplicate Propagated Keys
16.17 Access Level Denormalization
16.18 Consolidation of Entities
16.19 Implement Repeating Groups
16.20 Introduce Redundancy
16.21 Access Path Mapping
16.22 Conclusion
16.23 References
Section Five: Specialized Databases
17. Data Warehouses I
17.1 Early analysis in this area
17.2 Keen & Scott Morton
17.3 Decision discussion
17.4 Components of decisions
17.5 Responsibility
17.6 Report Writers and Query engines
17.7 Warehouses vs Reporting Databases
17.8 Higher level of abstraction
17.9 Based on perceived business use
17.10 Structure Evolution
17.11 Warehouse Components
17.12 Why can’t OLTP Data Stores be used?
17.13 DSS Requirements
17.14 Warehouse Characteristics
17.15 Warehouse Modeling
17.16 Warehouse Modeling is dependent on Architectures
17.17 Fundamental Benefits:
17.18 Enterprise Level Data Architecture
17.19 References:
18. Data Warehouses II
18.1 Reprise
18.2 Background
18.3 Many types and levels of Data
18.4 Data Modeling Definitions
18.5 Logical to Physical Transformation
18.6 Confusion over Terms such as Entity Relational model
18.7 Placement of Models
18.8 Denormalization and the Dimensional Model
18.9 Dimensional Model Evaluation
18.10 Data Evolution
18.11 What are the choices?
18.12 Applicability of the Dimensional and Relational and Hybrid Models
18.13 Dimensional Architecture
18.14 Where is the Relational Data Warehouse best suited?
18.15 Where is Dimensional Best Suited?
18.16 Hybrid ER-Dimensional
18.17 Problems Associated with the Hybrid Approach.
18.18 Target Enterprise Architecture
18.19 Building an Enterprise Data Model
18.20 Current Data Inventory
18.21 Standard or corporate business language
18.22 Conclusion of Hybrid Approach
18.23 References
18.24 Section Five: Specialty Databases
19. Dimensional Warehouses from Enterprise Models
19.1 Dimensional Databases from Enterprise Data Models
19.2 Warehouse Architecture
19.3 Dimensional Modeling
19.4 Dimensional Model Concepts
19.5 Review of Basic Components of Dimensional Models
19.6 Differences between Dimension and Fact Tables
19.7 Star Schema
19.8 Star Schema Design Approach
19.9 Enterprise Data Warehouse Design
19.10 Structure Design
19.11 Categorize the Entities
19.12 Identify Dependency Chains
19.13 Produce Dimensional Models
19.14 Options for Dimensional Design
19.15 The Flat Table Schema
19.16 The Stepped Table Schema
19.17 Star Schema
19.18 Snowflake Schema
19.19 Star Schema Clusters
19.20 Iterate to Refine the Design
19.21 Review of Design options
20. The Enterprise Data Warehouse
20.1 Enterprise Data Warehouses
20.2 Why would you want an Enterprise Data Warehouse?
20.3 Enterprise Data Warehouse Defined
20.4 What are the important EDW Drivers?
20.5 The Best Practices for EDW implementation
20.6 Enterprise Data Architecture Implementation Methods
20.7 Top Down approach
20.8 Bottom Up
20.9 These are your Choices
20.10 Preliminary Conclusion
20.11 Hybrid Approach
20.12 Implementation summary
20.13 References:
21. Object and Object/Relational Databases
21.1 Object Oriented Data Architecture
21.2 Illustration of OOD (Object Oriented Design) Concepts - Wiring Money
21.3 Examples of Different Actions
21.4 Elements of OOD - Overriding
21.5 Analogy and Problem Solving
21.6 Coping with Complexity
21.7 Interconnections - the Perpetrator of Complexity
21.8 Assembler Languages
21.9 Procedures and Functions
21.10 Modules
21.11 Parameter Passing
21.12Abstract Data Types
21.13 Objects -like Abstract Data Type’s with Parameter Passing
21.14 Object Oriented Architectures Summary
21.15 EER Concepts that support Object and Relational Models
21.16 Subclasses and Superclasses
21.17 Attribute Inheritance
21.18 Specialization
21.19 Generalization Hierarchies
21.20 Multiple Inheritance
21.21Messaging
21.22 Object Identity
21.23 Type ‘Generators’ and Type Constructors
21.24 Summary
21.25 References:
22. Distributed Databases
22.1 Some Distributed concepts
22.2 The Distributed Model(s)
22.3 How Does It Work?
22.4 Distributed Data Design Concepts
22.5 Fragmentation
22.6 Replication
22.7 Homogeneous Distributed Model
22.8 Federated or Heterogeneous Distributed Model
22.9 Reliability and Availability
22.10 Controlled Data Sharing
22.11 Performance
22.12 Qualities required in a DDBMS
22.13 Other Factors
22.14 An Overview of Client Server
22.15 Functionality within Client Server
22.16 A Typical DDBMS
22.17 Distribution Transparency
22.18 Types of DDBMSs
22.19 Problems in DDBMSs
22.20 Individual Site Failure’s Effect on Data Integrity
22.21 Individual Site Failure‘s Effect on Traffic Flow
22.22 Communication Failure
22.23 Distributed Commitment
22.24 Distributed Deadlocks
22.25 Summary
22.26 References
