»
Markov Processes for Stochastic Modeling
 
 

Markov Processes for Stochastic Modeling, 2nd Edition

 
Markov Processes for Stochastic Modeling, 2nd Edition,Oliver Ibe,ISBN9780124077959
 
 
 

  

Elsevier

9780124077959

9780124078390

514

229 X 152

This book brings into one volume the different applications of Markov processes.

Print Book + eBook

USD 119.94
USD 199.90

Buy both together and save 40%

Print Book

Hardcover

In Stock

Estimated Delivery Time
USD 99.95

eBook
eBook Overview

VST (VitalSource Bookshelf) format

DRM-free included formats : EPUB, Mobi (for Kindle), PDF

USD 99.95
Add to Cart
 
 

Key Features

  • Presents both the theory and applications of the different aspects of Markov processes
  • Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented
  • Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis.

Description

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems.

Covering a wide range of  areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader.

Readership

Graduate and upper-level undergraduate students, researchers and practitioners working in Markov Processes.

Oliver Ibe

Dr Ibe has been teaching at U Mass since 2003. He also has more than 20 years of experience in the corporate world, most recently as Chief Technology Officer at Sineria Networks and Director of Network Architecture for Spike Broadband Corp.

Affiliations and Expertise

University of Massachusetts, Lowell, USA

View additional works by Oliver Ibe

Markov Processes for Stochastic Modeling, 2nd Edition

Chapter 1: Basic Concepts

Review of Probability

Random Variables

Transform Methods

Bivariate Random Variables

Many Random Variables

Fubini’s Theorem

Sums of Independent Random Variables

Some Probability Distributions

Introduction to Stochastic Processes

Classification of Stochastic Processes

Characterizing a Stochastic Process

Stationary Stochastic Processes

Ergodic Stochastic Processes

Some Models of Stochastic Processes

Chapter 2: Introduction to Markov Processes

Introduction

Structure of Markov Processes

Strong Markov Property

Applications of Discrete-time Markov Processes

Applications of Continuous-time Markov Processes

Applications of Continuous-state Markov Processes

Chapter 3: Discrete-Time Markov Chains

Introduction

State Transition Probability Matrix

State Transition Diagrams

Classification of States

Limiting-State Probabilities

Sojourn Time

Transient Analysis of Discrete-Time Markov Chains

First Passage and Recurrence Times

Occupancy Times

Absorbing Markov Chains and the Fundamental Matrix

Reversible Markov Chains

Chapter 4: Continuous-Time Markov Chains

Introduction

Transient Analysis

Birth and Death Processes

First Passage Time

The Uniformization Method

Reversible Continuous-Time Markov Chains

 

Chapter 5: Markovian Queueing Systems

Introduction

Description of a Queueing System

The Kendall Notation

The Little’s Formula

The PASTA Property

The M/M/1 Queueing System

Examples of Other M/M Queueing Systems

M/G/1 Queue

G/M/1 Queue

Chapter 6: Markov Renewal Processes

Renewal Processes

The Renewal Equation

The Elementary Renewal Theorem

Random Incidence and Residual Time

Markov Renewal Process

Semi-Markov Processes

Markov Jump Processes

Chapter 7: Markovian Arrival Processes

Introduction

Overview of Matrix-Analytic Methods

Markovian Arrival Process

Batch Markovian Arrival Process

Markov-Modulated Poisson Process

Markov-Modulated Bernoulli Process

Sample Applications of MAP and Its Derivatives

Chapter 8: Random Walk

Introduction

The Two-Dimensional Random Walk

Random Walk as a Markov Chain

Symmetric Random Walk as a Martingale

Random Walk with Barriers

Gambler’s Ruin

First Return Times

First Passage Times

Maximum of a Random Walk

Correlated Random Walk

Continuous-time Random Walk

Sample Applications of Random Walk

Chapter 9: Brownian Motion and Diffusion Processes

Introduction

Brownian Motion

Introduction to Stochastic Calculus

Geometric Brownian Motion

Fractional Brownian Motion

Application of Brownian Motion to Option Pricing

Random Walk Approximation of Brownian Motion

The Ornstein-Uhlenbeck Process

Diffusion Processes

Examples of Diffusion Processes

Relationship Between the Diffusion Process and Random Walk

Chapter 10: Controlled Markov Processes

Introduction

Markov Decision Processes

Semi-Markov Decision Processes

Partially Observable Markov Decision Processes

Chapter 11: Hidden Markov Models

Introduction

HMM Basics

HMM Assumptions

Three Fundamental Problems

Solution Methods

Types of Hidden Markov Models

Hidden Markov Models with Silent States

Extensions of Hidden Markov Models

Other Extensions of HMM

Chapter 12: Markov Point Processes

Point Processes

Temporal Point Processes

Spatial Point Processes

Spatial-Temporal Point Processes

Operations on Point Processes

Marked Point Processes

Markov Point Processes

Markov Marked Point Processes

Applications of Markov Point Processes

Quotes and reviews

"Markov processes are the most popular modeling tools for stochastic systems in many different fields, and Ibe compiles in a single volume many of the Markovian models used indifferent disciplines. The information could be useful to graduate students and researchers in any field that uses Markov processes, he says, but he was thinking particularly of those in traffic engineering, image analysis, bioinformatics, biostatistics, financial engineering, and computational biology."--Reference & Research Book News, October 2013

 
 
Free Shipping
Shop with Confidence

Free Shipping around the world
▪ Broad range of products
▪ 30 days return policy
FAQ

Contact Us