Advanced Data Structures: An Introduction to Data Structures and Algorithms

Abstract

This book is a collection of lectures notes on Data Structures and Algorithms. The content found in this book supplements the free video lecture series, of the same name, "Advanced Data Structures", by the author, Dr. Daniel Page. This video lecture series is available at http://www.pagewizardgames.com/datastructures. This book:
-Contains Computer Science topics and materials comparable to those found among university courses at a similar level (second-year) at top Canadian universities.
-Provides an accessible written companion and supplemental notes for those that wish to learn the subject of Data Structures and Algorithms from the video lecture series, but have difficulties taking notes, or would prefer having a written alternative to follow along.

This book is ideal for those with already an introductory programming background, know a little bit about computing, and wish to learn more about Data Structures and Algorithms and begin a more formal study of Computer Science. The materials here are a great place to start for supplemental/additional learning materials on the subject for self-study, university students, or those that want to learn more about Computer Science.

Dr. Daniel Page places great emphasis on the introductory mathematical aspects of Computer Science, a natural transition from a basic programming background to thinking a bit more like a computer scientist about Computer Science.

This book is not a textbook. The author assumes the reader is familiar with algebra, functions, common finite and infinite series such as arithmetic series and geometric series, and basic control structures in programming or logic. All the algorithms in this book are described in English, or using Java-like pseudocode.

Chapters
Chapter 1 - Introduction: Data Structures, Problems, Input Size, Algorithms, The Search Problem.
Chapter 2 - Intro to Analysis of Algorithms I: Complexity Analysis, Comparing Algorithms, Growth Rate of Functions (Asymptotics), Showing f is O(g), Showing f is not O(g).
Chapter 3 - Intro to Analysis of Algorithms II: Some Properties of O, An Iterative Example, Back to our "Easy" Search Problem.
Chapter 4 - Dictionaries: The Dictionary Problem, Simple Implementations of a Dictionary.
Chapter 5 - Hashing: Hash Function, Hash Code, Separate Chaining, Open Addressing, Revisiting the Load Factor.
Chapter 6 - Trees: Tree ADT, Linked Tree Representation, Tree Property, Computing Height of a Tree, Tree Traversals
Chapter 7 - Priority Queues & Heaps: Priority Queues, Heaps, Array-Based Implementation, Building a Heap, Application: Sorting, Introduction to Amortized Analysis
Chapter 8 - Binary Search Trees: Ordered Dictionary ADT, BST Implementations, Inorder Traversal, Smallest, Get, Put, Remove, Successor.
Chapter 9 - AVL Trees: Height, AVL Trees, Re-Balancing AVL Trees, putAVL, removeAVL, AVL Tree Performance.
Chapter 10 - Graphs: Degrees and the Handshaking Lemma, Complete Graphs, Paths and Cycles, Trees, Forests, Subgraphs, and Connectivity, Graph Representations.
Chapter 11 - Graph Traversals: Depth-First Search (DFS), Path-Finding, Cycle Detection, Counting Vertices, DFS Tree, Breadth-First Search (BFS), Summary.
Chapter 12 - Minimum Spanning Trees: Weighted Graphs, Minimum Spanning Trees & Algorithms, Prim's Algorithm, Heap-Based Implementation of Prim's Algorithm and More!
Chapter 13 - Shortest Paths: Single-Source Shortest Path Problem, Dijkstra's Algorithm.
Chapter 14 - Multiway Search Trees: Beyond Binary Search Trees, Get, Put, Successor and Remove, (2,4)-Trees, B-Trees.

Full text

Advanced Data Structures
An Introduction to Data Structures and Algorithms
Daniel R. Page
PageWizard Games, Learning & Entertainment
Sunnyside, Manitoba, Canada

1
Copyright Information
This book is notes adapted from the lecturer’s notes of Dr. Daniel Page on the
subject, over various offerings of courses of this nature over various years. In prepara-
tion of these materials, no rights were intended on being infringed and was created for
educational purposes. This is not a textbook, nor intended to be used as a textbook,
it lacks a reference/bibliography section and hence is not meant as a comprehensive
scholarly source on this subject; these are supplemental to Dr. Daniel Page’s lec-
tures discussed on the next page. The name YouTube, as referred to on Page 2, is
not owned by the author, and is owned by Google Inc. Any images not created by
Dr. Daniel Page are in the public domain and were retrieved via http://freesvg.org,
thus are not credited directly here. This document was created as a written supple-
ment/companion to the lecture notes provided in lecture for a data structures and
algorithms course by Dr. Daniel Page, and a companion to the video-lecture series
discussed on the next page.
No part of this publication is to be sold, reproduced, scanned, freely distributed,
or re-sold, without explicit permission or prior agreement of/with Dr. Daniel Page
(PageWizard Games, Learning & Entertainment); with the exception of actions un-
der the fair dealings provision of the Canadian Copyright Act and any other applica-
ble exceptions, doing any of the mentioned actions without permission violates the
intellectual property rights of the author (Dr. Daniel Page) and the Canadian Copy-
right Act. The original written materials found in this publication are the property
of the author (Dr. Daniel Page), unless stated otherwise.
All business enquiries should be directed by e-mail to Dr. Daniel Page at
gle@pagewizardgames.com.
If cited or referenced, please cite this document (given the appropriate format) as:
Page, D.R. Advanced Data Structures: An Introduction to Data Structures and Al-
gorithms. PageWizard Games, Learning & Entertainment, 2020.
Published electronically/online and in print as of November 2020. Version history
will be disclosed below, where an errata will be available on the website for the book:
http://pagewizardgames.com/advanceddatastructures
Daniel R. Page, PageWizard Games, Learning & Entertainment
Sunnyside, Manitoba, Canada.
Book, 160 pages, ISBN 978-1-7774075-1-3 ISBN 978-1-7774075-0-6 (eBook)
First Edition (eBook, paperback), November 2020.
Copyright 2020 by Daniel R. Page, PageWizard Games, Learning & Entertainment.
All rights reserved.

2
How To Use This Book
The target audience of this book is a beginning second-year Computer Science
university student watching or listening to Dr. Daniel Page’s lectures. It is expected
that the reader is familiar with algebra, functions, common finite and infinite series
such as arithmetic series and geometric series, and basic control structures in pro-
gramming or logic. In addition, the author assumes students are knowledgeable of
some sorting algorithms and other fundamental programming concepts found often
in first-year Computer Science university courses; however, many assumptions are
avoided as the book lays out many common fundamental concepts in data structures
and algorithms. Algorithms are typically presented in pseudocode or in English in
this book, to emphasize the mathematical nature of algorithms and to ease students
away from viewing algorithms as a concept tethered to programming. Java-like pseu-
docode is often used for algorithms, and some Java code is provided in this book.
These notes are to supplement the lectures provided as part of Dr. Daniel Page’s
online lecture series called Advanced Data Structures. This is a compilation of lecture
notes, and not a textbook. At the time of publication these videos are available on
YouTube, and can be found at
http://www.pagewizardgames.com/videos
the playlist for this video lecture series is located at:
http://www.pagewizardgames.com/datastructures
This lecture series is freely available to all. The primary purpose of this book of
notes is to ensure students that have difficulties writing notes on their own or require
reference for the lectures have accessible notes. This publication is a mostly self-
contained reference for the lecture series, with minimal exceptions. Historically these
notes were compiled in lieu of preparing a course (, hence the main title of this book),
of the same name to be taught by the author, at St. Francis Xavier University.
To best learn the material, it is highly recommended taking written notes while
watching the lectures. A supplemental text that was in mind when these notes were
prepared (as part of course delivery) is:
Goodrich, Tamassia, Goldwasser. Data Structures & Algorithms in Java, 6e, Wiley.
Whenever referred to in these notes, the above book will be referred to as
“text [GTG]” or by the authors via “Goodrich et al.”. It is recommended should
you seek further readings.
Furthermore, each chapter does not reflect a week of material in a course. Mate-
rials covered in the lecture series are not segmented by week, and instead are covered
until completion of each topic. Happy learning, have a beautiful day!

3
About The Author
Dr. Daniel Page is a theoretical computer scientist and science educator, born
in Winnipeg, Manitoba, Canada. At the time of writing this, Dr. Page was most
recently in 2020 an Assistant Professor in the Department of Computer Science at
St. Francis Xavier University. With over a decade-plus experience working with
students and people of all ages in various educational settings, he has also worked as
an instructor and/or researcher for universities such as The University of Manitoba
and Western University, and worked for educational institutions such as the Royal
Aviation Museum of Western Canada and Mad Science of Manitoba. He received
his PhD in Computer Science at Western University in 2019, under the supervision
of Dr. Roberto Solis-Oba.

4
To my students.

Contents
1 Introduction 8
DataStructures................................. 8
Problems .................................... 12
InputSize.................................... 13
Algorithms ................................... 14
TheSearchProblem .............................. 15
2 Intro to Analysis of Algorithms I 17
ComplexityAnalysis .............................. 17
ComparingAlgorithms............................. 22
Growth Rate of Functions (Asymptotics) . . . . . . . . . . . . . . . . . . . 23
Showing fis O(g) ............................... 24
Showing fis not O(g) ............................. 27
3 Intro to Analysis of Algorithms II 28
Some Properties of O.............................. 28
AnIterativeExample.............................. 29
Back to our “Easy” Search Problem . . . . . . . . . . . . . . . . . . . . . . 30
4 Dictionaries 35
TheDictionaryProblem ............................ 35
Simple Implementations of a Dictionary . . . . . . . . . . . . . . . . . . . 38
5 Hashing 44
HashFunction.................................. 46
HashCode.................................... 46
SeparateChaining ............................... 47
OpenAddressing ................................ 49
Revisiting the Load Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5

6CONTENTS
6 Trees 58
TreeADT.................................... 60
Linked Tree Representation . . . . . . . . . . . . . . . . . . . . . . . . . . 61
TreeProperty.................................. 61
Computing Height of a Tree . . . . . . . . . . . . . . . . . . . . . . . . . . 62
TreeTraversals ................................. 63
7 Priority Queues & Heaps 65
PriorityQueues................................. 65
Heaps ...................................... 66
Array-Based Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 68
BuildingaHeap................................. 73
Application:Sorting .............................. 75
Introduction to Amortized Analysis . . . . . . . . . . . . . . . . . . . . . . 77
8 Binary Search Trees 81
OrderedDictionaryADT............................ 81
BSTImplementations ............................. 84
InorderTraversal ................................ 85
Smallest..................................... 86
Get........................................ 87
Put........................................ 88
Remove ..................................... 88
Successor .................................... 90
9 AVL Trees 91
Height ...................................... 92
AVLTrees.................................... 92
Re-BalancingAVLTrees............................ 94
putAVL ..................................... 96
removeAVL ................................... 98
AVLTreePerformance............................. 99
10 Graphs 100
Degrees and the Handshaking Lemma . . . . . . . . . . . . . . . . . . . . . 104
CompleteGraphs................................ 105
PathsandCycles ................................ 106
Trees, Forests, Subgraphs, and Connectivity . . . . . . . . . . . . . . . . . 107
GraphRepresentations............................. 109

CONTENTS 7
11 Graph Traversals 113
Depth-First Search (DFS) . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Path-Finding .................................. 116
CycleDetection................................. 117
CountingVertices................................ 117
DFSTree .................................... 118
Breadth-First Search (BFS) . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Summary .................................... 122
12 Minimum Spanning Trees 123
WeightedGraphs ................................ 123
Minimum Spanning Trees & Algorithms . . . . . . . . . . . . . . . . . . . 125
Prim’sAlgorithm ................................ 128
Heap-Based Implementation of Prim’s Algorithm and More! . . . . . . . . 132
13 Shortest Paths 135
Single-Source Shortest Path Problem . . . . . . . . . . . . . . . . . . . . . 135
Dijkstra’sAlgorithm .............................. 136
14 Multiway Search Trees 139
Beyond Binary Search Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Get........................................ 142
Put........................................ 145
SuccessorandRemove ............................. 146
(2,4)-Trees.................................... 148
B-Trees ..................................... 154