Inhaltsverzeichnis

Alle Kapitel aufklappen
Alle Kapitel zuklappen
Preface
15
1 Introduction
17
1.1 Why Neural Networks?
17
1.2 About This Book
18
1.3 The Contents in Brief
19
1.4 Is This Bee a Queen Bee?
22
1.5 An Artificial Neural Network for the Bee Colony
23
1.6 From Biology to the Artificial Neuron
28
1.6.1 The Biological Neuron and Its Technical Copy
28
1.6.2 The Artificial Neuron and Its Elements
29
1.7 Classification and the Rest
32
1.7.1 Big Picture
32
1.7.2 Artificial Intelligence
32
1.7.3 History
34
1.7.4 Machine Learning
35
1.7.5 Deep Neural Networks
37
1.7.6 Transformer Neural Networks
37
1.8 Summary
39
1.9 Further Reading
39
PART I Up and Running
41
2 Starter Kit for Developing Neural Networks with Python
43
2.1 The Technical Development Environment
43
2.1.1 The Anaconda Distribution
43
2.1.2 Our Cockpit: Jupyter Notebook
47
2.1.3 Major Python Modules
57
2.1.4 The Google Colab Platform for Jupyter Notebooks
59
2.1.5 Additional Jupyter Notebook Cloud Resources
62
2.2 Summary
63
3 A Simple Neural Network
65
3.1 Background
65
3.2 Bring on the Neural Network!
65
3.3 Neuron Zoom-In
68
3.4 Step Function
73
3.5 Perceptron
75
3.6 Points in Space: Vector Representation
76
3.6.1 Task: Completing Values
77
3.6.2 Task: Outputting the Iris Dataset as a Scatterplot
79
3.7 Horizontal and Vertical: Column and Line Notation
82
3.7.1 Task: Determining the Scalar Product Using NumPy
83
3.8 The Weighted Sum
84
3.9 Step-by-Step: Step Functions
85
3.10 The Weighted Sum Reloaded
85
3.11 All Together
86
3.12 Task: Robot Protection
89
3.13 Summary
91
3.14 Further Reading
91
4 Learning in a Simple Network
93
4.1 Background: Plans Are Being Made
93
4.2 Learning in Python Code
94
4.3 Perceptron Learning
94
4.4 Separating Line for a Learning Step
98
4.5 Perceptron Learning Algorithm
99
4.6 The Separating Lines or Hyperplanes for the Example
103
4.7 scikit-learn Compatible Estimator
106
4.8 scikit-learn Perceptron Estimator
113
4.9 Adaline
115
4.10 Summary
125
4.11 Further Reading
126
5 Multilayer Neural Networks
127
5.1 A Real Problem
127
5.2 Solving XOR
129
5.3 Preparations for the Launch
134
5.4 The Plan for Implementation
135
5.5 The Setup ("class")
136
5.6 The Initialization ("__init__")
138
5.7 Something for In-Between ("print")
141
5.8 The Analysis ("predict")
141
5.9 The Usage
143
5.10 Summary
145
6 Learning in a Multilayer Network
147
6.1 How Do You Measure an Error?
147
6.2 Gradient Descent: An Example
149
6.2.1 Gradient Descent: The Concept
149
6.2.2 Algorithm for the Gradient Descent
150
6.3 A Network of Sigmoid Neurons
157
6.4 The Cool Algorithm with Forward Delta and Backpropagation
158
6.4.1 The __init__ Method
158
6.4.2 The “predict” Method
161
6.4.3 The “fit” Method
165
6.4.4 The “plot” Method
167
6.4.5 The Complete Picture
168
6.5 A “fit” Run
170
6.5.1 Initialization
172
6.5.2 Forward
173
6.5.3 Output
174
6.5.4 Hidden
175
6.5.5 Delta W_kj
176
6.5.6 Delta W_ji
177
6.5.7 W_ji
177
6.5.8 W_kj
178
6.6 Summary
178
6.7 Further Reading
178
7 Examples of Deep Neural Networks
179
7.1 Convolutional Neural Networks
179
7.1.1 The Architecture of Convolutional Networks
181
7.1.2 The Coding Block
182
7.1.3 The Prediction Block
188
7.1.4 Training Convolutional Neural Networks
190
7.2 Transformer Neural Networks
194
7.2.1 The Network Structure
195
7.2.2 Embeddings
197
7.2.3 Positional Encoding
197
7.2.4 Encoder
200
7.2.5 Decoder
202
7.2.6 Training Transformer Neural Networks
203
7.3 The Optimization Method
204
7.3.1 Momentum Optimization
204
7.3.2 ADAM Optimization
205
7.4 Preventing Overfitting
205
7.4.1 Early Stopping
205
7.4.2 Dropout
206
7.5 Summary
207
7.6 Further Reading
207
8 Programming Deep Neural Networks Using TensorFlow 2
209
8.1 Convolutional Networks for Handwriting Recognition
209
8.1.1 The MNIST Dataset
209
8.1.2 A Simple Convolutional Neural Network
213
8.1.3 The Results
217
8.2 Transfer Learning with Convolutional Neural Networks
223
8.2.1 The Pretrained Network
224
8.2.2 Data Preparation
226
8.2.3 The Pretrained Network
227
8.2.4 The Results
229
8.3 Transfer Learning with Transformer Neural Networks
231
8.3.1 The Transformer Library
232
8.3.2 Tokenizers and Models
234
8.3.3 The Model Hub from Hugging Face
235
8.4 Summary
236
8.5 Further Reading
236
PART II Deep Dive
239
9 From Brain to Network
241
9.1 Your Brain in Action
241
9.2 The Nervous System
242
9.3 The Brain
243
9.3.1 The Parts
243
9.3.2 A Section
244
9.4 Neurons and Glial Cells
245
9.5 A Transfer in Detail
247
9.6 Representation of Cells and Networks
249
9.7 Summary
251
9.8 Further Reading
251
10 The Evolution of Artificial Neural Networks
253
10.1 The 1940s
254
10.1.1 1943 McCulloch-Pitts Neurons
254
10.1.2 1949: Donald Hebb
255
10.2 The 1950s
255
10.2.1 1951: Marvin Minsky and Dean Edmonds – SNARC
255
10.2.2 1955/1956: Artificial Intelligence
256
10.2.3 1958: Rosenblatt’s Perceptron
256
10.2.4 1960: Bernard Widrow and Marcian Hoff – Adaline and Madaline
256
10.3 The 1960s
257
10.3.1 1969: Marvin Minsky and Seymour Papert
257
10.4 The 1970s
257
10.4.1 1972: Kohonen – Associative Memory
258
10.4.2 1973: Lighthill Report
258
10.4.3 1974: Backpropagation
258
10.5 The 1980s
258
10.5.1 1980: Fukushima’s Neocognitron
258
10.5.2 1982: John Hopfield
260
10.5.3 1982: Kohonen’s SOM
269
10.5.4 1986: Backpropagation
269
10.5.5 1987: NN Conference
270
10.5.6 1989: Yann LeCun: Convolutional Neural Networks
270
10.6 The 1990s
270
10.6.1 1997: Sepp Hochreiter and Jürgen Schmidhuber – Long Short-Term Memory
271
10.7 The 2000s
271
10.7.1 2006: Geoffrey Hinton et al.
271
10.8 The 2010s
272
10.8.1 2014: Ian J. Goodfellow et al. – Generative Adversarial Networks
272
10.8.2 2017: Ashish Vaswani et al. – Attention Is All You Need
274
10.9 Summary
274
10.10 Further Reading
274
11 The Machine Learning Process
277
11.1 The CRISP-DM Model
277
11.1.1 Business Understanding
278
11.1.2 Data Understanding
279
11.1.3 Data Preparation
279
11.1.4 Modeling
280
11.1.5 Evaluation
280
11.1.6 Deployment
280
11.2 Ethical and Legal Aspects
281
11.2.1 Algorithmic Fairness and Bias
282
11.2.2 Explainability and Interpretability
284
11.2.3 Ecological Aspects
288
11.2.4 Legal Aspects
289
11.3 Feature Engineering
290
11.3.1 Feature Coding
292
11.3.2 Feature Extraction
302
11.3.3 The Curse of Dimensionality
311
11.3.4 Feature Transformation
312
11.3.5 Feature Selection
316
11.4 Summary
317
11.5 Further Reading
318
12 Learning Methods
319
12.1 Learning Strategies
319
12.1.1 Supervised Learning
320
12.1.2 Unsupervised Learning
324
12.1.3 Reinforcement Learning
335
12.1.4 Semi-Supervised Learning
344
12.2 Tools
345
12.2.1 Confusion Matrix
345
12.2.2 Receiver Operating Characteristic Curves
347
12.3 Summary
350
12.4 Further Reading
350
13 Areas of Application and Real-Life Examples
351
13.1 Warm-Up
351
13.2 Image Classification
354
13.2.1 Definitions
354
13.2.2 On Bees and Bumblebees
356
13.2.3 Pretrained Networks
366
13.3 Dreamed Images
373
13.3.1 The Algorithm
373
13.3.2 Implementation
376
13.4 Deployment with Pretrained Networks
382
13.4.1 A Web Application for a Neural Network to Generate Image Descriptions
383
13.4.2 A Web Application for Image Generation
384
13.5 Summary
386
13.6 Further Reading
386
Appendices
387
A Python in Brief
389
A.1 The First Climb: Data Types, Variables, and Values
390
A.2 At the Top of the Hill: if-then-else and Loops
396
A.3 Downhill: Functions, Classes, Modules, and Some Greek
400
A.4 Right in the Middle: Special Data Types for Vectors, Matrixes, and Tensors
407
A.5 At the Finish: A Complete k-nearest neighbor Classifier
411
A.6 After the Ride Is Before the Ride
415
A.7 Further Reading
415
B Mathematics in Brief
417
B.1 Linear Algebra
417
B.2 Calculus
427
B.3 Derivative of the Sigmoid Function
430
B.4 Key Statements
431
B.5 Notation
431
B.6 Summary
433
B.7 Further Reading
433
C TensorFlow 2 and Keras
435
C.1 Introduction to TensorFlow 2
435
C.2 Features of TensorFlow 2
437
C.3 Integrated Keras Library
441
C.4 Summary
444
C.5 Further Reading
444
The Authors
445
Index
447