1 Introduction

13

1.1 Future Skill: Programming

13

1.2 Python

14

1.3 Artificial Intelligence

15

1.4 Machine Learning

16

1.5 Programming with the Help of AI

17

1.6 Prompt Engineering

19

2 Getting Started with Python

21

2.1 Installing Python on Windows

21

2.2 Installing Python on macOS and Linux

22

2.3 Using Python in Interactive Mode

22

2.4 Python Scripts

23

2.5 Visual Studio Code and Integrated Development Environments

23

2.6 Jupyter Notebooks

25

2.6.1 Creating Jupyter Notebooks in Visual Studio Code

25

2.6.2 Markdown Cells

26

2.6.3 Code Cells

26

3 Basics of the Python Language

29

3.1 Variables and Data Types

30

3.2 Comments

31

3.3 Functions

31

3.4 Conditionals with "if," "elif," and "else"

32

3.5 Comparison Operators

33

3.6 Numbers

34

3.7 The "while" Loop

35

3.8 The “for” Loop

36

3.9 More on "print"

37

3.10 Lists, Sets, Tuples, and Dictionaries

38

3.11 "for" Loop for Lists and More

39

3.12 Sample Program: Counting Words

41

3.13 Writing Your Own Functions

44

3.14 Pythonic Code

47

3.14.1 Create a New List from a List

48

3.14.2 Filtering Elements from a List

49

3.14.3 Concatenating Strings Without "+"

49

3.14.4 Searching for a Value in a List

49

3.14.5 Tuple Decomposition

50

3.14.6 Loop Counters

50

3.14.7 Iterating over the Key-Value Pairs in a Dictionary

50

3.14.8 Swapping Variable Values

51

3.14.9 Assigning a Boolean Directly

51

3.14.10 Case Differentiation

51

3.14.11 Retrieving a Default Value from a Dictionary

51

3.14.12 Walrus Operator

52

3.15 Importing Modules and Installing Packages with "pip"

52

3.16 Virtual Environments

54

4 Working with Files

57

4.1 Reading and Writing Text Files

58

4.2 Comma-Separated Values Files

60

4.3 Managing Files

64

4.3.1 Creating a File

64

4.3.2 Checking for File Existence and Deleting Files

65

4.3.3 Moving, Renaming, and Copying Files

65

4.3.4 Listing Files in a Directory

65

4.4 Example: Text Analysis

66

4.5 Excel Files

68

4.6 Image Files

70

4.7 JSON Files

73

4.8 XML Files

77

4.9 Configuration Files

78

5 Data Analysis

81

5.1 NumPy

81

5.2 Pandas

83

5.3 Loading Data from Files into Pandas DataFrames

87

5.4 Data Cleaning with Pandas

90

5.4.1 Filtering Data

90

5.4.2 Identifying and Handling Missing Values

91

5.4.3 Removing Duplicates

92

5.4.4 Identifying and Correcting Errors and Outliers

94

5.4.5 Transforming Data into the Desired Format

96

5.4.6 (De)coding Categories

97

5.4.7 Normalizing: Making Values Comparable

98

5.5 Calculations and Analyses with Pandas

99

5.5.1 Row-by-Row Calculations

99

5.5.2 Differences from the Previous Value

100

5.5.3 Aggregating Values

101

5.5.4 Cumulative Sums

101

5.5.5 Grouping Data

102

5.5.6 Sorting and Ranking Data

103

5.6 Merging Data from Multiple Sources

104

6 Visualizations with Matplotlib

113

6.1 Creating Plots

113

6.2 Design Options

115

6.3 Subplots: Multiple Plots in a Single Figure

117

6.4 Line Charts, Bar Charts, and More

118

6.5 Creating Charts from DataFrames

121

6.6 Interactive Charts

124

6.7 Zooming and Scrolling

125

7 Machine Learning and Artificial Intelligence

131

7.1 Predicting Numbers Using Linear Regression

133

7.2 Linear Regression with Multiple Factors

135

7.3 Classification Using Logistic Regression

139

7.4 Decision Trees and Random Forests

141

7.5 K-Nearest Neighbors

144

7.6 Support Vector Machines

148

7.7 Training Data, Test Data, and Model Evaluation

151

7.8 Clustering (Unsupervised Learning)

158

8 AI in Action: Text and Image Analysis

163

8.1 AI for Text and Language

163

8.2 Text Analysis and Word Clouds

164

8.3 Text Preprocessing

166

8.4 Sentiment Analysis

170

8.5 Recognizing Entities in Text: Named Entity Recognition

175

8.6 Transfer Learning

178

8.7 AI for Images

181

8.8 Image Preprocessing: Grayscale Conversion and More

182

8.9 Detecting Edges and Contours in Images

185

8.10 Classic Machine Learning Methods for Images

189

8.11 Image Classification with Deep Learning

195

9 Using APIs

199

9.1 API Requests with "requests"

200

9.2 API Access with Special SDKs

205

9.3 Visualizing and Analyzing API Data

206

9.4 ChatGPT API

212

10 Using Python on the Web

217

10.1 HTML

218

10.2 Flask: A Python Web Server

220

10.3 Interactive Web Tools with Streamlit

222

10.4 Reading Web Content with Beautiful Soup

225

10.5 Remotely Controlling the Browser with Selenium

228

10.6 Sending Emails and Messenger Messages

231

11 Databases

235

11.1 The SQL Language and the SQLite Console

236

11.2 Creating Tables with CREATE TABLE

238

11.3 Querying, Inserting, Updating, and Deleting with SELECT, INSERT, UPDATE, and DELETE

240

11.4 Accessing an SQLite Database with Python

242

11.5 Pandas DataFrame: Reading and Writing Data from Databases

244

12 Automating Routine Tasks

247

12.1 Retrieving Data via the API and Storing It in a Database

247

12.2 Creating Charts from the Database and Save as Image Files

250

12.3 Sending an Email When There’s Something New

252

12.4 Sending Screenshots of Web Pages via Messenger

255

12.5 Images: Reducing File Sizes, Finding Locations, and Cleaning Up

258

12.6 PDF: Splitting, Merging, and Consolidating Files

261

12.7 Recognizing Text on Business Card Images Using OCR and Saving to Your Phone’s Address Book

263

12.8 Creating Invoices, Mail Merges, and Other Documents

265

12.9 Running Python Scripts on a Schedule

268

12.10 Logging in Automations

271