1 System Considerations

23

1.1 Building to Fit Your Organization

23

1.1.1 Data Services Architecture Scenarios

24

1.1.2 Determining Which Environments Your Organization Requires

29

1.1.3 Multi-Developer Environment

31

1.1.4 IT and Company Policies

33

1.2 Operating System Considerations

36

1.2.1 Source Operating System Considerations

37

1.2.2 Monitoring System Resources

37

1.2.3 CPU

39

1.2.4 Memory Considerations

40

1.2.5 Target Operating System Considerations

41

1.3 File System Settings

41

1.3.1 Locales

41

1.3.2 Commands

42

1.4 Network

45

1.5 Sizing Appropriately

45

1.5.1 Services That Make Up Data Services

46

1.5.2 Estimating Usage and Growth

52

1.6 Summary

55

2 Installation

57

2.1 SAP BusinessObjects Business Intelligence Platform and Information Platform Services

57

2.1.1 Security and Administration Foundation

57

2.1.2 Reporting Environment

62

2.2 Repositories

64

2.2.1 Planning for Repositories

65

2.2.2 Preparing for Repository Creation

67

2.2.3 Creating Repositories

67

2.3 Postal Directories

74

2.3.1 USA Postal Directories

74

2.3.2 Global Postal Directories

81

2.4 Installing SAP Server Functions

82

2.5 Configuration for Excel Sources in Linux

84

2.5.1 Enabling Adapter Management in a Linux Job Server

84

2.5.2 Configuring an Adapter for Excel on a Linux Job Server

86

2.6 SAP Information Steward

89

2.7 Summary

90

3 Configuration and Administration

91

3.1 Server Tools

91

3.1.1 Central Management Console

91

3.1.2 Data Services Management Console

94

3.1.3 Data Services Server Manager for Linux/UNIX

113

3.1.4 Data Services Server Manager for Windows

125

3.1.5 Data Services Repository Manager

132

3.2 Set Up Landscape Components

132

3.2.1 Datastores

133

3.2.2 Table Owner Aliases

136

3.2.3 Substitute Parameters

138

3.2.4 System Configuration

139

3.3 Security and Securing Sensitive Data

140

3.3.1 Encryption

141

3.3.2 Secure Socket Layer (SSL)

142

3.3.3 Enable SSL Communication via CMS

143

3.3.4 Data Services SSL Configuration

147

3.4 Path to Production

150

3.4.1 Repository-Based Promotion

150

3.4.2 File-Based Promotion

152

3.4.3 Central Repository-Based Promotion

153

3.4.4 Object Promotion

155

3.4.5 Object Promotion with CTS+

157

3.5 Operation Readiness

157

3.5.1 Scheduling

157

3.5.2 Support

164

3.6 How to Troubleshoot Execution Exceptions

166

3.6.1 Viewing Job Execution Logs

166

3.6.2 Common Causes of Job Execution Failures

168

3.7 Summary

172

4 Application Navigation

175

4.1 Introduction to Data Services Object Types

175

4.2 Hypothetical Work Request

177

4.3 Data Services Designer

180

4.3.1 Datastore

181

4.3.2 File Format

183

4.3.3 Data Flow

184

4.4 Data Services Workbench

190

4.4.1 Project

192

4.4.2 Datastore

193

4.4.3 File Format

196

4.4.4 Data Flow

198

4.4.5 Validation

202

4.4.6 Data Flow Execution

202

4.5 Summary

205

5 Objects

207

5.1 Jobs

208

5.1.1 Batch Job Object

208

5.1.2 Real-Time Job Object

211

5.2 Workflow

217

5.2.1 Areas of a Workflow

219

5.2.2 Continuous Workflow

220

5.3 Logical Flow Objects

222

5.3.1 Conditional

222

5.3.2 While Loop

223

5.3.3 Try and Catch Blocks

223

5.4 Data Flows

223

5.4.1 Creating a Standard Data Flow with an Embedded ABAP Data Flow

224

5.4.2 Creating an Embedded ABAP Data Flow

225

5.5 Transforms

231

5.5.1 Platform Transforms

231

5.5.2 Data Integrator Transforms

249

5.5.3 Data Quality Transforms

271

5.6 Datastores

299

5.6.1 SAP BW Source Datastores

299

5.6.2 SAP BW Target Datastore

303

5.6.3 RESTful Web Services

305

5.6.4 Using RESTful Applications in Data Services

307

5.7 File Formats

309

5.7.1 Flat File Format

310

5.7.2 Creating a Flat File Template from a Query Transform

316

5.7.3 Excel File Format

316

5.7.4 Hadoop

324

5.8 Summary

327

6 Variables, Parameters, and Substitution Parameters

329

6.1 Substitution Parameters

331

6.2 Global Variables

335

6.3 Variables

336

6.4 Local Variables

337

6.5 Parameters

339

6.6 Example: Variables and Parameters in Action

340

6.7 Summary

342

7 Programming with SAP Data Services Scripting Language and Python

343

7.1 Why Code?

343

7.2 Functions

345

7.3 Script Object

349

7.4 Coding within Transform Objects

352

7.4.1 User-Defined Transforms

352

7.4.2 SQL Transform

356

7.5 Summary

357

8 Change Data Capture

359

8.1 Comparing CDC Types

360

8.2 CDC Design Considerations

362

8.3 Source-Based CDC Solutions

363

8.3.1 Using CDC Datastores in Data Services

363

8.3.2 Oracle

369

8.3.3 SQL Server

369

8.4 Target-Based CDC Solution

372

8.5 Timestamp CDC Process

376

8.5.1 Limitations

376

8.5.2 Salesforce

377

8.5.3 Example

377

8.6 Summary

379

9 Social Media Analytics

383

9.1 The Use Case for Social Media Analytics

384

9.1.1 It’s Not Just Social

385

9.1.2 The Voice of Your Customer

386

9.2 The Process of Structuring Unstructured Data

387

9.2.1 Text Data Processing Overview

387

9.2.2 Entity and Fact Extraction

388

9.2.3 Grammatical Parsing and Disambiguation

393

9.3 The Entity Extraction Transform

394

9.3.1 Language Support

395

9.3.2 Entity Extraction Transform: Input, Output, and Options

396

9.4 Approach to Creating a Social Media Analytics Application

404

9.4.1 A Note on Data Sources

405

9.4.2 The Data Services Social Media Analysis Project

407

9.5 Summary

411

10 Design Considerations

413

10.1 Performance

414

10.1.1 Constraining Results

414

10.1.2 Pushdown

414

10.1.3 Enhancing Performance When Joins Occur on the Job Server

422

10.1.4 Caching

423

10.1.5 Degree of Parallelism (DoP)

424

10.1.6 Bulk Loading

425

10.2 Simplicity

426

10.2.1 Rerunnable

426

10.2.2 Framework

426

10.3 Summary

427

11 Integration into Data Warehouses

429

11.1 Kimball Methodology

430

11.1.1 Dimensional Data Model Overview

430

11.1.2 Conformed Dimensions and the Bus Matrix

431

11.1.3 Dimensional Model Design Patterns

433

11.1.4 Example Orders Star Schema

437

11.1.5 Processing Slowly Changing Dimensions

438

11.1.6 Loading Fact Tables

446

11.2 Hadoop and SAP HANA

451

11.3 Summary

454

12 Industry-Specific Integrations

455

12.1 Retail: Facilitating Customer Loyalty

455

12.1.1 The Solution

455

12.1.2 Results

464

12.2 Distribution: SAP BW/SAP APO Integration with SAP ECC

465

12.2.1 The Solution

466

12.2.2 Challenges

472

12.2.3 Results

473

12.3 Summary

473

13 SAP Information Steward

477

13.1 Match Review

477

13.1.1 Use Cases for Match Review

479

13.1.2 Terminology

479

13.1.3 Scenario

481

13.2 Cleansing Package Builder

490

13.3 Metadata Management

496

13.4 Summary

498

14 Where Is SAP Data Services Headed?

499

14.1 The Key Themes for SAP Data Services

500

14.2 The State of the State

501

14.2.1 Simple

502

14.2.2 Big Data

502

14.2.3 Enterprise Readiness

502

14.3 Beyond SP03

504

14.4 Help Shape the Future of Data Services

505

14.5 Summary

508