Designing Event Driven Systems

 

1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2. The Origins of Streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3. Is Kafka What You Think It Is?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

• Kafka Is Like REST but Asynchronous? 13
• Kafka Is Like a Service Bus? 14
• Kafka Is Like a Database? 15
• What Is Kafka Really? A Streaming Platform 15

4. Beyond Messaging: An Overview of the Kafka Broker. . . . . . . . . . . . . . . . . . . . . . . 17

• The Log: An Efficient Structure for Retaining and Distributing Messages 18
• Linear Scalability 19
• Segregating Load in Multiservice Ecosystems 21
• Maintaining Strong Ordering Guarantees 21
• Ensuring Messages Are Durable 22
• Load-Balance Services and Make Them Highly Available 23
• Compacted Topics 24
• Long-Term Data Storage 25
• Security 25
• Summary 25

Part II. Designing Event-Driven Systems

5. Events: A Basis for Collaboration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

• Commands, Events, and Queries 30
• Coupling and Message Brokers 32
• Using Events for Notification 34
• Using Events to Provide State Transfer 37
• Which Approach to Use 38
• The Event Collaboration Pattern 39
• Relationship with Stream Processing 41
• Mixing Request- and Event-Driven Protocols 42
• Summary 44

6. Processing Events with Stateful Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

• Making Services Stateful 47
• Summary 52

7. Event Sourcing, CQRS, and Other Stateful Patterns. . . . . . . . . . . . . . . . . . . . . . . . . 55

• Event Sourcing, Command Sourcing, and CQRS in a Nutshell 55
• Version Control for Your Data 57
• Making Events the Source of Truth 59
• Command Query Responsibility Segregation 61
• Materialized Views 62
• Polyglot Views 63
• Whole Fact or Delta? 64
• Implementing Event Sourcing and CQRS with Kafka 65
• Summary 71

Part III. Rethinking Architecture at Company Scales

8. Sharing Data and Services Across an Organization. . . . . . . . . . . . . . . . . . . . . . . . . . 75

• Encapsulation Isn’t Always Your Friend 77
• The Data Dichotomy 79
• What Happens to Systems as They Evolve? 80
• Make Data on the Outside a First-Class Citizen 83
• Don’t Be Afraid to Evolve 84
• Summary 85

9. Event Streams as a Shared Source of Truth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

• A Database Inside Out 87
• Summary 90

10. Lean Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

• If Messaging Remembers, Databases Don’t Have To 91
• Take Only the Data You Need, Nothing More 92
• Rebuilding Event-Sourced Views 93
• Automation and Schema Migration 94
• Summary 96

Part IV. Consistency, Concurrency, and Evolution

11. Consistency and Concurrency in Event-Driven Systems. . . . . . . . . . . . . . . . . . . . . 101

• Eventual Consistency 102
• The Single Writer Principle 105
• Atomicity with Transactions 108
• Identity and Concurrency Control 108
• Limitations 110
• Summary 110

12. Transactions, but Not as We Know Them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

• The Duplicates Problem 111
• Using the Transactions API to Remove Duplicates 114
• Exactly Once Is Both Idempotence and Atomic Commit 115
• How Kafka’s Transactions Work Under the Covers 116
• Store State and Send Events Atomically 118
• Do We Need Transactions? Can We Do All This with Idempotence? 119
• What Can’t Transactions Do? 119
• Making Use of Transactions in Your Services 120
• Summary 120

13. Evolving Schemas and Data over Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

• Using Schemas to Manage the Evolution of Data in Time 123
• Handling Schema Change and Breaking Backward Compatibility 124
• Collaborating over Schema Change 126
• Handling Unreadable Messages 127
• Deleting Data 127
• Segregating Public and Private Topics 129
• Summary 129

Part V. Implementing Streaming Services with Kafka

14. Kafka Streams and KSQL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

• A Simple Email Service Built with Kafka Streams and KSQL 133
• Table of Contents | v
• Windows, Joins, Tables, and State Stores 135
• Summary 138

15. Building Streaming Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

• An Order Validation Ecosystem 139
• Join-Filter-Process 140
• Event-Sourced Views in Kafka Streams 141
• Collapsing CQRS with a Blocking Read 142
• Scaling Concurrent Operations in Streaming Systems 142
• Rekey to Join 145
• Repartitioning and Staged Execution 146
• Waiting for N Events 147
• Reflecting on the Design 148
• A More Holistic Streaming Ecosystem 148
• Summary 150

Download:  

Source: Internet Articles / Linked in / etc..