microservices

Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith

by Sam Newman  · 14 Nov 2019  · 355pp  · 81,788 words

the technology choices we make, perhaps mixing different programming languages, programming styles, deployment platforms, or databases to find the right mix. Perhaps, above all, microservice architectures give you flexibility. They open up many more options regarding how you can solve problems in the future. However, it’s important to note

…

decomposition, and fundamentally what you are trying to achieve will drive this decomposition in different directions. Understanding what you are trying to get from your microservice architecture therefore becomes important. What Problems Do They Create? Service-oriented architecture became a thing partly because computers got cheaper, so we had more

…

delivery teams. Figure 1-5. An example of how true technology companies are integrating software delivery While not all organizations have made this shift, microservice architectures make this change much easier. If you want delivery teams aligned around product lines, and the services are aligned around the business domain,

…

single-process monolith, without having enough upsides of either. Encountering distributed monoliths in my work has in large part influenced my own interest in microservice architecture. Distributed monoliths typically emerge in an environment where not enough focus was placed on concepts like information hiding and cohesion of business functionality,

…

the code that handles the state transitions of an aggregate are grouped together, along with the state itself. When thinking about aggregates and microservices, a single microservice will handle the life cycle and data storage of one or more different types of aggregates. If functionality in another service wants to change

…

contexts may have relationships with other bounded contexts—when mapped to services, these dependencies become inter-service dependencies. Mapping Aggregates and Bounded Contexts to Microservices Both the aggregate and the bounded context give us units of cohesion with well-defined interfaces with the wider system. The aggregate is a self

…

either Eric Evans’s original Domain Driven Design or Vaughn Vernon’s Domain-Driven Design Distilled.12 Summary As we’ve discussed in this chapter, microservices are independently deployable services modeled around a business domain. They communicate with each other via networks. We use the principles of information hiding together

…

from and understand the impact of the decomposition itself. The vast majority of the important lessons will not be learned until your service hits production. Microservice decomposition can cause issues with troubleshooting, tracing flows, latency, referential integrity, cascading failures, and a host of other things. Most of those problems

…

the monolith. Refactoring the Monolith I’ve observed that often the biggest barrier to making use of existing code in the monolith in your new microservices is that existing codebases are traditionally not organized around business domain concepts. Technical categorizations are more prominent (think of all the Model, View, Controller

…

Extraction of functionality that still needs to use the monolith Rather than also reimplementing the User Notifications functionality, we expose this functionality to our new microservice by exposing it from the monolith—something that obviously would require changes to the monolith itself. For the strangler to work, though, you need

…

the need to explore these tools further, do take a look at Camunda and Zeebe, both of which are open source orchestration frameworks targeting microservice developers. Choreographed sagas Choreographed sagas aim to distribute responsibility for the operation of the saga among multiple collaborating services. If orchestration is command-and-

…

transferring them to the project). Potential Solutions In my experience, strong code ownership is almost universally the model adopted by organizations implementing large-scale microservice architectures consisting of multiple teams and over 100 developers. It becomes easier for the rules of what constitutes a good change to be decided by

…

OK?” Potential Solutions A multitude of mechanisms—some easy to implement, others more complex—can help change how you monitor and troubleshoot problems with a microservice architecture. What follows is a nonexhaustive overview of some key things to consider. Log aggregation With a small number of machines, especially machines that

…

Kotter, John P. Leading Change. Harvard Business Review Press, 1996. Mitchell, Lorna Jane. PHP Web Services, Second Edition. O’Reilly, 2016. Newman, Sam. Building Microservices. O’Reilly, 2015. Nygard, Michael T. Release It!: Design and Deploy Production-Ready Software, Second Edition. Pragmatic Bookshelf, 2018. Parnas, David. “On the Criteria to

…

of Computer Program and Systems Design. Prentice-Hall, 1979. Appendix B. Pattern Index Aggregate exposing monolith Exposing domain aggregates from a monolith to allow microservices to access entities managed by the monolith. Branch by abstraction Coexisting two implementations of the same functionality in the same codebase at the same time

…

an existing shared database, allowing for services to migrate away from direct use of the database. Decorating collaborator Trigger functionality running in a separate microservice by sniffing requests sent to the monolith, and the responses that are sent back in return. Dedicated reference data schema A dedicated database to

…

house all static reference data. This database can be accessed by multiple different services. Duplicate static reference data Copy static reference data into microservice databases. Monolith as data access layer Accessing data managed by the monolith via APIs rather than directly accessing the database. Move foreign key to

…

twice before making a breaking change when the problem might occur, When Might This Problem Occur? Brooks, Frederick P., Scale the Number of Developers Building Microservices (Newman), Modeled Around a Business Domain, Where to Use It, Sagas Versus Distributed Transactions, Growing Pains “burning platform”, How else could you do this

…

? business domainsdomain-driven design, Just Enough Domain-Driven Design microservices modeled around, Modeled Around a Business Domain business process modeling (BPM) tools, Orchestrated sagas C cachingclient-side caches, firing updates for, Pattern: Static reference

…

, Content-based routing continuous delivery (CD), Deployment coupling Continuous Delivery (Humble and Farley), Deployment coupling contracts, Breaking Changesconsumer-driven, Use consumer-driven contracts one microservice exposing two contracts, Give consumers time to migrate Conway's law, Modeled Around a Business Domain copying code from the monolith, Cut, Copy, or Reimplement

…

No transferring data ownership, Transferring Ownership-Where to use it dedicated reporting database, Potential Solutions in modular monoliths, And the modular monolith no sharing by microservices, Own Their Own Data schemas, But It Can’t Be Done! sharing, in implementation coupling, Implementation coupling decisions, reversible and irreversible, Reversible and Irreversible

…

Handbook, The (Kim, Humble, and Debois), It’s Not One Size Fits All distributed monoliths, The Distributed Monolith distributed systems, The Distributed Monolithevolution to microservices, Technology Distributed Systems Observability (Sridharan), Toward observability distributed transactionsavoiding use of, Distributed Transactions—Just Say No problems with, Sagas Versus Distributed Transactions sagas versus, Sagas

…

-Driven Design Domain-Driven Design Distilled (Vernon), Further Reading domains, Just Enough Domain-Driven Design(see also business domains; domain-driven design) domain coupling in microservices, Domain coupling unclear, getting service boundaries wrong, Unclear Domain duplicate static reference data pattern, Pattern: duplicate static reference data durability (ACID transactions), ACID Transactions

…

E edge-side includes (ESI), Example: Widget Composition empowering employees, Empowering Employees for Broad-Based Action encapsulation, Coupling end-to-end testing, problem with microservices, End-to-End Testing-Continually refine your quality feedback cycleshow the problem might occur, When Might This Problem Occur? how the problem shows itself, How

…

migration pattern, Example: FTP Function as a Service (FaaS), Potential Solutions G GitHub, Test in productionScientist library, GitHub Scientist global vs. local optimization of microservices, Global Versus Local Optimization-Potential Solutions Google, rolling out test automation at, Communicating the Change Vision Greenspun’s 10th rule, Deployment coupling Guardian, The (newspaper

…

3: Redirect calls I implementation coupling, Implementation coupling incremental migration, importance of, Importance of Incremental Migration incremental rewrites of monolithic code, Incremental rewrites independently deployable (microservices), Independent Deployability, Give consumers time to migrate information hiding, Coupling irreversible decisions, Reversible and Irreversible Decisions, Potential Solutions isolation (ACID transactions), ACID Transactions IT/

…

This Problem Show Itself? potential solutions to the problem, Potential Solutions when the problem might occur, When Might This Occur? local vs. global optimization of microservices, Global Versus Local Optimization-Potential Solutions local/remote developer workflows (hybrid), Potential Solutions lock-step releases, How Can This Problem Show Itself? log aggregation

…

Understanding the Goalfailing to understand why, Understanding the Goal key questions to ask, Three Key Questions reasons to choose and alternatives, Why Might You Choose Microservices?-How else could you do this? history of the term, Size independent deployability, Independent Deployability key takeaway points, Closing Words mapping aggregates and bounded contexts

…

to, Mapping Aggregates and Bounded Contexts to Microservices modeled around a business domain, Modeled Around a Business Domain modeled around databases, Own Their Own Data ownership within the organization, And Ownership problems created

…

Reason! startups, Startups unclear domain, Unclear Domain size of, Size technologies, Technology user interfaces, User Interfaces using shared libraries, Pattern: Static reference data library microservices, problems with, Growing Pains-Summarybreaking changes, Breaking Changes-Reportinghow the problem shows itself, How Can This Problem Show Itself? potential solutions, Potential Solutions-Give consumers

…

Use It modular monoliths, And the modular monolith, A modular monolith?Content Orchestration Service of The Guardian, Example: Widget Composition monitoring and troubleshooting, problems with microservices, Monitoring and Troubleshooting-Local Developer Experiencehow the problems show themselves, How Can These Problems Occur? potential solutions, Potential Solutions-Local Developer Experiencelog aggregation, Log

…

storage pattern, Pattern: Multischema storage, Pattern Index N N-version programming, Example: Homegate Listings networkslatencies and failures, What Problems Do They Create? microservices communicating via, What Are Microservices? NGINX proxy, Proxy Options NoSQL databases, Schema Separation Examples O observability, Toward observability operationsDevOps not meaning NoOps, It’s Not One Size Fits

…

embedding operations team members in delivery teams, It’s Not One Size Fits All operations team provisioning test environment, eliminating, Making a Change optimization of microservices, global vs. local, Global Versus Local Optimization-Potential Solutionshow the problem shows itself, How Can This Problem Show Itself? potential solutions, Potential Solutions when

…

refining, Continually refine your quality feedback cycles R Refactoring Databases (Ambler and Sadalage), What would I do? refactoring the monolith, Refactoring the Monolith registries for microservices, Potential Solutions relational databases, Schema Separation Examples Release It! (Nygard), Potential Solutions release-on-demand techniques, Deployment coupling releasesautomated release remediation, Use automated release

…

remediation and progressive delivery deployment versus, How It Works smaller, reducing risks with, Deployment coupling reportingproblems with microservices, Reporting-Potential Solutionspotential solutions, Potential Solutions when the problem might occur, When Might This Problem Occur? reporting database pattern, Pattern: Database-as-a-Service

…

in two-phase commits, Two-Phase Commits routing, content-based, in message interception strangler fig migration, Content-based routing running too many things in microservices, Running Too Many Things-Potential Solutionshow the problem shows itself, How Might This Problem Show Itself? potential solutions to the problem, Potential Solutions when

…

s Not One Size Fits All Spotify UI, componentized, And mobile applications Square orders example, tracer write pattern, Example: Orders at Square-Migrating consumers startups, microservices and, Startups State of DevOps Report (2017), Pattern: Branch by Abstraction static reference data library pattern, Pattern: Static reference data library, Pattern Index static

…

Might This Occur? organization of, three-tiered architecture reflecting, Modeled Around a Business Domain qualitative measures of transition success, Qualitative Measures reorganizing for transition to microservices, Reorganizing Teams-Changing Skillschanging skills, Changing Skills deciding where to start, Making a Change no one-size-fits-all, It’s Not One Size

…

data, Synchronizing the data retiring old source of truth, Pattern: Tracer Write where to use it, Where to Use It tracing sequences of calls between microservices, Tracing transaction log pollers implementation, change-data capture pattern, Transaction log pollers transactions, Transactions-Distributed Transactions—Just Say No, Sagas(see also sagas) ACID,

…

use it, Where to Use It user interfaces (UIs)leaving monolithic, problems with, User Interfaces Music Corp example, Modeled Around a Business Domain treating microservice interfaces as, Implementation coupling V Vault, Hashicorp, The Database as a Public Contract verify branch by abstraction pattern, As a Fallback Mechanism vertical scaling,

Building Microservices

by Sam Newman  · 25 Dec 2014  · 540pp  · 103,101 words

emerged over the last decade, which helps them avoid the pitfalls of many service-oriented architecture implementations. This book is full of concrete examples of microservice use around the world, including in organizations like Netflix, Amazon, Gilt, and the REA group, who have all found that the increased autonomy this

…

fine-grained systems and deal with some of the emergent complexity of distributed systems. Chapter 9, Security Here we’ll examine the security aspects of microservices and consider how to handle user-to-service and service-to-service authentication and authorization. Security is a very important topic in computing, one that

…

have been hard to comprehend just 10 years ago. Domain-driven design. Continuous delivery. On-demand virtualization. Infrastructure automation. Small autonomous teams. Systems at scale. Microservices have emerged from this world. They weren’t invented or described before the fact; they emerged as a trend, or a pattern, from real-world

…

more cohesive, often by creating abstractions or modules. Cohesion — the drive to have related code grouped together — is an important concept when we think about microservices. This is reinforced by Robert C. Martin’s definition of the Single Responsibility Principle, which states “Gather together those things that change for the same

…

reason, and separate those things that change for different reasons.” Microservices take this same approach to independent services. We focus our service boundaries on business boundaries, making it obvious where code lives for a given piece

…

10. Composability One of the key promises of distributed systems and service-oriented architectures is that we open up opportunities for reuse of functionality. With microservices, we allow for our functionality to be consumed in different ways for different purposes. This can be especially important when we think about how our

…

domain, make sure they are modeled within your codebase as modules, with shared and hidden models. These modular boundaries then become excellent candidates for microservices. In general, microservices should cleanly align to bounded contexts. Once you become very proficient, you may decide to skip the step of keeping the bounded context modeled

…

contexts. For example, you could decompose the warehouse into capabilities associated with order fulfillment, inventory management, or goods receiving. When considering the boundaries of your microservices, first think in terms of the larger, coarser-grained contexts, and then subdivide along these nested contexts when you’re looking for the benefits of

…

in Figure 3-3, so rather than a single warehouse boundary, you might instead split out inventory, order fulfillment, and goods receiving. Figure 3-2. Microservices representing nested bounded contexts hidden inside the warehouse Figure 3-3. The bounded contexts inside the warehouse being popped up into their own top-level

…

or change the behavior in three different places, and deploy those changes too. Goodbye, cohesion. Remember when we talked about the core principles behind good microservices? Strong cohesion and loose coupling — with database integration, we lose both things. Database integration makes it easy for services to share data, but does

…

request/response patterns. What about event-based, asynchronous communication? Technology Choices There are two main parts we need to consider here: a way for our microservices to emit events, and a way for our consumers to find out those events have happened. Traditionally, message brokers like RabbitMQ try to handle both

…

we cannot predict exactly how a customer might end up interacting with our company has driven adoption of more granular APIs, like those delivered by microservices. By combining the capabilities our services expose in different ways, we can curate different experiences for our customers on their desktop application, mobile device,

…

monolithic systems can be challenging, especially if you don’t have access to systems that are easily automatable. When you think about multiple environments per microservice, this can be even more daunting. We’ll look shortly at some different deployment platforms that can make this much easier for us. Service Configuration

…

benefits in terms of improved manageability, such as clustering support to handle grouping multiple instances together, monitoring tools, and the like. Figure 6-7. Multiple microservices per host This setup can also yield benefits in terms of reducing overhead of language runtimes. Consider running five Java services in a single Java

…

written automatically? Can you deploy database changes without manual intervention? Embracing a culture of automation is key if you want to keep the complexities of microservice architectures in check. Two Case Studies on the Power of Automation It is probably helpful to give you a couple of concrete examples that explain

…

-Functional Testing The bulk of this chapter has been focused on testing specific pieces of functionality, and how this differs when you are testing a microservice-based system. However, there is another category of testing that is important to discuss. Nonfunctional requirements is an umbrella term used to describe those

…

actions like rate-limiting specific callers to protect quality of service for other people. When it comes to using API keys to handle your own microservice-to-microservice approach, the exact mechanics of how it works will depend on the technology you use. Some systems use a single API key that is

…

a more active capacity. Network Segregation With a monolithic system, we have limits to how we can structure our networks to provide additional protections. With microservices, though, you can put them into different network segments to further control how services talk to each other. AWS, for example, provides the ability

…

of the UI containing the cart control with a phone number for placing an order. But for every customer-facing interface that uses multiple microservices, or every microservice that depends on multiple downstream collaborators, you need to ask yourself, “What happens if this is down?” and know what to do. By thinking

…

a cat: you might find your own practices to help deliver on these principles, but this should get you started. Figure 12-1. Principles of microservices Model Around Business Concepts Experience has shown us that interfaces structured around business-bounded contexts are more stable than those structured around technical concepts. By

…

implementation details, although even if using remote procedure calls (RPCs), you can still embrace these ideas. Decentralize All the Things To maximize the autonomy that microservices make possible, we need to constantly be looking for the chance to delegate decision making and control to the teams that own the services themselves

…

ensure that you keep associated logic and data within service boundaries, helping keep things cohesive. Independently Deployable We should always strive to ensure that our microservices can and are deployed by themselves. Even when breaking changes are required, we should seek to coexist versioned endpoints to allow our consumers to change

…

to me the importance of starting gradually so you understand your organization’s appetite and ability to change, which will help you properly adopt microservices. Parting Words Microservice architectures give you more options, and more decisions to make. Making decisions in this world is a far more common activity than in simpler

…

Keys, It’s All About the Keys application containers, Application Containers architects (see systems architects) architectural principlesdevelopment of, Principles Heroku's 12 factors, Principles key microservices principles, Bringing It All Together real-world example, A Real-World Example architectural safety, Architectural Safety, Architectural Safety Measures artifactsimages, Images as Artifacts operating system

…

Two Case Studies on the Power of Automation autonomymicroservices and, Autonomous role of systems architect in, Summary autoscaling, Autoscaling availabilityin CAP theorem, CAP Theorem key microservices principle of, How Much Is Too Much? sacrificing, Sacrificing Availability B backends for frontends (BFFs), Backends for Frontends backup data pumps, Backup Data Pump backups

…

client libraries, Client Libraries client-side caching, Client-Side, Proxy, and Server-Side Caching code reuse, DRY and the Perils of Code Reuse in a Microservice World coding architect, Zoning cohesion, Small, and Focused on Doing One Thing Well, High Cohesion Cohn's Test Pyramid, Test Scope collaboration, Summaryevent-based,

…

Pipelines and Continuous Delivery continuous integration (CI)basics, A Brief Introduction to Continuous Integration checklist for, Are You Really Doing It? mapping to microservices, Mapping Continuous Integration to Microservices Conway's lawevidence of, Evidence in reverse, Conway’s Law in Reverse statement of, Conway’s Law and System Design summary of, Summary

…

code (HMAC), HMAC Over HTTP HATEOS principle, Hypermedia As the Engine of Application State heterogeneitybenefits of, Technology Heterogeneity shared libraries and, Shared Libraries hexagonal architecture, Microservices hidden models, Shared and Hidden Models high cohesion, High Cohesion HMAC (hash-based messaging code), HMAC Over HTTP HTTP (Hypertext Transfer Protocol)caching in, Caching

…

I idempotent operations, Idempotency identity provider, Common Single Sign-On Implementations imagesas artifacts, Images as Artifacts custom, Custom Images immutable servers, Immutable Servers infrastructure automation, Microservices integrationaccess by reference, Access by Reference asynchronous event-based collaboration, Implementing Asynchronous Event-Based Collaboration customer interface, Interfacing with Customers DRY (Don't Repeat Yourself

…

), DRY and the Perils of Code Reuse in a Microservice World goals for, Looking for the Ideal Integration Technology guidelines for, Summary importance of, Integration orchestration vs. choreography, Orchestration Versus Choreography reactive extensions, Reactive

…

based authentication, API Keys Kibana, Logs, Logs, and Yet More Logs… L latency, How Much Is Too Much? Latency Monkey, The Antifragile Organization layered architectures, Microservices librariesclient, Client Libraries service metrics, Service Metrics shared, Shared Libraries Linux containers, Linux Containers load balancing, Load Balancing load shedding, Bulkheads local calls, Local Calls

…

One Thing Well technology heterogeneity of, Technology Heterogeneity vs. modules, Modules vs. service-oriented architecture, What About Service-Oriented Architecture? vs. shared libraries, Shared Libraries microservices at scaleantifragile systems, The Antifragile Organization architectural safety measures, Architectural Safety Measures autoscaling, Autoscaling caching, Caching caching for writes, Caching for Writes CAP theorem, CAP

…

network segregation, Network Segregation nonfunctional requirements, Cross-Functional Testing normalization of deviance, Flaky and Brittle Tests O on-demand provisioning systems, Scaling on-demand virtualization, Microservices onion architecture, The Technical Boundary Open Web Application Security Project (OWASP), Baking Security In OpenID Connect, Common Single Sign-On Implementations, Use SAML or OpenID

…

Tests service-oriented architectures (SOA)concept of, What About Service-Oriented Architecture? drawbacks of, What About Service-Oriented Architecture? reuse of functionality in, Composability vs. microservices, What About Service-Oriented Architecture? service-to-host mapping, Service-to-Host Mapping-Platform as a Serviceapplication containers, Application Containers multiple services per host, Multiple

…

the Perimeter SAML/OpenID Connect, Use SAML or OpenID Connect sharding, Scaling for Writes shared code, DRY and the Perils of Code Reuse in a Microservice World shared data, Example: Shared Data shared libraries, Shared Libraries shared models, Shared and Hidden Models shared static data, Example: Shared Static Data shared

…

virtual private clouds (VPC), Network Segregation virtualizationhypervisors, Traditional Virtualization traditional, Traditional Virtualization type 2, Traditional Virtualization virtualization platformsDocker, Docker Linux containers, Linux Containers on-demand, Microservices storage area networks in, Spreading Your Risk Vagrant, Vagrant vision, Summary W worker-based systems, Worker-Based Systems write-behind caches, Caching for Resilience X

…

better software systems.” He has written articles, presented at conferences, and sporadically commits to open source projects. Colophon The animals on the cover of Building Microservices are honey bees (of the genus Apis). Of 20,000 known species of bees, only seven are considered honey bees. They are distinct because they

…

Asynchronous Event-Based Collaboration Technology Choices Complexities of Asynchronous Architectures Services as State Machines Reactive Extensions DRY and the Perils of Code Reuse in a Microservice World Client Libraries Access by Reference Versioning Defer It for as Long as Possible Catch Breaking Changes Early Use Semantic Versioning Coexist Different Endpoints Use

…

Time Cost of Change Understanding Root Causes Summary 6. Deployment A Brief Introduction to Continuous Integration Are You Really Doing It? Mapping Continuous Integration to Microservices Build Pipelines and Continuous Delivery And the Inevitable Exceptions Platform-Specific Artifacts Operating System Artifacts Custom Images Images as Artifacts Immutable Servers Environments Service Configuration

…

the Custodians Maturity Tooling Bounded Contexts and Team Structures The Orphaned Service? Case Study: RealEstate.com.au Conway’s Law in Reverse People Summary 11. Microservices at Scale Failure Is Everywhere How Much Is Too Much? Degrading Functionality Architectural Safety Measures The Antifragile Organization Timeouts Circuit Breakers Bulkheads Isolation Idempotency Scaling

…

Don’t Forget the Humans! Documenting Services Swagger HAL and the HAL Browser The Self-Describing System Summary 12. Bringing It All Together Principles of Microservices Model Around Business Concepts Adopt a Culture of Automation Hide Internal Implementation Details Decentralize All the Things Independently Deployable Isolate Failure Highly Observable When Shouldn

Principles of Web API Design: Delivering Value with APIs and Microservices

by James Higginbotham  · 20 Dec 2021  · 283pp  · 78,705 words

V: Refining the Design – improves the API design based on insights from documentation, testing, and feedback. It also includes a chapter on decomposing APIs into microservices. Finally, the book closes with tips on how to scale the design process in larger organizations. For those that need a refresher on HTTP, the

…

API Design 8. RPC and Query-Based API Design 9. Async APIs for Eventing and Streaming V: Refining the API Design 10. From APIs to Microservices 11. Improving the Developer Experience 12. API Testing Strategies 13. Document the API Design 14. Designing for Change 15. Protecting APIs 16. Continuing the API

…

APIs Summary V: Refining the API Design 10. From APIs to Microservices What Are Microservices? Microservices Reduce Coordination Costs The Difference Between APIs and Microservices Weighing the Complexity of Microservices Synchronous and Asynchronous Microservices Microservice Architecture Styles Right-Sizing Microservices Decomposing APIs Into Microservices Considerations When Transitioning to Microservices Summary 11. Improving the Developer Experience Creating a Mock API Implementation

…

systems integration. The ADDR process is focused on identifying and realizing these outcomes. What are Job Stories? Customers and users don’t care about APIs, microservices, serverless, or the flavor of front-end framework used. They want a solution to a problem. They care about outcomes. Job stories capture the jobs

…

efforts and division of labor to minimize cross-team dependency coordination There are circumstances when EventStorming is most effective. These include: ■ prior to API and microservice design and implementation to help establish outside-in design thinking ■ prior to software design and development to clarify assumptions and address open questions ■ after clearly

…

, large API product makes it difficult for developers to find what they need quickly. Likewise, many small API products, perhaps as a result of externalizing microservices individually, may result in fragmentation and frustration. Applying clear API boundaries helps to reduce confusion around very large or many small APIs. The Overloaded API

…

helper libraries and command-line interfaces is also explored. Finally, tips are provided for scaling the ADDR Process for larger organizations. 10. From APIs to Microservices The biggest fallacy about monoliths is you can have only one. Kelsey Hightower All organizations want to deliver business value as fast as possible. At

…

10.1 Refining the API design may include decomposing it into smaller services to reduce the overall complexity of the solution. What Are Microservices? Microservices are small, independently deployed components that deliver one or a small number of bounded digital capabilities. Each service offers one of the many digital capabilities

…

required, ensuring that each service has a limited scope. When combined, microservices deliver a highly complex solution using smaller building blocks than the traditional service-oriented approach. This is illustrated in Figure 10.2. Figure 10.2

…

decompose highly complex systems into independently deployed components rather than containing the complexity within a single codebase. The cognitive load required to understand a single microservice is lowered compared to that of understanding a single codebase. Testing becomes more approachable as well as automated test suites become more focused on the

…

between services unnecessarily. There are other definitions that exist as well. Use caution when the organization makes a broad declaration that they are moving to microservices. First, be sure to understand what is meant by the term. Be specific in the definition and intent. Next, seek a reference architecture and one

…

a team with more time to design, code, test, and deliver their services. Coordination across teams is not eliminated with microservices, however. Integration must be coordinated to ensure that all the microservices fit the solution needs. Timelines must still be coordinated between product managers, business, and service teams. Therefore, the number of

…

ownership, allowing each service to own and manage the data associated with their services Without incorporating these important factors, any shift to microservices will be met with challenges, including bloated microservices, slower velocity of delivery, and even project failure. This is discussed further in Chapter 6 of the book Strategic Monoliths and

…

requirement, the service must be transitioned to an API product with a stable interface Just because the codebase is small doesn't make it a microservice. A microservice is an internal component and shouldn't be shared directly with external consumers. API products may be shared within a specific team, across teams

…

, across the organization, and/or with partners/public developers. Weighing the Complexity of Microservices The most important factor when considering microservices is the complexity of the solution. Complexity cannot be fully removed from a software solution. However, it may be distributed across the

…

, the infrastructure and automation requirements to deliver, monitor, and protect the service at runtime increases. If the solution has a low factor of complexity, then microservices are often unnecessary and may even be detrimental to solution success. If the complexity of the solution is unknown, weight the factors below and then

…

reporting structures and team alignment. Count the cost before proceeding. Otherwise, the organization may be trading code complexity for organizational complexity. Shift in Data Ownership Microservices must own their own data. This can be a challenging item, as rarely do teams think beyond the source code when it comes to shifting

…

efforts to bring every service in line with a breaking schema change within a shared data source. Distributed Data Management and Governance Microservices require considerable data management and governance. Since microservices own their own data, investment must be made to ensure that proper data management policies exist for reporting and analytics. Today

…

this is typically handled through ETL-based processes that migrate data into an OLAP-based data store for optimized queries and decision support. Shifting to microservices requires shifting to data streaming rather than ETL processes to bring together data from multiple services for the purposes of reporting. More emphasis needs to

…

ontology and taxonomy to unify distributed data models. Organizations with centralized data model governance and large shared databases must use caution when migrating to a microservice architecture. Finally, don’t underestimate the effort required to separate a monolithic data store into a data store per service. Distributed Systems Challenges The journey

…

toward microservices requires a deep understanding of distributed systems. Those not as familiar with the concepts of distributed tracing, observability, eventual consistency, fault tolerance, and failover will

…

encounter a more difficult time with microservices. The eight fallacies of distributed computing, written in 1994 and still applicable today, must be understood by every developer. Additionally, many find that architectural oversight

…

decompose and subsequently integrate services into solutions. Teams unable to have architectural support may suffer from lack of architectural consideration in the design of their microservices, resulting in poor boundaries and overlapping team responsibilities that produces increased cross-team coordination. The Align Phase of the ADDR Process seeks to address this

…

were used to create distributed transactions, usually through the use of 2-phase commit (2PC) transactions. This isn’t an option with a highly distributed microservice architecture. Instead, distributed transactions are often implemented using the saga pattern. A transactional context is applied within each service call, with compensating transactions used to

…

a shared codebase for common code. Sharing code between services can create coordination coupling, requiring more meetings to ensure a change to code shared across microservices doesn’t negatively impact others. When sharing code between services, all changes must be optional to avoid forcing other teams to be in lockstep. Do

…

apply loose coupling and high cohesion within a single codebase to avoid the complexity of distributed computing. Over time, the monolith may be decomposed into microservices if the solution becomes too complex for a single codebase. However, only apply this approach once all paths to refactoring and re-organizing the modules

…

offers one or a few APIs that support the operations within the bounded contexts contained within the monolith. Synchronous and Asynchronous Microservices Microservices may be designed to be synchronous or asynchronous. Synchronous microservices apply a more traditional request/response model, typically via HTTP using REST constraints, RPC, or a query-based API. While

…

as the service client only called Service A, which results in more service calls that can fail due to a downstream error. Figure 10.4 Microservices that use a synchronous, request/response style result in call chains that can fail without the client’s knowledge. Alternatively, an asynchronous access pattern may

…

be used for microservice integration. In this style, messages are submitted to a message queue or topic hosted on a message broker or streaming server. One or more

…

service communicates with other services directly using a synchronous or asynchronous model. This approach is the most common style found during the early days of microservices. Those using a synchronous model encounter challenges such as service communication failure and call chain fragility. The introduction of a service mesh helps to overcome

…

these challenges, as does the shift to a more asynchronous model that is message driven. Figure 10.6 depicts this more traditional microservice architecture style. Figure 10.6 Direct service communication allows any service to invoke any other service. API-Based Orchestration This style starts with the design

…

by organizations that have struggled with some of the challenges of the direct service communication model. Many of the organizations that were early adopters of microservices are moving to this model. This is shown in Figure 10.7. Figure 10.7 The API-based orchestration style offers increased contract stability while

…

often. This can only be done efficiently if service ownership resides with a single team. Services shared across teams require further coordination meetings. Right sizing microservices requires a continuous process of design and re-evaluation: 1. First, identifying where transactional boundaries exist to find candidate service boundaries. This will help to

…

of transactional boundaries while keeping team coordination costs low Tip It is best to focus less on the size of the microservice and more on the purpose of the service. Microservices should seek to make future change easier, even if that means the service is more course-grained at the start. Decomposing

…

created API sequence diagrams with more detail, identifying candidate services, and capturing the service design details. Step 1: Identify Candidate Microservices The first step in decomposing APIs is to identify candidate microservices. Start by expanding the web sequence diagrams, created during the API modeling and design phases, to include external systems and

…

search books operation of the Shopping API, this is a good candidate for decomposing into a separate service. The team that will own this candidate microservice will be responsible for ensuring the search engine indexes are both performant and deliver the search capabilities required by customers. Figure 10.10 show the

…

boundary for the candidate microservice that will support book searches. Figure 10.10 Since the search engine integration will require specialized knowledge of how to properly index and search entities

…

supporting the needs of the API. Figure 10.12 shows an example MDC for the Book Search Service. Figure 10.12 A Microservice Design Canvas that captures the candidate microservice, including design considerations, prior to implementation. At this point, the MDC provides sufficient details to proceed with building and integrating the service

…

that may suffer from degraded performance during peak usage, this could result in several seconds of wait time for an end user. Finally, for some microservice ecosystems, it may not be possible to know how many services are involved or predict the total time required for execution. When possible, keep a

…

to increased complexity. For example, the complexity of payment processing integration may merit shifting this behavior to a separate microservice. Considerations When Transitioning to Microservices While there are many benefits to moving to microservices, the transition shouldn't be taken lightly. After some time and reflection, some organizations have chosen to simplify their

…

.g., an application that offers CRUD-based forms to manage a dataset. The result is wasted time and effort to decompose a simple solution into microservices that introduce unnecessary complexity around runtime management, troubleshooting complexity, and distributed transaction management. Next, be sure that the organization’s reporting structure and culture are

…

doubt, apply the “you ain’t gonna need it” (YAGNI) principle of agile software by starting with a modular monolith API and decomposing it into microservices when the need arises. 11. Improving the Developer Experience Every useful API that delivers value will typically have multiple consumers. This is a natural asymmetry

…

of any specific programming languages and frameworks selected by each process, making it portable. Service meshes have grown in popularity due to the introduction of microservices but may be used for any architecture or combination of architectural styles. Service meshes replace the direct communication of processes with a series of proxies

Seeking SRE: Conversations About Running Production Systems at Scale

by David N. Blank-Edelman  · 16 Sep 2018

request or statement. At the highest level, availability-related context as shared at Netflix with an engineering team would be the trended availability of their microservice[s] and how that relates to the desired goal, including availability of downstream dependencies. With this domain-specific context, an engineering team has the responsibility

…

punitive action. This action might involve removing their ability to push code to production. At Netflix, we err toward the former model, sharing context on microservice-level availability, then working with teams when needed to help improve availability. The challenge is making sure sufficient context is provided to teams. When someone

…

. The framework serves a fallback experience when possible to still provide service to an end user. In conclusion, setting and measuring reliability goals at the microservice level lets you move toward a desired aggregate service-level availability. Different organizations might use the terms availability and reliability differently, but this is how

…

they’re significantly impacting actual production availability, such as consistently having production incidents that result in customers not being able to stream content, then the microservice availability scorecard context is probably not the way to solve that problem. In that case, they have most likely been receiving the report but not

…

human-delivered versus a system-delivered report. At Netflix, the Core SRE team doesn’t have the responsibility to step in and troubleshoot and resolve microservice-specific issues, but the team is instead considered the “central nervous system of Netflix.” Taking the central nervous system analogy a bit further, this is

…

successfully apply SRE principles in our scenario, we needed to instill the SRE mindset into everybody. The Deployment Platform Very early in our migration to microservices, we learned that our processes and infrastructure were not ready for the resulting large number of services. Each service required dedicated servers, the configuration management

…

easy to use the right solutions, without mandating them, we added extensive Prometheus instrumentation to the preferred application framework, which wraps Finagle. Most of our microservices work in a network-heavy scatter-gather fashion with only limited internal logic. By exposing metrics about incoming and outgoing network calls, the users of

…

in less common languages will often lack the more sophisticated instrumentation and client-side load-balancing capabilities that we need to maintain a fleet of microservices. At SoundCloud, technical direction more often emerges from the bottom up than it is decided from the top down. In the transition away from being

…

hold separate meetings for every incident. Decoupling production readiness reviews from the initial release of a feature was inspired by Susan Fowler’s Production-Ready Microservices (O’Reilly, 2016). Until its release, we had struggled to come up with a definition of “good” services that would not be seen as a

…

environment; create a simple, Dockerized service; manage secrets; add storage; prepare for on-call; and, finally, properly deprecate a service. We built Apollo, our Java microservice framework, which would give a developer many features for free, like metrics instrumentation, logging, and service discovery. Heroic and Alien, our time series database (TSDB

…

more than a hundred cross-functional teams, we might find machine learning an effective way to make sense of an ever-growing, complex ecosystem of microservices. Furthermore, we also have the opportunity to revolutionize what has largely been manual tunables in the past, whether it be “right-sizing” our cloud capacity

…

consistent hash of the ID directs requests for any particular customer to one particular primary node. Because teams are responsible for the operations of their microservices, they also ensure that the clusters autoscale to responsibly use resources, and hand off data in the case that a node goes down. They also

…

back quickly enough, we can respond to the unreasonable nature of a complex system before unwanted systemic effects take hold or cause too much damage. Microservices help with this by decoupling deployments and allowing small parts to change and roll back asynchronously. Some development practices like XP and Agile also optimize

…

prevent this. At Netflix, there is no CTO or, as I mentioned earlier, chief architect to pick best practices and issue an edict that all microservices must conform. The best practice solutions were at hand, but there was no efficient mechanism of delivery. Instead of dictating best practices, the engineers at

…

) and very-large-scale disruptions (regions disappearing). Not much existed to generate a Safety signal for the systemic effects: all of the interesting interactions between microservices that lead to unforeseeable effects, both good and bad. What would such a system look like? I was given a small budget (two headcount) to

…

architecture, it can mean producing component and consuming component. Examples of a producing component would be a network accessible block storage system and an authorization microservice API; each would have many disparate customers and make promises to those other services about availability, performance, and sometimes even safety. SLAs are actually quite

…

of those single points of failure experiences an outage. To guard against such a scenario, site reliability practices will segment the code (using either a microservice architecture or through functional segmentation across different failure domains) and also the underlying data store so that a failure in any critical component will not

…

not doing SRE. Or not successfully, anyway. Sublinear scaling is the SRE watchword, for people, processes, systems — everything.14 Pets < Cattle15 < Poultry.16 Containers and microservices are the One True Path.17 At least until they are potentially replaced by “serverless” functions.18 In the meantime,19 get your fleet standardized

…

buzzes with phrases such as “immutable container provisioning and scheduling,” “continuous integration and deployment,” “decentralized control,” and “polyglot language implementations.” It is absolutely true that microservice architectures do allow large development teams to operate with more agility. However, often overlooked is the harsh reality that large internet companies have typically invested

…

? How are distributed services observed and debugged? How are services packaged and deployed? What kind of complex failure scenarios can happen? As on-the-ground microservice practitioners soon realize, the majority of operational problems that arise when moving to a distributed architecture are ultimately grounded in two areas: networking and observability

…

paradigm1 has emerged that is most commonly known as the service mesh. The service mesh offers a reprieve for those attempting to build and operate microservice architectures by creating a generic substrate on which applications communicate. Developers can write applications in any language, largely unaware of how the distributed network is

…

, instrumented, and, most important, reliably operated. This chapter explores the reasoning behind the introduction of the service mesh as well as the architectural benefits to microservice developers and reliability engineers alike. The chapter ends with a case study of the deployment of an Envoy-based service mesh at Lyft. Ready to

…

Get Rid of the Monolith? Before anyone starts thinking about microservices, they typically already have a functioning monolithic application composed of the following pieces, as shown in Figure 26-1: Internet load balancer (e.g., Amazon

…

architecture versus a fully distributed one. This chapter does not go into detail on why companies eventually almost always make the switch from monolith to microservice architecture; much has already been written about that topic.2 However, from a networking and observability perspective it’s instructive to discuss some of the

…

primarily due to networking issues and observability (or rather lack thereof). When a decision is made to move away from the monolith toward a distributed microservice architecture (often for good reasons), the nascent networking and observability issues already seen become substantially worse almost immediately. In fact, it is not uncommon for

…

how to debug it and thus they do not trust it. Developers then typically return to adding features to the monolith, leaving a few “noncritical” microservices running. This is a vicious cycle. In some sense, making the network reliable, transparent, and easy to operate is the requirement for a successful distributed

…

each other. Mechanisms range in complexity from Domain Name System (DNS) to fully consistent solutions like Consul. Distributed system best practices At a theoretical level, microservice practitioners are told that they need to employ best practices such as retries with exponential back-off, circuit breaking, rate limiting, and timeouts. The actual

…

that arise is almost impossible. This leads directly to the general perception that distributed networking is unreliable for application development. To have a successful distributed microservice rollout, it is absolutely critical that all deployed services can access a consistent set of features and observability. As an industry can we do better

…

the face of the confusing networking landscape presented in the previous section, how can developers and reliability engineers take control and bring sanity back to microservice application development? They have two real options: Option 1 Limit the number of languages used within the organization and introduce extremely complex libraries that encapsulate

…

entire system Tracing is an incredibly powerful tool to visualize distributed system request flows. Typically, however, it’s quite complex to introduce it into a microservice architecture because every application must be modified to create spans, forward context, and so on. The service mesh can provide 100% tracing coverage with no

…

. Configuration Management (Control Plane Versus Data Plane) So far, we have discussed many of the features that the service mesh can provide when building reliable microservice architectures. We have not discussed how the entire system is configured, which can become quite complicated. First, though, a few definitions: Data plane This is

…

26-3. Service mesh with dynamic configuration APIs The Service Mesh in Practice Although the service mesh concept has only recently gained favor among the microservice community, there are already several competing solutions and large deployments. Proxy solutions include HAProxy, NGINX, Linkerd, Traefik, and Envoy. Fully managed solutions include SmartStack (built

…

vendors as well as the large cloud vendors. The benefits to application developers and reliability engineers are tremendous; after an engineer uses and operates a microservice built on top of a well-functioning service mesh, it is very unlikely that they will ever want to deploy an application without one again

…

too large. Further Reading Envoy proxy “Service mesh data plane vs. control plane” Istio service mesh “Lyft’s Envoy dashboards” “The universal data plane API” “Microservices Patterns With Envoy Sidecar Proxy: The series” “Introduction to modern network load balancing and proxying” “Embracing eventual consistency in SoA networking” Bio Matt Klein is

…

. Service meshes have origins going all the way back to mainframes and Enterprise Service Buses (ESBs). 2 The writings of Martin Fowler (https://martinfowler.com/microservices/) are probably the most well known on this topic. 3 The typical solution to this problem is to install a high-performance proxy such as

…

performance. Operations now need to understand the ORM layer and why it impacts the database. Monolithic designs are often easier to develop and extend than microservices. There can be valid business reasons to avoid duplication of sensitive or complex code, and they have simpler orchestration configuration. However, because they attract heterogeneous

…

the ground up.18 This leads to a situation in which a number of building blocks are indeed composed together, typically in some kind of microservice architecture, but because each organization does it from first principles every time, there is no meaningful industry-wide cooperation on a single stack for serving

…

, this often means keeping your tactics and actions physically separated from others’ so as to contain risks. Model-View-Controller paradigms, virtual machines, containers, and microservices are all highly specific variations on this theme. SREs define clear interfaces between, and SLOs for, systems and ensure that each component has a specific

…

Everywhere-Conclusion Kissner, Lea, The General Landscape of Privacy Engineering Klein, Matt, SRE throughout the development cycleon service meshes, The Service Mesh: Wrangler of Your Microservices?-The Future of the Service Mesh Knight Capital, Sacrifice Decisions Take Place Under Uncertainty known-knowns, software failure and, Underlying Assumptions Driving On-Call for

…

Future: Speed at Scale, Safely microservicescontext vs. control at Netflix, Context Versus Control in SRE-Context Versus Control in SRE current state of microservice networking, Current State of Microservice Networking service mesh and (see service mesh) mid-sized organizations (see Soundcloud, SRE at) migration cost, Project Operating Expense and Abandonment Expense(see

…

Come From? Obama, Barack, Elegy for Complex Systems object stores, Object stores object-relational mapping (ORM), Cognitive overload observability, service mesh and, Current State of Microservice Networking, Observability and Alarming on-callaccommodations for on-call personnel, Accommodations alternatives to current approach, Actual Solutions-Cognitive hacks arguments for keeping current system, Counterarguments

…

servers as cattle vs. servers as pets, Databases Are Not Special service discovery, Eventually Consistent Service Discovery service meshes, The Service Mesh: Wrangler of Your Microservices?-The Future of the Service Meshconfiguration management, Configuration Management (Control Plane Versus Data Plane) context propagation, Thin Libraries and Context Propagation control plane vs. data

…

plane, Configuration Management (Control Plane Versus Data Plane) current state of microservice networking, Current State of Microservice Networking eventually consistent service discovery system, Eventually Consistent Service Discovery in practice, The Service Mesh in Practice-Technical learnings monolithic architecture vs., Ready

Ask Your Developer: How to Harness the Power of Software Developers and Win in the 21st Century

by Jeff Lawson  · 12 Jan 2021  · 282pp  · 85,658 words

got us really enthusiastic. After that hackathon we were convinced that we could build a contact center and move toward architecture that uses APIs and microservices.” This experience gave Theo the courage to pitch his idea to management. Maybe it was not as insane as it seemed at first glance. But

…

, the surface area of their service was typically somewhat limited as well. Over time, these became known as “microservices” because each individual service typically did one thing, and did it well. These microservices were delivered not as a pile of code, nor as a website, but as a web-based API. APIs

…

here’s where it gets even more interesting. Once you’ve divided and subdivided the business into small teams that specialize in particular areas, offering microservices for each other to use, with well-documented interfaces and pricing that represents the true costs of delivering those services—well, why develop all of

…

those microservices internally? Why devote your own developers to microservices that you could instead buy from other companies? Why write your own microservice for calculating currency translation on international sales when you can just buy that

…

microservice from a vendor that specializes in currency translation software? So your developers start plugging in pieces from specialist providers, and boom—you have a software

…

supply chain. Does it really matter what company’s logo these microservice providers have on their business cards? Soon people started figuring out that you could make a business out of building

…

microservices and selling them to others. New Relic launched in 2008 and developed software that monitors website performance. Stripe developed payment processing services. Twilio developed a

…

plug our code into theirs, write a bit of their own code, and off they go. In total, Twilio now operates more than one thousand microservices. We sell them on a pay-for-what-you-use basis, the same way Amazon sells its computing power. Cloud platforms are the new building

…

benefits of the Third Great Era of Software, but also a new set of questions to answer. Developers and company leadership are constantly deciding which microservices to buy from third-party cloud suppliers, and which ones to build themselves. Decision makers need to stay up to speed with the Cambrian explosion

…

of new microservice providers that are racing into the market. Each microservice is continually and rapidly changing and improving. “Tech companies are constantly debating” which microservices to build and which to buy, says Ashton Kutcher. He has invested in dozens

…

that make it up are taken off the shelf and rapidly integrated and integrated upon. You’ll add into the mix many of your own microservices, but instead of re-creating the wheel, the services you build are those that represent your “secret sauce” areas of competitive advantage. Maybe it’s

…

of these off-the-shelf building blocks are gaining maturity all the time. Cloud platforms are replacing legacy infrastructure in nearly every category. These commercial microservices are the raw ingredients of almost every app you use. Behind that user interface, apps are actually a patchwork comprising hundreds or even thousands of

…

microservices, some created by a company’s own developers and some from commercial providers. Today, there are still many places where developers have no choice but

…

to build microservices from scratch because off-the-shelf alternatives don’t exist yet. But the beauty of the microservice model is that it’s possible to swap out a microservice without interfering with the rest of the code in your app. As

…

new commercial microservices emerge, it’s not uncommon to remove the homegrown pieces over time and

…

one day be possible for a software company to produce an app without writing any code of its own, just by assembling a bunch of microservices created by other companies. In fact, people have theorized about the “one-person unicorn,” meaning a company that is valued at $1 billion or more

…

but is run by one person—a developer whose app sits on top of all those commercial microservices. This hasn’t happened yet, but it might be only a matter of time. The process that we think of as “writing software” might become

…

” of your company might be the way you integrate the pieces, and how well you develop the customer-facing software that sits on top of microservices. Apple’s iPhone contains commodity pieces, like memory chips and flash drives that any company can buy. Apple doesn’t design or build its own

…

they’ve decided to differentiate, and it’s paid off. Just like Apple, software developers now blend their own bespoke software with off-the-shelf microservices provided by others. Uber is a good example. What you think of as “the Uber app” is actually a patchwork of about four thousand

…

microservices—some developed by Uber engineers, and many provided by external cloud platform operators. When a passenger calls a driver, their command zips from the Uber

…

; as far as they’re concerned, Uber is letting them talk to each other. Payments are handled by a different microservice, while currency and exchange rate translations are delivered by a microservice, Tincup, that Uber developed internally. This is how all new companies in Silicon Valley create their software, and it is

…

to pick the right areas to build. Deciding which areas are core competencies—an oft-discussed question of corporate leaders—now extends down to the microservice level. I don’t expect that most corporate executives will gain an expertise or desire to inspect down at this level—but executives should understand

…

valuable areas because common, off-the-shelf building blocks will serve us better. That’s not just an up-front, Build vs. Buy decision. With microservices, the answer to that question is likely to change over time. It’s worth revisiting the question with regularity. Developers are usually the first to

…

, where talent will be successful, and where your customers can trust you. A lot of what we’ve outlined in this book—small teams, platforms, microservices, and so on—are parts of such a system. Another facet of autonomy is influence—inviting developers to key decisions shows you trust them and

…

these team splits at least six months ahead of time. But one upside is that you’re continually investing in your codebase, refactoring it into microservices, and fixing prior debt in the process. It’s like a regular spring clean, which is good hygiene when a team and product are growing

…

first and put the other request on our backlog.” Composable over Monolithic Our software is based on a microservices architecture composed of hundreds of microservices. Each microservice performs a single function or capability. The advantage of microservices is that we can route around or absorb a failure. If one service fails, it won’t

…

the entire Twilio voice system, for example. The services are all loosely coupled. They’re all built by different teams, who can work independently. One microservice might be version one or two, and another might be on version five. But as long as they all “speak” to the API that connects

…

. But there are several default, preconfigured pipelines that teams can start from. These represent the most “paved paths” for standard workflows, such as for websites, microservices, or database clusters. A typical pipeline starts by running unit tests—the most basic kind of code tests that developers write. Then it runs more

…

let developers problem-solve and invent solutions. The crisis also showed how much faster it has become to build and deploy software. Software building blocks, microservices, and APIs have radically accelerated the process. For this we can thank thousands of developers who created those tools. Without all that modern infrastructure, the

Team Topologies: Organizing Business and Technology Teams for Fast Flow

by Matthew Skelton and Manuel Pais  · 16 Sep 2019

2.1: Four Teams Working on a Software System 2.2: Software Architecture from Four-Team Organization 2.3: Microservices Architecture with Independent Services and Data Stores 2.4: Team Design for Microservices Architecture with Independent Services and Data Stores 2.5: Inter-Team Communication 3.1: Scaling Teams Using Dunbar’s

…

structures must match the required software architecture or risk producing unintended designs. A sort of “revival” of Conway’s law took place around 2015, when microservices architectures were on the rise. In particular, James Lewis, Technical Director at Thoughtworks, and others came up with the idea of applying an “inverse Conway

…

teams is fundamentally wrong. This gap between architecture and team structures is visible across all types of architectures, from client-server to SOA and even microservices. Specifically, that is why monoliths need to be broken down (in particular, any indivisible software part that exceeds the cognitive capacity of any one team

…

you might wonder if a law from 1968 about software architecture has stood the test of time. We’ve come a long way after all: microservices, the cloud, containers, serverless. In our experience, such novelties can help teams improve locally, but the larger the organization, the harder it becomes to reap

…

reasons for the predominance of monolithic databases (such as the rise in specialism of people and teams on technical stack layers) up until DevOps and microservices gained traction. Factors such as project orientation, cost cutting via outsourcing, or junior teams without sufficient experience have contributed to the perpetuation of this (now

…

the “natural” software architecture to emerge from the current organization design and communication paths. For example, let’s say that we want to use a microservices architecture for some new cloud-based software systems, where each separate service is independent and has its own data store (Figure 2.3, see page

…

21). Figure 2.3: Microservices Architecture with Independent Services and Data Stores A microservices-based architecture with four separate services, each with its own data store, API layer, and front-end client. By applying the

…

API, and a database developer within the team rather than separate from it (Figure 2.4, see page 22). Figure 2.4: Team Design for Microservices Architecture with Independent Services and Data Stores An organization design that anticipates the homomorphic force behind Conway’s law to help produce a software architecture

…

with four independent microservices. (Again, this is basically the diagram in Figure 2.3 rotated ninety degrees.) According to Conway’s law, this team design will most “naturally” produce

…

’s number. Expect to change the architecture of software systems to fit with the limits on human interactions set by Dunbar’s number. Approaches like microservices can help if applied with a team-first perspective. Work Flows to Long-Lived Teams Teams take time to form and be effective. Typically, a

…

is misguided. Computer researcher Graylin Jay and colleagues found in 2009 that some programming languages are more verbose than others (and after the emergence of microservices, polyglot systems became increasingly more common), and teams using more abstractions and reusing code will have smaller but not necessarily simpler codebases.25 When measuring

…

design the system and its software boundaries to fit the available cognitive load within delivery teams. Instead of choosing between a monolithic architecture or a microservices architecture, design the software to fit the maximum team cognitive load. Only then can we hope to achieve sustainable, safe, rapid software delivery. This team

…

their contributions are not limited to it. There is very little coordination required between service teams, leading to a highly distributed and heterogeneous stack of microservices. Interestingly, there is an exception for testing, as software development engineers in testing (SDETs) work across the whole organization, looking to promote good testing practices

…

team were not going to scale. We put DevOps and continuous delivery practices at the center of our design choices and started transitioning to a microservices-type architecture from our existing (successful) monolithic system. We began by adopting a stronger focus on “the team” as the means to get work done

…

the system architecture is already modular (for example, shared databases, coupled builds and/or releases, and more). As Amy Phillips puts it, “If you have microservices but you wait and do end-to-end testing of a combination of them before a release, what you have is a distributed monolith.”10

…

mental overheads. This “team-sized architecture” focuses on people first, and is a more sustainable and humane approach to software architecture than either monolithic or microservices architectures, both of which focus on the technology first. Strategic Application of Conway’s Law Back in 1968, Mel Conway offered superb insight into the

…

are doomed to fail unless the designs align with the way in which the teams communicate. Approaches to building and operating cloud software such as microservices accommodate the need for human-sized chunks of functionality alongside enhanced deployability. Teams have a greater chance of innovating and supporting a system if they

…

Engineering: How Google Runs Production Systems. Sebastopol, CA: O’Reilly, 2016. Blalock, Micah. “Of Mustard Seeds and Microservices.” Credera (blog), May 6, 2015. https://www.credera.com/blog/technology-insights/java/mustard-seeds-microservices/. Bosch, Jan. “On the Development of Software Product-Family Components.” In Software Product Lines, edited by Robert L

…

Design.” Applied Ergonomics 31 no. 5 (2000): 463–477. https://doi.org/10.1016/S0003-6870(00)00009-0. Cockcroft, Adrian. “Goto Berlin—Migrating to Microservices (Fast Delivery).” Presented at the GOTO Berlin conference, Berlin, November 15, 2014. http://www.slideshare.net/adriancockcroft/goto-berlin. Cohn, Mike. “Nine Questions To Assess

…

Practice: Implications for Information Systems’ Implementation and Use.” MIS Quarterly 29 no. 2 (June 2005): 335–363. https://papers.ssrn.com/abstract=1276022. Lewis, James. “Microservices and the Inverse Conway Manoeuvre—James Lewis.” YouTube video, 57:57, posted by NDC Conferences, February 16, 2017. https://www.youtube.com/watch?v=uamh7xppO3E

…

a System. Hartland, VT: Sustainability Institute, 1999. http://donellameadows.org/wp-content/userfiles/Leverage_Points.pdf. “Microservices: Organizing Large Teams for Rapid Delivery.” SlideShare, posted by Pivotal, August 10, 2016. https://www.slideshare.net/Pivotal/microservices-organizing-large-teams-for-rapid-delivery. Mihaljov, Timo. “Having a Dedicated DevOps Person Who Does All

…

: Conflux Digital, 2019. Morris, Kief. Infrastructure as Code: Managing Servers in the Cloud. Sebastopol, CA: O’Reilly Media, 2016. Munns, Chris. “Chris Munns, DevOps @ Amazon: Microservices, 2 Pizza Teams, & 50 Million Deploys per Year.” SlideShare.net, posted by TriNimbus, May 6, 2016. http://www.slideshare.net/TriNimbus/chris-munns-devops-amazon

…

-microservices-2-pizza-teams-50-million-deploys-a-year. Murphy, Niall. “What is ‘Site Reliability Engineering’?” Landing.Google.com, https://landing.google.com/sre/interview/ben-

…

. Netflix Technology Blog. “The Netflix Simian Army.” Netflix TechBlog, July 19, 2011. https://medium.com/netflix-techblog/the-netflix-simian-army-16e57fbab116. Newman, Sam. Building Microservices: Design Fine-Grained Systems. Sebastopol, CA: O’Reilly Media, 2015. Newman, Sam. “Demystifying Conway’s Law.” ThoughtWorks (blog) June 30, 2014. https://www.thoughtworks.com

…

Invent?”; Kelly, “Conway’s Law & Continuous Delivery.” 8. Kelly, “Conway’s Law v. Software Architecture.” 9. Raymond, The New Hacker’s Dictionary, 124. 10. Lewis, “Microservices and the Inverse Conway.” 11. Pink, Drive, 49. Chapter 2 1. “DevOps Over Coffee – Adidas;” Fernando Cornago, person email communication with the authors, March 2019

…

of Social Networks;” Burgess, Thinking in Promises, 87. 7. Snowden, “The Rule of 5, 15 & 150;” Karlgaard and Malone, Team Genius, 201–205. 8. Lewis, “Microservices and the Inverse Conway Manoeuvre.” 9. Munns, “Chris Munns, DevOps @ Amazon.” 10. Brooks, The Mythical Man-Month. 11. Tuckman, “Developmental Sequence in Small Groups,” 384

Chaos Engineering: System Resiliency in Practice

by Casey Rosenthal and Nora Jones  · 27 Apr 2020  · 419pp  · 102,488 words

would not expect a reasonable engineering team to anticipate the undesirable interaction in advance. Example 1: Mismatch Between Business Logic and Application Logic Consider the microservice architecture described here and illustrated in Figure 1-1. In this system, we have four components: Service P Stores personalized information. An ID represents a

…

a columnar storage system like Cassandra or DynamoDB. Service T An in-memory cache, perhaps something like Redis or Memcached. Figure 1-1. Diagram of microservice components showing flow of requests coming in to P and proceeding through storage To add some rational fallbacks to this system, the teams responsible for

…

an example, several layers of virtualization have become the norm from cloud instances to containers to functions as a service. With a general migration toward microservice architectures, the industry seems to have acted against its best interest in terms of managing complexity by purposefully increasing the Relationships between parts within a

…

and apply them to our own. In that spirit, we can explore Chaos Engineering in industries and companies that look very different from the prototypical microservice-at-scale examples commonly associated with the practice. FinTech, Autonomous Vehicles (AV), and Adversarial Machine Learning can teach us about the potential and the limitations

…

’s robustness against data corruption is another important aspect of data integrity. Do you thoroughly fuzz test8 even internal services? Fuzzing API endpoints in a microservice architecture is an outstanding way to harden services against unanticipated inputs that lead to crashes (“queries of death”). There are a host of open source

…

value of your dependencies, you might not be able to see the full picture of end-to-end customer experience. With the current rise of microservice architectures often forming dependency clouds1 too complex for a human to understand, it might be a futile effort to understand all potential variations of dependencies

…

few of the factors that need to be considered. Capital One has also built its own tool for Chaos Engineering. An all-in approach to microservices architecture required a higher level of enterprise-wide resiliency than that offered in third-party tools. Resilience is a necessary core competency for Capital One

…

to the developer’s laptop. Along with her team, she creates strategies, roadmaps, and platforms to test and improve the reliability of Capital One’s microservices. 1 A neobank is a type of direct bank that is entirely digital and serves its customers only through mobile apps and computer platforms 2

…

drawn to technical solutions for human problems. Technology has made a massive impact on our lives. It’s in everything we do. The “monolith-to-microservices” stories are a dime a dozen, and many of those journeys are centered around solving very real organizational scaling problems. But changing the technology we

…

wearing the Pain Suit for a few short days. You wake up one morning and feel a pain in your shoulder. “Of course,” you think. “Microservice X is misbehaving again.” It would not take you long to get a visceral sense of the holistic state of the system. Very quickly, you

…

was published in 2010. By 2012, engineers at Twitter had implemented an open source version called Zipkin; by 2015 Zipkin clones had proliferated to many microservice-based organizations (e.g., Salp at Netflix, Jaeger at Uber). Today, OpenTracing provides an open standard for call graph tracing. Among organizations that have call

…

the advanced principles highlighted throughout this book as well as the feedback loop we expect in CV. ChAP is fully automated. It introspects the microservice architecture, selects microservices for inspection, constructs experiments for that microservice, and runs the experiment during business hours. The experiment hypotheses take the following form: under conditions X for

…

microservice Y, customers still watch a normal amount of video streams. X is usually an amount of downstream latency, and normal is defined by a control

…

or not the system owner has things to fix before they can safely run a chaos experiment. If it is safe to fail, then that microservice may be added to the list and prioritized for experimentation. ChAP: Running Experiments Through integration with Netflix’s CD tool Spinnaker, ChAP is able to

…

introspect the code deployed for the chosen microservice in the prioritization process provided by Monocle. For this microservice, there are a number of instances running in a cluster. ChAP will spin up two additional instances. One acts as

…

requests are routed to the experiment. The control and variable instances are torn down. Most importantly, the team responsible for that microservice is notified that under conditions X for their microservice, customers have a bad time. This discovery informs the team about their own safety margin and it is up to them

…

the service and the business value. Customers who watch a lot of Netflix tend to recommend it to others. The hypothesis, “Under conditions X for microservice Y, customers still watch a normal amount of video streams,” captures this notion of the desired steady-state behavior. Vary real-world events Most incidents

…

utilization pattern in real time based on observations of real production traffic. Not many of them test the performance of subsystems, or squeeze test individual microservices to establish provisioning and capacity data. Smarter tools are on the horizon, and will give operators more information about their systems. Data Artifacts Databases and

…

are impossible for the human mind to mentally model. Our systems are now vastly distributed and operationally ephemeral. Transformational technology shifts such as cloud computing, microservices, and continuous delivery (CD) have each brought forth new advances in customer value but have in turn resulted in a new series of future challenges

…

a particular state in mind (i.e., production release on Day 0). Meanwhile, the system ecosystem that surrounds these controls is changing rapidly every day. Microservices, machines, and other components are spinning up and spinning down. Component changes are occurring multiple times a day through continuous delivery. External APIs are constantly

Hands-On RESTful API Design Patterns and Best Practices

by Harihara Subramanian  · 31 Jan 2019  · 422pp  · 86,414 words

implementations. He has been an active contributor to various online and offline forums in different technologies, and focuses his time on technology consulting, software development, microservices architecture (MSA), Service-oriented architecture (SOA), and more. Pethuru Raj (PhD) works as the chief architect at the Site Reliability Engineering Center of Excellence,

…

paradigm REST architectural constraints The advantages of REST Self-descriptive messages SOAP versus REST When to use REST versus SOAP Best practices for REST-based microservices The API-first approach Developing API-first Building services API-first Summary Further reading Frameworks, Standard Languages, and Toolkits Technical requirements Core features of

…

modernization Service extraction&#xA0; Service composition&#xA0; Service migration Container-centric legacy application modernization Refactoring and rewriting Modernization technique terms Legacy modernization through microservices The distinctions of microservices The code samples The major hurdles to overcome Modernizing and migrating legacy applications&#xA0;&#x2013; the role of cloud environments The need for cloud

…

this chapter has a reference table for various frameworks and their supported languages, along with their prominent features. Chapter 11, Legacy Modernization to Microservices-Centric Apps, discusses how microservices architectures (MSAs) are the way forward for modern applications that are highly nimble, versatile, and resilient. This chapter provides the reasons for legacy

…

client-server concepts. However, we strongly encourage our readers to refresh their knowledge of design principles such as SOLID, OO designs, enterprise integration, SOA, and microservice architecture fundamentals. Learning about REST API and its importance In the first chapter, we learned about RESTful principles such as client-server, statelessness, cacheable, and

…

through the smart leveraging of an API gateway infrastructure. Technical requirements This chapter mainly talks about the crucial contributions of API gateway solutions to empower microservices to be the right and rewarding ingredient for producing enterprise-scale, mission-critical, cloud-hosted, service-oriented, event-driven, innovation-filled, process-aware, production-

…

with the business to deal with changes in an agile fashion, match business changes with an agile response, and deliver solutions in a decentralized manner. Microservices are independently deployable, horizontally scalable, composable, interoperable, publicly discoverable, network accessible, technology-agnostic, modular (loosely coupled and highly cohesive), fine-grained, and so now. The

…

container orchestration and management solutions in order to form container/service clusters to realize multi-container, business-aware, mission-critical, adaptive, and composite applications. With microservices emerging and becoming established as the simple and optimal building block for designing, developing, deploying, and managing production-grade business applications and IT platforms, existing

…

directly, the security of services is greatly strengthened. Through using an API gateway, it is possible to stop all kinds of malicious attacks on internal microservices. Every microservice has its own data store. Thus, not only service security but also data security is enabled. Through rate limiting/throttling offered by API gateways

…

impenetrable data security. There are several mechanisms such as encryption and decryption, digital signature, hashing for securing data while in transit, persistence, and usage. For microservice-centric applications running on cloud infrastructures, the security aspect starts with identification, authentication, and authorization. Security policies are another solution widely used in public cloud

…

the resiliency and reliability design patterns such as circuit breakers, timeouts, retries, and health checks are intrinsically implemented and incorporated into service mesh solutions. Summary Microservices are being proclaimed as a groundbreaking architectural style for producing and sustaining business and IT applications and platforms. Cloud environments are filled up with bare

…

and inspiring thing for business and IT organizations across the globe. Currently-running web, cloud, and enterprise applications are being redesigned, refactored, and remedied using microservices in order to be flexible and futuristic. Further on, all kinds of devices, such as mobile devices (cellphones, smartphones, digital assistants, and tablets), handhelds,

…

be publicly-discoverable, network-accessible, interoperable, and composable. Hence, the concept of MSA has started to flourish with the proper nourishment from worldwide IT experts. Microservices are fine-grained, horizontally-scalable, independently-deployable, API-driven, usable and reusable, portable, and technology-agnostic. With the containerization movement picking up steadily, it's

…

easy to have containerized microservices, the images of which can be downloaded and committed to be executable insight containers, which emerge as the most optimal runtime platform. As indicated earlier

…

applications are becoming cloud-enabled through the leverage of MSA. With the pervasiveness of cloud centers, software engineers and architects are designing, developing, and deploying microservices-centric applications in cloud environments. This scenario is being touted as cloud-native applications. Precisely speaking, in order to decisively reap all of the originally

…

operational, and analytical applications are being built as cloud-native applications. The growing ecosystem of IoT device services We talked about business-specific and agnostic microservices, which are increasingly containerized. The container images of any software can be uniformly packaged, quickly shipped, and easily deployed across any software platforms to run

…

among different and distributed devices. That is, every device is being projected as a findable, flexible, and usable service. The robustness, versatility, and resiliency of microservices come in handy when presenting devices as services to the outside world, which eliminates all kinds of dependency-induced challenges. The changing application ecosystem Mainframe

…

the highest recognition for producing smarter applications towards user-empowerment. Hence, to realize all kinds of smart applications, API-stuffed, extensible, configurable, interoperable, and composable microservices are the need of the hour. Composition tools and techniques are flourishing, and hence service composites are the best fit for constructing insight-driven, highly

…

about service-composition methods Orchestration and choreography are the well-known and widely-used ways of accomplishing composite services. With the containerization movement picking up, microservices are being containerized and run in cloud environments. With the adoption of container-orchestration platforms, multi-container applications are being easily and quickly realized. Hence

…

services join. Hence, experts have come out with competent alternatives that solve these bad implications by invoking tightly-coupled services. Service orchestration and choreography Orchestration microservices have to be composed through an orchestration engine. The composition has to happen through predefined patterns, which are typically described using an orchestration language. Orchestration

…

core architectural style and pattern for the increasingly event-driven world. Choreography is more suitable for implementing event-driven applications. Due to their decoupled nature, microservices can be replaced and substituted with better service implementations. Additional services and their instances can be incorporated without affecting the system. New technologies and algorithms

…

orchestration. Synchronous communication blocks other service requests. The following diagram illustrates how the orchestration process gets accomplished through the participation of disparate and geographically-distributed microservices: This composition model typically doesn't address failure cases. Most of the time, the service request is being attended. The failure rate is mostly one

…

size: Data size is varies considerably these days. There are object-storage options that accommodate staggering amounts of data. Hence, data management in the ensuring microservices era is quite different and challenging too. However, data professionals have come out with competent data persistence, representation, interchange, masking, wrangling, management, analytics, and

…

security solutions. Summary Microservices have to be composed through the orchestration engine. The composition has to happen through predefined patterns, which are typically described using an orchestration language. Orchestration

…

the most optimized and organized application architecture, most business-critical applications are being meticulously designed, developed, and deployed as a set of independent, yet interactive, microservices. Another noteworthy trend is the tremendous success of REST APIs for services and applications. All kinds of applications (operational, transactional, and analytical) are being fitted

…

the prime architectural pattern and style for producing multifaceted and enterprise-scale applications to be elegantly hosted and run on cloud environments (local and remote). Microservices are lightweight, simple to build and deploy, self-defined, fine-grained, and network accessible. They also follow the single functionality principle. Nowadays, everything that

…

the emerging application types The REST paradigm for application modernization and integration The RESTful services for digital transformation and intelligence Best practices for REST-based microservices Technical requirements There are no special technical requirements for this chapter as it doesn't talk about designing, developing, or deploying a software package.

…

, massive applications are being partitioned into a number of fine-grained, publicly-discoverable, network-accessible, interoperable, API-enabled, composable, portable, horizontally-scalable, and independently-deployable microservices. There are application-modernization and -migration toolkits to move refactored and remedied applications to cloud environments in order to reap all the originally envisaged benefits

…

operation or code of other services. Any kind of service failure can be gracefully handled. These need to be carefully considered when designing and developing microservices-based enterprise applications. As technologies become increasingly complex, best practices and procedures sourced through various experimentation come in handy for architects and developers to create

…

not only your prototyping but also production-grade RESTful applications. In the next chapter, we'll explore best practices for migrating legacy applications to capable microservices. Further reading Building RESTful Python Web Services by Gastón C. Hillar, https://www.packtpub.com/application-development/building-restful-python-web-services October 2016 Building

…

needs for legacy application modernization Delineating why applications have to be modernized to be migrated and run in cloud environments Depicting how the combination of microservices and containers is the preferred one for legacy modernization Detailing the legacy modernization methodology Technical requirements Readers should be comfortable with the following popular technologies

…

and independent processes that communicate with one other using well-defined and language-agnostic APIs. They are usable, reusable, interoperable, dynamically-composable, and self-defined. Microservices expose their unique and business-centric capabilities through well-intended interfaces to the outside world. Thus software integration gets simplified and sped up in a

…

technologies. Thus, not only application development, but also the maintenance aspect are both worrisome factors for worldwide enterprises. These specialized functionalities can be built as microservices and then exposed so they can be easily found and bound in software applications. Why legacy modernization? Legacy software applications were written decades ago using

…

any problem and to reap all the originally envisaged benefits of the cloud. The emergence of MSA can help to accelerate legacy modernization. Microservices-centric legacy application modernization Microservices are fine-grained, self-defined, and autonomous services that produce and sustain next-generation, enterprise-grade applications. With MSA, every application is

…

subdivide the legacy application into multiple components, where each component caters to a business functionality. Now every business functionality can be converted into a microservice. Thus, there are microservices-centric and container-centric methods for effecting legacy modernization and migration to cloud environments. Any application can be directly containerized, but this transition

…

containers and hence there will be many containers in a typical IT environment; that is, for an application, there can be multiple interactive and collaborative microservices. Each microservice can be run in multiple containers in order to support redundancy. Widely-demanded high availability can be achieved through multiple containers for a single

…

automated tools to augment the various activities associated with cloud orchestration, realizing multi-cloud container applications, and infrastructure as code. The convergence of containers and microservices opens up a growing array of innovations, disruptions, and transformations. There are integrated platforms that enable the dream service era. Application components have to be

…

refurbished using the latest technologies, so they can be hosted and managed in cloud environments. That is, massive applications are partitioned into a pool of microservices, then they are containerized, and stocked in container-image directories. Container orchestration platforms then provision container resources, fetch container images, and deploy them. The

…

modernization. This offering automates and accelerates modernizing legacy applications. It can do both reverse and forward engineering. That is, it can translate legacy applications into microservices-centric applications. Readers can find many case studies and references for automated application modernization at https://www.bluage.com/references. We all know that there

Architecting For Scale

by Lee Atchison  · 25 Jul 2016  · 255pp  · 55,018 words

, and they all go hand in hand. Thus, to properly discuss scale, we must also consider availability, risk management, and modern architecture paradigms such as microservices and cloud computing. As such, this book is organized as follows: Part I, “Availability” Availability and availability management are often the first areas that are

…

In this chapter, I give suggestions on how to reduce risk within your applications and build applications with lower risk. Part III, “Services and Microservices” Services and microservices are an architecture strategy for building larger and more complicated applications that need to operate at higher scale. Chapter 11, Why Use Services? This

…

chapter explores why services are important to building highly scalable applications. Chapter 12, Using Microservices Here, I provide an introduction on creating microservice-based architectures, focusing on sizing of services and determining where service boundaries should be created in order to improve scaling and availability

…

the space shuttle), it often is overkill and results in increased complexity, as well. Increased complexity means increased risk. Another example is overtly separated activities. Microservices are a great model to impove the quality of your application and hence reduce risk. Chapter 12 contains more information on using services and

…

microservices. However, if taken to an extreme, building your systems into too finely decomposed microservices can result in an overall increase in application complexity, which increases risk. Independence Multiple components utilizing shared capabilities

…

maintainable, secure, and low risk. One common place where modern software construction principles tend to increase complexity more than perhaps is necessary is in microservice-based architectures. Microservice-based architectures reduce the complexity of individual components substantially, making it possible for the individual services to be easily understood and built using simpler

…

techniques and designs. However, although they reduce the complexity of the individual microservice, they increase the number of independent modules (microservices) necessary to build a large-scale application. By having a larger number of independent modules working together, you increase the interdependence

…

on the modules, and increase the overall complexity of the application. It is important as you build your microservice-based application that you manage the trade-off between simpler individual services and more complex overall system design. Self-Repair Building self-righting and self

…

control over who and how the operation is performed. 1 O’Reilly has many titles available about security and web security. Part III. Services and Microservices A service is a distinct enclosed system that provides business functionality in support of building one or more larger products. Chapter 11. Why Use Services

…

. Provide ownership of scaling decisions back to the team that knows the most about the needs of the service (the service owner). Chapter 12. Using Microservices A service provides some capabilities that are needed by the rest of the application. Example include billing services (which offer the component that bills customers

…

one that does not have a single correct answer. Some companies that “service-ize” split their application into many (hundreds or thousands) of very tiny microservices. Others split their application into only a handful of larger services. There is no right answer to this problem. However, the industry is trending toward

…

smaller microservices, and more of them. Technologies such as Docker have made these larger number of microservices a viable system topology. We use the term services and microservices interchangeably in this book. Dividing into Services So, how do you decide where

…

become trivially simple, but the overall application becomes overly complicated with complex interactions between the services. I’ve actually heard of an example application utilizing microservices that defined a “Yes” service and a “No” service that simply returned those boolean results—this is extreme taken to extreme. It would be great

…

. This example “most certainly” goes too far in splitting things up. Chapter 13. Dealing with Service Failures One of the vulnerabilities in building a large microservice-based application is dealing with service failures. The more services you have, the greater the likelihood of a service failing, and the larger number of

…

application and organization? To be STOSA, you must meet the following criteria: Have an application that is constructed using a service-based architecture or a microservice-based architecture. Have multiple development teams that are responsible for building and maintaining the application. All services in your application must be assigned to a

…

traffic and data increases, these problems simply become worse. Before doing anything else, get your availability and risk management in order. Microservices In Part III, we discussed service- and microservice-oriented architectures. Although there are many different architectural decisions you need to make and architectural directions you need to set, make the

…

changes the cloud is bringing to us and our applications? The following sections outline some key changes. Acceptance of Microservice-Based Architectures As we have discussed in this book, service- and microservice-based architectures have grown in popularity in recent years. Migrating applications to some form of a service-based architecture is

…

strategy. This modernization strategy includes moving to state-of-the-art application architectures. In recent years, this state-of-the-art application architecture involves using microservices and other service-based architectures as part of that strategy. This is because technologies such as Docker have made

…

application development. Realizing this, cloud providers have begun to provide higher-value managed offerings, such as the EC2 Container Service, for use in managing our microservice-based containers. Smaller, More Specialized Services As we modernize our applications and move them to the cloud, we begin looking at cloud services and how

…

a dynamic container environment, an application, wrapped in a container, is designed to be easily started, stopped, and relocated quickly and easily. Combined with a microservice-based application architecture, dynamic containers can provide a highly dynamic and fluid application that can be scaled easily and effectively, and existing compute resources can

…

in container scheduling and management has not yet been realized. Optimized Use Cases Dynamic containers are great for easy deployment of complex applications in a microservice-based architecture. They also are great for rapidly starting and stopping service instances (container start/stop is very similar in scope to starting/stopping a

…

, when taken together, are designed to help you scale your applications: Availability and availability management Risk and risk management Building applications by using services and microservices Scaling your application and your development team to work on your application Using the cloud to help scale your application Availability Availability is the ability

…

for availability reasons, Maintaining Location Diversity for Availability Reasons managed infrastructure, Managed Infrastructure-Monitoring and CloudWatch micro startups, The Micro Startup microcomputing, Microcompute microservice-based architectures, Acceptance of Microservice-Based Architectures monitoring, Monitoring and CloudWatch non-server-based managed resource, Managed Resource (Non-server-based) raw resource, Raw Resource-Raw Resource resource

…

partitioning, Data Partitioning-Data Partitioning importance of, The Importance of Continuous Improvement regular examination of application, Examine Your Application Regularly service ownership, Service Ownership services, Microservices stateless services, Stateless Services contracts, What are Service-Level Agreements?(see also Service Level Agreements (SLAs)) in service-oriented architecture, The Ownership Benefit SLAs and

…

) management, sharing risk matrix with, Maintaining the Risk Matrix micro startups, The Micro Startup microcomputing, Microcompute(see also AWS Lambda) microservices, Using Microservices-The Right Balance(see also services) cloud, Acceptance of Microservice-Based Architectures complexity and, Simplicity redundancy and, Redundancy Improvements That Increase Complexity mitigation plans, Focus #3: Mitigate Risk, Mitigation Plan

…

SLAs) services, Servicesbalance in number of, The Right Balance benefits, The Service-Based Application creating excessive boundaries, Going Too Far criteria for, Using Microservices defined, Services and Microservices failure of (see service failures) guidelines for separating applications into, What Should Be a Service?-Mixed Reasons monolithic application vs., The Monolith Application preparing

…

for scaling, Microservices reasons for using, Why Use Services?-The Scaling Benefit scaling benefits, The Scaling Benefit service boundaries for, What Should Be a Service?-Mixed Reasons service

…

-based application, The Service-Based Application-The Service-Based Application stateless, Stateless Services using, Using Microservices-The Right Balance severity, riskchanges in risk matrix reviews, Review Regularly, Maintaining the Risk Matrix likelihood vs., Likelihood Versus Severity-T-Shirt Photos: High Likelihood

Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps

by Valliappa Lakshmanan, Sara Robinson and Michael Munn  · 31 Oct 2020

it to make predictions. We use the term serving to refer to accepting incoming requests and sending back predictions by deploying the model as a microservice. The serving infrastructure could be in the cloud, on-premises, or on-device. The process of sending new data to your model and making use

…

out the linear model, containerized it, and deployed the container image into a serving platform. Prediction library Instead of deploying the serving function as a microservice that can be invoked via a REST API, it is possible to implement the prediction code as a library function. The library function would load

…

developers who need to predict with the library can then include the library with their applications. A library function is a better alternative than a microservice if the model cannot be called over a network either because of physical reasons (there is no network connectivity) or because of performance constraints. The

…

model will have to be updated to use the new version of the library. The more commonly a model is updated, the more attractive a microservices approach becomes. A secondary drawback is that the library approach is restricted to programming languages for which libraries are written, whereas the REST API approach

…

individual request synchronously and as quickly as possible, as discussed in “Design Pattern 16: Stateless Serving Function”. The serving infrastructure is usually designed as a microservice that offloads the heavy computation (such as with deep convolutional neural networks) to high-performance hardware such as tensor processing units (TPUs) or graphics processing

…

parts of the application. In recent years, monolithic apps have been replaced in favor of a microservices architecture where individual pieces of business logic are built and deployed as isolated (micro) packages of code. With microservices, a large application is split into smaller, more manageable parts so that developers can build, debug

…

, and deploy pieces of an application independently. This monolith-versus-microservice discussion provides a good analogy for scaling ML workflows, enabling collaboration, and ensuring ML steps are reproducible and reusable across different workflows. When someone is

…

this capability today. Design Pattern 27: Model Versioning In the Model Versioning design pattern, backward compatibility is achieved by deploying a changed model as a microservice with a different REST endpoint. This is a necessary prerequisite for many of the other patterns discussed in this chapter. Problem As we’ve seen

…

, the Model Versioning design pattern makes it possible to perform A/B testing. Additionally, with model versioning, each deployed version of our model is a microservice—thus decoupling changes to our model from our application frontend. To add support for a new version, our team’s application developers only need to

…

different machine learning jobs. Lastly, we looked at the Model Versioning design pattern, where backward compatibility is achieved by deploying a changed model as a microservice with a different REST endpoint. In the next chapter, we look into design patterns that help carry out AI responsibly. 1 Note that the overall

…

split test model changes while having a single model in production or to update models without breaking existing users. Deploy a changed model as a microservice with a different REST endpoint to achieve backward compatibility for deployed models. Responsible AI Heuristic Benchmark Explaining model performance using complicated evaluation metrics does not

…

Patterns MD5 hash, Cryptographic hash mean absolute error (MAE), Problem mean average precision (MAP), Problem Mesh TensorFlow, Model parallelism mesh-free approximation, Data-driven discretizations microservices architecture, Problem min-max scaling, Linear scaling-Linear scaling Mirrored Variable, Synchronous training MirroredStrategy, Synchronous training, Synchronous training Mixed Input Representation, Deterministic inputs ML approximation

API Marketplace Engineering: Design, Build, and Run a Platform for External Developers

by Rennay Dorasamy  · 2 Dec 2021  · 328pp  · 77,877 words

Industry 4.0: The Industrial Internet of Things

by Alasdair Gilchrist  · 27 Jun 2016

Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems

by Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski and Adam Stubblefield  · 29 Mar 2020  · 1,380pp  · 190,710 words

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann  · 16 Mar 2017  · 1,237pp  · 227,370 words

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann  · 17 Apr 2017

Docker Deep Dive

by Nigel Poulton  · 10 May 2020

Whistleblower: My Journey to Silicon Valley and Fight for Justice at Uber

by Susan Fowler  · 18 Feb 2020  · 205pp  · 71,872 words

The Nature of Software Development: Keep It Simple, Make It Valuable, Build It Piece by Piece

by Ron Jeffries  · 14 Aug 2015  · 444pp  · 118,393 words

Terraform: Up and Running: Writing Infrastructure as Code

by Yevgeniy Brikman  · 13 Mar 2017

WTF?: What's the Future and Why It's Up to Us

by Tim O'Reilly  · 9 Oct 2017  · 561pp  · 157,589 words

Architecting Modern Data Platforms: A Guide to Enterprise Hadoop at Scale

by Jan Kunigk, Ian Buss, Paul Wilkinson and Lars George  · 8 Jan 2019  · 1,409pp  · 205,237 words

Working Backwards: Insights, Stories, and Secrets From Inside Amazon

by Colin Bryar and Bill Carr  · 9 Feb 2021  · 302pp  · 100,493 words

Extremely Hardcore: Inside Elon Musk's Twitter

by Zoë Schiffer  · 13 Feb 2024  · 343pp  · 92,693 words

Docker: Up & Running: Shipping Reliable Containers in Production

by Sean P. Kane and Karl Matthias  · 15 Mar 2018  · 350pp  · 114,454 words

Site Reliability Engineering: How Google Runs Production Systems

by Betsy Beyer, Chris Jones, Jennifer Petoff and Niall Richard Murphy  · 15 Apr 2016  · 719pp  · 181,090 words

Designing Reactive Systems: The Role of Actors in Distributed Architecture

by Hugh McKee  · 5 Sep 2016  · 31pp  · 9,168 words

Clean Agile: Back to Basics

by Robert C. Martin  · 13 Oct 2019  · 333pp  · 64,581 words

Kill It With Fire: Manage Aging Computer Systems

by Marianne Bellotti  · 17 Mar 2021  · 232pp  · 71,237 words

Ansible for DevOps: Server and Configuration Management for Humans

by Jeff Geerling  · 9 Oct 2015  · 313pp  · 75,583 words

Essential Sqlalchemy

by Jason Myers and Rick Copeland  · 27 Nov 2015  · 458pp  · 46,761 words

The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

by Stephen Witt  · 8 Apr 2025  · 260pp  · 82,629 words

The Art of Monitoring

by James Turnbull  · 1 Dec 2014  · 514pp  · 111,012 words

Docker: Up & Running: Shipping Reliable Containers in Production

by Sean Kane and Karl Matthias  · 14 May 2023  · 433pp  · 130,334 words

An Elegant Puzzle: Systems of Engineering Management

by Will Larson  · 19 May 2019  · 227pp  · 63,186 words

API Design Patterns

by Jj Geewax  · 19 Jul 2021  · 725pp  · 168,262 words

The Docker Book

by James Turnbull  · 13 Jul 2014  · 265pp  · 60,880 words

Docker in Action

by Jeff Nickoloff and Stephen Kuenzli  · 10 Dec 2019  · 629pp  · 109,663 words

The Manager’s Path

by Camille Fournier  · 7 Mar 2017

Software Engineering at Google: Lessons Learned From Programming Over Time

by Titus Winters, Tom Manshreck and Hyrum Wright  · 17 Mar 2020  · 214pp  · 31,751 words

Making Work Visible: Exposing Time Theft to Optimize Workflow

by Dominica Degrandis and Tonianne Demaria  · 14 May 2017  · 153pp  · 45,721 words

Forge Your Future with Open Source

by VM (Vicky) Brasseur  · 266pp  · 79,297 words

Coders: The Making of a New Tribe and the Remaking of the World

by Clive Thompson  · 26 Mar 2019  · 499pp  · 144,278 words

Brotopia: Breaking Up the Boys' Club of Silicon Valley

by Emily Chang  · 6 Feb 2018  · 334pp  · 104,382 words

Modern Vim: Craft Your Development Environment With Vim 8 and Neovim

by Drew Neil  · 2 May 2018  · 241pp  · 43,252 words

Demystifying Smart Cities

by Anders Lisdorf