Pivotal + VMware: Transforming how more of the world builds software

API Gateway Pattern

Demonstrate how to use the API Gateway pattern to control the authentication and access to the backend resources using Spring Cloud.

Context

In a microservice-based architecture, you will mostly likely have multiple types of clients communicating with your backend. The backend is comprised of many different services that could be exposed over various protocols (e.g, soap+xml, rest+http, jms, amqp, websockets, etc) and so clients will need to collect data that is spread over multiple services. In some cases, these services may come from legacy applications and each application may have its own way of handling security.

Problem

How do clients minimize the number of requests to the backend to reduce the chattiness between the clients and the services? And how do clients communicate securely with multiple services that may have been originally designed with different types of clients in mind?

Forces

In most microservice implementations, internal microservice endpoints are not exposed outside. They are kept as private services. A set of public services will be exposed to the clients using an API gateway. There are many reasons to do this:

  1. Only a selected set of microservices are required by the clients.
  2. It is hard to implement client-specific transformations at the service endpoint.
  3. If there is data aggregation required, especially to avoid multiple client calls in a bandwidth-restricted environment, then a gateway is required in the middle.
  4. The number of service instances and their locations (host+port) changes dynamically
  5. If there are client-specific policies to be applied, it is easy to apply them in a single place rather than in multiple places. An example of such a scenario is the cross-origin access policy.

Solution

The API Gateway pattern offers client specific APIs (e.g. mobile, tablet, etc.) that offer protocol translation between a web friendly client front-end and back-end services, such as a message queue or database, and allow a single point of entry (and control) into the set of exposed services. The client only has to know the URL of one server, and the backend can be refactored at will with no change to the client, which is a significant advantage. There are other advantages in terms of centralization and control: rate limiting, authentication, auditing and logging. The API Gateway is responsible for aggregating data or, in some cases, acting as a simple routing layer to deal with canary based deployments or A/B testing scenarios.

Implementing the API Gateway pattern is really simple with Spring Cloud and Zuul. Zuul comes from the Netflix family of microservice products for which Spring Cloud provides many enhancements for developers using Spring. Unlike many enterprise API gateway products, Zuul provides complete control for the developers to configure or program based on specific requirements.

The Zuul proxy internally uses the Eureka server for service discovery, and Ribbon for load balancing between service instances. The Zuul proxy is also capable of routing, monitoring, managing resiliency, security, and so on. In simple terms, you can consider Zuul a reverse proxy. However, as demonstrated in the referenced example, you can also build a sophisticated edge service that can modify the results of the proxied services by overriding them at the API layer.

Example: Rate limiting

Using a rate limiter to throttle traffic is a common use-case for preventing a system from getting overloaded with too many requests all at once (e.g. avoiding a denial of service). You can place this limiter in the API Gateway so it throttles traffic as it comes in -- before it hits the backend. In this example, we want to restrict the rate at which our API is accessed to roughly 100 requests per second. (see ThrottlingConfiguration)

see Throttling ZuulFilter

This means we can acquire up to 100 permits during one second and if we try to do it more often tryAcquire() will return false (or thread will block if acquire() is used instead1). So the code above guarantees that the listener won't be called more that two times per second.

Example: Aggregation

To demonstrate this concept, we've implemented a couple of microservices that we are going to use as the basis for aggregation: signage and bays. Each of these services belong to their own domain and expose a unique set of public APIs. Since these belong to separate domain classifications potentially worked on by different teams, each one is checked in under a separate git repository. And each service in the domain is deployed as a separate application.

In this hypothetical example, we want to print labels for a set of skus located in bays at the store. The bays API has an endpoint for retrieving a list of bays for a given store and sku. After we call this service, we need to iterate through the list of bays and create the labels. The signage API has a label service endpoint to create labels for a given store. So we call this service to get back a list of labels that need to be printed in the store. Now if we want to print the labels, we need to send another request to the signage API, specifically to the printer service, that can print labels at a physical store location. We give it the set of labels we got back from the label service.

Example: Authentication

In this example, we show how to create an API Gateway to control the authentication and access to the backend resources using Spring Cloud[4]. The goal is to use an API Gateway to secure API resources using the Single Sign On (SSO) protocol based on OAuth2. Secure token processing is relayed by the Gateway to a central server to handle all the authentication to the backend microservices. In this manner, we can control identity and authentication at the Gateway while allowing individual microservices to make granular access decisions based on their own rules.

Resulting Context

Benefits

  1. Clients are isolated from the partitioning of the microservice architecture behind the gateway.
  2. Clients do not have to worry about locations of specific services.
  3. If there are client-specific policies to be applied, it is easy to apply them in a single place rather than in multiple places. An example of such a scenario is the cross-origin access policy.
  4. Provides the optimal API for each client.
  5. Reduces the number of requests/roundtrips.
  6. Simplifies the clients by moving the aggregation logic into the API gateway.

Drawbacks

  1. Increased complexity - the API gateway is one more moving part to be managed in your microservices architecture.
  2. Increased response time compared to direct calls, because of the additional network hop through the API gateway.
  3. Danger of implementing too much logic in the aggregation layer.

The API gateway can be seen as a single point of failure or, as Sam Newman describes it, as a single monolithic gateway. A better approach may be to create multiple API gateways for the different platform or frontends that your application needs to support. This is also referred to as backends for frontends. Also, it's worth noting that Netflix no longer puts any business logic in the Zuul edge services[3]; they strictly route requests down to their origin servers that handle this logic. So data aggregation is not done by Zuul.

References

  1. API Gateway Pattern
  2. Netflix Zuul
  3. Netflix Zuul Gets a Makeover to a Asynchronous and Non-Blocking Architecture
  4. Spring Cloud Security
  5. Spring Boot OAuth2
  6. Spring Cloud Netflix See section: "Router and Filter: Zuul"
お問い合わせ