Aeraki Series: How to set local rate limiting rules

Aeraki can help you manage any Layer 7 protocol in a service mesh. Currently, Aeraki already supports open source protocols such as Dubbo, Thrit, and Redis. You can also use Aeraki's MetaProtocol protocol extension framework to manage Layer 7 traffic for private protocols.

This series of tutorials will introduce how to use Aeraki to provide seven-layer traffic routing, local flow control, and global flow control for services using protocols such as Dubbo and Thrift in a service mesh, as well as how to quickly develop a custom protocol based on the Aeraki Protocol and manage services using custom protocols in the Istio service mesh.

This tutorial describes how to use the MetaRouter CRD resource provided by Areaaki to set local throttling rules for application protocols developed based on MetaProtocol.

Installing the Sample Program

If you haven't installed the sample application yet, refer to the Quick Start guide to install Aeraki, Istio, and the sample application.

After the installation is complete, you can see that the following two NSs are added to the cluster. The two NSs are installed with sample programs for Dubbo and Thrift protocols based on MetaProtocol. You can choose any program for testing.

 ➜ ~ kubectl get ns|grep meta
 meta-dubbo Active 16m
 meta-thrift Active 16m

Aeraki's throttling rules are designed to be intuitive and flexible, supporting both throttling of all incoming requests to a service and fine-grained throttling of requests to a server based on different conditions.

Limit all incoming requests to the service

 kubectl apply -f- <<EOF
 apiVersion: metaprotocol.aeraki.io/v1alpha1
 kind: MetaRouter
 metadata:
 name : test-metaprotocol-thrift-route
 namespace: meta-thrift
 spec:
 hosts:
  - thrift-sample-server.meta-thrift.svc.cluster. local  
 localRateLimit:
 tokenBucket:
 fillInterval: 60s
 maxTokens: 2
 tokensPerFill: 2
 EOF

Note: Because local throttling is processed on a service instance, when a service has multiple instances, the actual throttling effect is the throttling number multiplied by the number of instances.

Use the aerakictl command to view the client's application logs. You can see that the client can only successfully execute 4 requests per minute (there are two service instances, and each service instance is limited to 2 requests per minute):

 ➜ ~ aerakictl_app_log client meta-thrift -f --tail 10  
 Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
 Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
 Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
 Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
 org.apache.thrift.TApplicationException: meta protocol local rate limit: request '5' has been rate limited
 at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
 at org.aeraki.HelloService$Client.recv_sayHello(HelloService.java:61)
 at org.aeraki.HelloService$Client.sayHello(HelloService.java:48)
 at org.aeraki.HelloClient.main(HelloClient.java:44)
 Connected to thrift-sample-server
 org.apache.thrift.TApplicationException: meta protocol local rate limit: request '1' has been rate limited
 ...

Limit incoming requests to services based on conditions

Aeraki supports setting multiple throttling rules for services based on conditions to meet fine-grained throttling requirements, such as grouping requests by user or interface and setting different throttling rules for each group.

The matching conditions for packet current limiting are the same as those for routing. Any attribute that can be extracted from the request data packet can be used as the matching condition for the current limiting rule.

For example, the following rules set different current limiting conditions for the sayHello and ping interfaces:

 apiVersion: metaprotocol.aeraki.io/v1alpha1
 kind: MetaRouter
 metadata:
 name : test-metaprotocol-thrift-route
 namespace: meta-thrift
 spec:
 hosts:
  - thrift-sample- server.meta -thrift.svc.cluster.local  
 localRateLimit:
 conditions:
 - match:
 attributes:
 method:
 exact: sayHello
 tokenBucket:
 fillInterval: 60s
 maxTokens: 10
 tokensPerFill: 10
 - match:
 attributes:
 method:
 exact: ping
 tokenBucket:
 fillInterval: 60s
 maxTokens: 100
 tokensPerFill: 100

Set up traffic limiting rules by service and by condition at the same time

You can set both service-level throttling rules and conditional throttling rules at the same time. This is suitable for situations where you need to set an overall throttling rule for all requests of a service, while also setting exceptions for one or several groups of requests.

For example, the following rate limiting rule sets an overall rate limiting rule of 1000 messages per minute for the service, and sets a rate limiting condition of 100 messages per minute for the ping interface.

 apiVersion: metaprotocol.aeraki.io/v1alpha1
 kind: MetaRouter
 metadata:
 name : test-metaprotocol-thrift-route
 namespace: meta-thrift
 spec:
 hosts:
  - thrift-sample- server.meta -thrift.svc.cluster.local  
 localRateLimit:
 tokenBucket:
 fillInterval: 60s
 maxTokens: 1000
 tokensPerFill: 1000
 conditions:
 - match:
 attributes:
 method:
 exact: ping
 tokenBucket:
 fillInterval: 60s
 maxTokens: 100
 tokensPerFill: 100

Understanding the principles

In the configuration sent by Aeraki to the Sidecar Proxy, the MetaProtocol Proxy is set for the Listener corresponding to the service, and the local rate limit filter is set in the configuration.

Aeraki will translate the rate limiting rules configured in MetaRouter into rate limiting configurations of the local rate limit filter and send them to MetaProtocol Proxy through Aeraki.

You can view the configuration of the service's sidecar proxy with the following command:

 aerakictl_sidecar_config server-v1 meta-thrift |fx

The MetaProtocol Proxy configuration in the Inbound Listener of the Thrift service is as follows:

 {
 "name" : "envoy.filters.network.meta_protocol_proxy" ,
 "typed_config" : {
 "@type" : "type.googleapis.com/udpa.type.v1.TypedStruct" ,
 "type_url" : "type.googleapis.com/aeraki.meta_protocol_proxy.v1alpha.MetaProtocolProxy" ,
 "value" : {
 "stat_prefix" : "inbound|9090||" ,
 "application_protocol" : "thrift" ,
 "route_config" : {
 "name" : "inbound|9090||" ,
 "routes" : [
 {
 "route" : {
 "cluster" : "inbound|9090||"  
 }
 }
 ]
 },
 "codec" : {
 "name" : "aeraki.meta_protocol.codec.thrift"  
 },
 "meta_protocol_filters" : [
 {
 "name" : "aeraki.meta_protocol.filters.local_ratelimit" ,
 "config" : {
 "@type" : "type.googleapis.com/aeraki.meta_protocol_proxy.filters.local_ratelimit.v1alpha.LocalRateLimit" ,
 "stat_prefix" : "thrift-sample-server.meta-thrift.svc.cluster.local" ,
 "token_bucket" : {
 "max_tokens" : 2,
 "tokens_per_fill" : 2,
 "fill_interval" : "60s"  
 }
 }
 },
 {
 "name" : "aeraki.meta_protocol.filters.router"  
 }
 ]
 }
 }
 }