protect-and-rate-limit-your-api
Before you start, make sure you have Kong installed. Install Kong Using Docker
Community Edition
Enterprise Edition
Kong version: 0.11.X
Scenario
You have just finished building a RESTful API. You are ready to expose it to potential developers. You want to avoid making too many requests to the service but allow certain developers to make more than others. You do not want to expose this service to the public yet.
You will configure Kong to protect the API with key authentication, secure the API with an ACL, rate limit all authenticated users, while allowing certain developers to make more requests than others.
High Level Tasks
- Provision the API endpoint
- Authentication: Protect the API with Key-Auth plugin
- Consumers: Provision consumers with credentials
- Security: Secure the API with Access Control List (ACL-whitelist)
- Traffic Control: Rate Limit consumers
Provision the API endpoint
Add API using the Admin API
Specify an API name, hosts (proxy host address), and the Upstream_url (API endpoint)
$ http post localhost:8001/apis \
name=demo \
hosts=api.demo.com \
upstream_url=http://mockbin.org/request
The hosts configuration is a proxy address that consumers will use to access the API endpoint.
The upstream_url is the API endpoint URL.
Verify configurations
Lets check the API configuration
$ http get localhost:8001/apis
The response should look like this.
{ "data": [ { "created_at": 1501081983000, "hosts": [ "api.demo.com" ], "http_if_terminated": false, "https_only": false, "id": "42112de8-f667-47ef-98ff-d2c1f19ce453", "name": "demo", "preserve_host": false, "retries": 5, "strip_uri": true, "upstream_connect_timeout": 60000, "upstream_read_timeout": 60000, "upstream_send_timeout": 60000, "upstream_url": "http://mockbin.org/requests" } ], "total": 1 }
Forward request through Kong
Now, lets verify that Kong is forwarding your request to the API Endpoint.
$ http get localhost:8000 \
host:api.demo.com
The response looks like this.
HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: host,connection,accept-encoding,x-forwarded-for,cf-ray,x-forwarded-proto,cf-visitor,x-forwarded-host,x-forwarded-port,user-agent,accept,cf-connecting-ip,x-request-id,via,connect-time,x-request-start,total-route-time Access-Control-Allow-Methods: GET Access-Control-Allow-Origin: * CF-RAY: 3af3672be77a26d8-FRA Connection: keep-alive Content-Encoding: gzip Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 12:53:50 GMT Etag: W/"3d1-dXCMQXY/GFw9ZxLV/yOyVA" Server: cloudflare-nginx Set-Cookie: __cfduid=d20832d4a9324ffecbcbf2f89659392da1508244830; expires=Wed, 17-Oct-18 12:53:50 GMT; path=/; domain=.mockbin.org; HttpOnly Transfer-Encoding: chunked Vary: Accept, Accept-Encoding Via: kong/0.11.0 X-Kong-Proxy-Latency: 307 X-Kong-Upstream-Latency: 1217 X-Powered-By: mockbin ...A successful response means Kong is now forwarding requests made to api.demo.com to the API endpoint - http://mockbin.org/requests - and sending the response back.
Great! You provisioned your first API and confirmed that Kong is forwarding and responding correctly to your API request.
Next, lets protect your API by requiring key authentication.
Authenticate: Protect the API with Key-Auth plugin
Enable and configure the Key Auth plugin
Run this command enable and configure the Key-Auth plugin.
$ http post localhost:8001/apis/demo/plugins \
name=key-auth \
config.key_names=X-AUTH
Q. What happens when we enable a plugin on the specific API?
Verify protection
Lets try to access the API.
$ http get localhost:8000 \
host:api.demo.com
The response is HTTP/1.1 401 Unauthorized ```sh HTTP/1.1 401 Unauthorized Connection: keep-alive Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 12:56:31 GMT Server: kong/0.11.0 Transfer-Encoding: chunked WWW-Authenticate: Key realm=”kong”
{ “message”: “No API key found in request” }
#### Verify protection
Lets try to access the API with invalid credentials
```sh
$ http get localhost:8000 \
host:api.demo.com \
X-Auth:invalidcredentials
The response is a HTTP/1.1 403 Forbidden
```sh HTTP/1.1 403 Forbidden Connection: keep-alive Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 12:57:20 GMT Server: kong/0.11.0 Transfer-Encoding: chunked
{ “message”: “Invalid authentication credentials” }
##### Great! Your API is now protected via Key-Authentication. You do not have any consumers provisioned for your API yet.
##### Next, lets allow for consumers to access your API.
---
### **Consumers:** Provision Consumers with Credentials
#### Add consumer
Lets add consumer with username **Viet**
```sh
$ http post localhost:8001/consumers \
username=viet
Give the consumer a password (key)
$ http post localhost:8001/consumers/viet/key-auth \
key=vietpass
Verify consumer configuration:
Access the API with an authorized users
$ http get localhost:8000 \
host:api.demo.com \
X-AUTH:vietpass
Results: You should get “HTTP 200 OK” from API endpoint. ```sh HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: host,connection,accept-encoding,x-forwarded-for,cf-ray,x-forwarded-proto,cf-visitor,x-forwarded-host,x-forwarded-port,user-agent,accept,x-auth,x-consumer-id,x-consumer-username,cf-connecting-ip,x-request-id,via,connect-time,x-request-start,total-route-time Access-Control-Allow-Methods: GET Access-Control-Allow-Origin: * CF-RAY: 3af36d87c2a5641b-FRA Connection: keep-alive Content-Encoding: gzip Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 12:58:10 GMT Etag: W/”44b-pXzw1n8WkAcCmoWLNOMOMw” Server: cloudflare-nginx Set-Cookie: __cfduid=d644fc618272d3cecc23ca895d83692c31508245090; expires=Wed, 17-Oct-18 12:58:10 GMT; path=/; domain=.mockbin.org; HttpOnly Transfer-Encoding: chunked Vary: Accept, Accept-Encoding Via: kong/0.11.0 X-Kong-Proxy-Latency: 20 X-Kong-Upstream-Latency: 563 X-Powered-By: mockbin
{ configurations… }
---
#### Add another consumer
Lets add a second consumer with username **Olya**.
```sh
$ http post localhost:8001/consumers \
username=olya
Give the user a password (key)
$ http post localhost:8001/consumers/olya/key-auth key=olyapass
Great! Consumers are now allowed access to your API via the Key-Auth plugin. It’s important to note that this allows all authenticated users access to the API.
Next, lets secure the API to only certain users with Access Control Lists (ACLs). This will allows only users that belong to the ACL access to your API. In our example, Viet and Olya.
Security: Secure the API with Access Control List (ACL-whitelist)
Enable ACL plugin and create whitelist groups for your API
$ http post localhost:8001/apis/demo/plugins \
name=acl \
config.whitelist=partner_group,private_group,public_group
Note: This configures the API named demo with the ACL plugin. You include 3 ACL groups. In this lab, you will use private_group. You will use the other groups in subsequent labs.
Add the consumer to whitelist group to allow access to the API
In the previous step, you added the ACL, now lets add the consumer Viet to the private_group whitelist.
$ http post localhost:8001/consumers/viet/acls \
group=private_group
Verify a forbidden consumer
Lets try to access the API with a consumer NOT in the ACL whitelist. e.g Olya.
http get localhost:8000 \
host:api.demo.com \
X-AUTH:olyapass
The response is a “HTTP 403 Forbidden”.
```sh HTTP/1.1 403 Forbidden Connection: keep-alive Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 12:59:52 GMT Server: kong/0.11.0 Transfer-Encoding: chunked
{ “message”: “You cannot consume this service” }
>**Note:** If the API is configured properly, this consumer should be forbidden since it's not part of the whitelist group you specified in the previous steps.
#### Verify an allowed consumer
Lets try to access the API with a consumer in the ACL whitelist. e.g Viet.
```sh
http get localhost:8000 \
host:api.demo.com \
X-AUTH:vietpass
The Response is a “HTTP 200 OK. This users is allowed.
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: host,connection,accept-encoding,x-forwarded-for,cf-ray,x-forwarded-proto,cf-visitor,x-forwarded-host,x-forwarded-port,user-agent,accept,x-auth,x-consumer-id,x-consumer-username,x-consumer-groups,cf-connecting-ip,x-request-id,via,connect-time,x-request-start,total-route-time
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
CF-RAY: 3af370c8078f26d8-FRA
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json; charset=utf-8
Date: Tue, 17 Oct 2017 13:00:23 GMT
Etag: W/"475-Vc/hN6CNZrYnGOeMNnZgVQ"
Server: cloudflare-nginx
Set-Cookie: __cfduid=d29b5ba0312870b1c8355405b5f330eaa1508245223; expires=Wed, 17-Oct-18 13:00:23 GMT; path=/; domain=.mockbin.org; HttpOnly
Transfer-Encoding: chunked
Vary: Accept, Accept-Encoding
Via: kong/0.11.0
X-Kong-Proxy-Latency: 195
X-Kong-Upstream-Latency: 599
X-Powered-By: mockbin
Note: Notice the Access-Control-Allow-Credentials: true Your ACL plugin is configured properly.
Great! You have secured access to your API with an ACL-whitelist - only authenticated users that are in the whitelist have access to your API. In this example, user Viet in the private_group ACL whitelist has access to the API.
Next, lets limit the amount of request to your API with the Rate-Limiting plugin.
Traffic Control: Rate Limit Your API for Specific Consumers
Enable and configure the Rate Limiting plugin for your API
Enable rate limting and limit requests per minute to 3.
$ http post localhost:8001/apis/demo/plugins/ \
name=rate-limiting \
config.minute=3
This enables the Rate Limiting plugin on the API called demo. It sets the rate-limit to 3 requests per minute
####Customize the rate limit for your API for an internal developer. The consumer viet is considered an internal developer so you want to allow more requests to your API. Lets configure this.
IMPORTANT NOTE: Replace the below consumer_id specific to your instance of KONG.
Use: $ http get localhost:8001/consumers command to retrieve the consumer ids.
$ http post localhost:8001/apis/demo/plugins \
name=rate-limiting \
consumer_id=71589bd3-ad98-489b-835f-606aa9ae2bf7 \
config.minute=10
This configures the rate limit for demo API specific to consumer object with id 71589bd3-ad98-489b-835f-606aa9ae2bf7. e.g. the consumer viet.
The limit is set to 10 request per minute.
Verify Rate Limit for Viet
Lets verify that Viet cannot access the API more than 10 times. Run this command 11 or more times.
$ http get localhost:8000 \
host:api.demo.com \
X-Auth:vietpass
The response after 10 times should look like this:
HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: host,connection,accept-encoding,x-forwarded-for,cf-ray,x-forwarded-proto,cf-visitor,user-agent,accept,x-auth,x-consumer-id,x-consumer-username,x-consumer-groups,cf-connecting-ip,x-request-id,x-forwarded-port,via,connect-time,x-request-start,total-route-time Access-Control-Allow-Methods: GET Access-Control-Allow-Origin: * CF-RAY: 38ab7799e45d4e42-DME Connection: keep-alive Content-Encoding: gzip Content-Type: application/json; charset=utf-8 Date: Mon, 07 Oct 2017 16:03:39 GMT Etag: W/"44b-DzOIcPzBHWXUPZCS3ukibQ" Server: cloudflare-nginx Set-Cookie: __cfduid=d5256090c5f8c55796d370975a47897c61502121819; expires=Tue, 07-Oct-18 16:03:39 GMT; path=/; domain=.mockbin.org; HttpOnly Transfer-Encoding: chunked Vary: Accept, Accept-Encoding Via: kong/0.11.0 X-Kong-Proxy-Latency: 3 X-Kong-Upstream-Latency: 504 X-Powered-By: mockbin X-RateLimit-Limit-minute: 10 X-RateLimit-Remaining-minute: 9Note: Ntioce the “X-RateLimit-Limit-Minute:10”. This is the rate limit that you set for this user.
You should get the following response if the rate limit is exceeded.
HTTP/1.1 429
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Tue, 17 Oct 2017 13:03:38 GMT
Server: kong/0.11.0
Transfer-Encoding: chunked
X-RateLimit-Limit-minute: 10
X-RateLimit-Remaining-minute: 0
{
"message": "API rate limit exceeded"
}
Customize the rate limit for your API for a partner.
The consumer Olya is considered an partner vendor. You want to give access to the API but allow for less access than your internal developers. Lets configure this.
IMPORTANT NOTE: Replace the below consumer_id specific to your instance of KONG.
Use: $ http get localhost:8001/consumers command to retreive the consumer ids.
$ http post localhost:8001/apis/demo/plugins \
name=rate-limiting \
consumer_id=abc1e799-6a12-427e-8303-357e4714bee7 \
config.minute=5
This configures the rate limit for demo API specific to consumer object with id abc1e799-6a12-427e-8303-357e4714bee7. e.g. consumer olya.
The limit is set to 5 request per minute.
Add Olya to the partner_group whitelist ACL.
$ http post localhost:8001/consumers/olya/acls \
group=partner_group
Verify Rate Limit for Olya
Lets verify that Olya cannot access the API more than 5 times. Run this command 6 or more times.
$ http get localhost:8000 \
host:api.demo.com \
X-Auth:olyapass
The response after 6 times should look like this:
HTTP/1.1 429 Connection: keep-alive Content-Type: application/json; charset=utf-8 Date: Tue, 17 Oct 2017 13:05:24 GMT Server: kong/0.11.0 Transfer-Encoding: chunked X-RateLimit-Limit-minute: 5 X-RateLimit-Remaining-minute: 0 { "message": "API rate limit exceeded" }
Summary
Congratulations! You configured Kong to protect the API with Key-Authentication, secured your API with ACL whilelists, and customized the rate limit for different consumers. Specifically, consumer Viet - who is considered an internal developer has access to the API no more than 10 request/minute. Consumer Olya - who is considered a partner - has access to the API no more than 5 request/minute.
What Next
If you have some extra time, feel free to do bonus exercise. Otherwise, lets become familiar with the KONG/NGINX configuration and log files.
Explore NGINX-KONG Configuration Files
####Bonus Exercise
This is a bonus exercise. This expands on this exercise by allowing anonymous users access to the API although at a significantly reduced rate.
Allow Anonymous Users Access