Skip to content

Commit

Permalink
Fix shenanigans with crowdsec plugin and update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@TheophileDiot
TheophileDiot committed Jul 8, 2024
1 parent 24bc2b1 commit 6097cd7
Show file tree
Hide file tree
Showing 6 changed files with 53 additions and 129 deletions.
111 changes: 33 additions & 78 deletions crowdsec/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<img alt="BunkerWeb CrowdSec diagram" src="https://github.com/bunkerity/bunkerweb-plugins/raw/main/crowdsec/docs/diagram.svg" />
</p>

This [BunkerWeb](https://www.bunkerweb.io) plugin acts as a [CrowdSec](https://crowdsec.net/) bouncer. It will deny requests based on the decision of your CrowdSec API. Not only you will benefinit from the crowdsourced blacklist, you can also configure [scenarios](https://docs.crowdsec.net/docs/concepts#scenarios) to automatically ban IPs based on suspicious behaviors.
This [BunkerWeb](https://www.bunkerweb.io) plugin acts as a [CrowdSec](https://crowdsec.net/) bouncer. It will deny requests based on the decision of your CrowdSec API. Not only you will benefit from the crowdsourced blacklist, you can also configure [scenarios](https://docs.crowdsec.net/docs/concepts#scenarios) to automatically ban IPs based on suspicious behaviors.

# Table of contents

Expand All @@ -15,7 +15,7 @@ This [BunkerWeb](https://www.bunkerweb.io) plugin acts as a [CrowdSec](https://c
- [Optional : Application Security Component](#optional--application-security-component)
- [Syslog](#syslog)
- [Setup](#setup)
- [Docker](#docker)
- [Docker/Swarm](#dockerswarm)
- [Linux](#linux)
- [Optional : Application Security Component](#optional--application-security-component-1)
- [Linux Configuration](#linux-configuration)
Expand Down Expand Up @@ -81,109 +81,61 @@ log {

See the [plugins section](https://docs.bunkerweb.io/latest/plugins) of the BunkerWeb documentation for the installation procedure depending on your integration.

## Docker
## Docker/Swarm

```yaml
version: "3"
services:
bunkerweb:
image: bunkerity/bunkerweb:1.5.8
ports:
- 80:8080
- 443:8443
labels:
- "bunkerweb.INSTANCE=yes"
environment:
- SERVER_NAME=www.example.com
- API_WHITELIST_IP=127.0.0.0/24 10.20.30.0/24
- USE_CROWDSEC=yes
- CROWDSEC_API=http://crowdsec:8080
- CROWDSEC_APPSEC_URL=http://crowdsec:7422
- CROWDSEC_API_KEY=s3cr3tb0unc3rk3y
- USE_REVERSE_PROXY=yes
- REVERSE_PROXY_URL=/
- REVERSE_PROXY_HOST=http://myapp:8080
networks:
- bw-universe
- bw-services
- bw-plugins
logging:
driver: syslog
options:
syslog-address: "udp://10.10.10.254:514"
bw-scheduler:
image: bunkerity/bunkerweb-scheduler:1.5.8
depends_on:
- bunkerweb
- bw-docker
...
# BunkerWeb services
environment:
- DOCKER_HOST=tcp://bw-docker:2375
networks:
- bw-universe
- bw-docker
...
USE_MODSECURITY: "no" # Disable ModSecurity to let CrowdSec handle the security (only if the AppSec Component is used)
USE_CROWDSEC: "yes"
CROWDSEC_API: "http://crowdsec:8080" # This is the address of the CrowdSec container API in the same network
CROWDSEC_APPSEC_URL: "http://crowdsec:7422" # Comment if you don't want to use the AppSec Component
CROWDSEC_API_KEY: "s3cr3tb0unc3rk3y" # Remember to set a stronger key for the bouncer
bw-docker:
image: tecnativa/docker-socket-proxy:nightly
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- CONTAINERS=1
- LOG_LEVEL=warning
networks:
- bw-docker
...
crowdsec:
image: crowdsecurity/crowdsec:v1.6.2
image: crowdsecurity/crowdsec:v1.6.2 # Use the latest version but always pin the version for a better stability/security
volumes:
- cs-data:/var/lib/crowdsec/data
- ./acquis.yaml:/etc/crowdsec/acquis.yaml
- cs-data:/var/lib/crowdsec/data # To persist the CrowdSec data
- bw-logs:/var/log:ro # The logs of BunkerWeb for CrowdSec to parse
- ./acquis.yaml:/etc/crowdsec/acquis.yaml # The acquisition file for BunkerWeb logs
- ./appsec.yaml:/etc/crowdsec/acquis.d/appsec.yaml # Comment if you don't want to use the AppSec Component
- bw-logs:/var/log:ro
environment:
- BOUNCER_KEY_bunkerweb=s3cr3tb0unc3rk3y
- COLLECTIONS=crowdsecurity/nginx crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
BOUNCER_KEY_bunkerweb: "s3cr3tb0unc3rk3y" # Remember to set a stronger key for the bouncer
COLLECTIONS: "crowdsecurity/nginx crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules crowdsecurity/appsec-crs"
# COLLECTIONS: "crowdsecurity/nginx" # If you don't want to use the AppSec Component use this line instead
networks:
- bw-plugins
syslog:
image: balabit/syslog-ng:4.7.1
image: balabit/syslog-ng:4.7.1 # Use the latest version but always pin the version for a better stability/security
# image: lscr.io/linuxserver/syslog-ng:4.7.1-r1-ls116 # For aarch64 architecture
command: --no-caps
volumes:
- ./syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf
- bw-logs:/var/log
- bw-logs:/var/log # The logs of BunkerWeb for syslog-ng to store
- ./syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf # The syslog-ng configuration file
networks:
bw-plugins:
ipv4_address: 10.10.10.254
myapp:
image: nginxdemos/nginx-hello
networks:
- bw-services
ipv4_address: 10.10.10.254 # The IP address of the syslog service so BunkerWeb can send logs to it
networks:
bw-docker:
bw-services:
bw-universe:
ipam:
driver: default
config:
- subnet: 10.20.30.0/24
# BunkerWeb networks
...
bw-plugins:
ipam:
driver: default
config:
- subnet: 10.10.10.0/24
volumes:
bw-data:
bw-logs:
cs-data:
```

> [!TIP]
> The `balabit/syslog-ng` image used in the example is only compatible with amd64 architecture. If you want to use an arm64 compatible image, you can use `lscr.io/linuxserver/syslog-ng` instead.

## Linux

You'll need to install CrowdSec and configure it to parse BunkerWeb logs. To do so, you can follow the [official documentation](https://doc.crowdsec.net/docs/getting_started/install_crowdsec).
Expand Down Expand Up @@ -231,6 +183,7 @@ And you will need to install the AppSec Component's collections :
```shell
sudo cscli collections install crowdsecurity/appsec-virtual-patching
sudo cscli collections install crowdsecurity/appsec-generic-rules
sudo cscli collections install crowdsecurity/appsec-crs
```

Now you just have to restart the CrowdSec service :
Expand All @@ -246,10 +199,11 @@ If you need more information about the AppSec Component, you can refer to the [o
Now you can configure the plugin by adding the following settings to your BunkerWeb configuration file :

```env
USE_MODSECURITY=no # Disable ModSecurity to let CrowdSec handle the security (only if the AppSec Component is used)
USE_CROWDSEC=yes
CROWDSEC_API=http://127.0.0.1:8080
CROWDSEC_API_KEY=<The key provided by cscli>
CROWDSEC_APPSEC_URL=http://127.0.0.1:7422
CROWDSEC_APPSEC_URL=http://127.0.0.1:7422 # Comment if you don't want to use the AppSec Component
```

And finally reload the BunkerWeb service :
Expand All @@ -260,6 +214,9 @@ sudo systemctl reload bunkerweb

## Kubernetes

> [!WARNING]
> Keep in mind that the helm chart is still in beta and may not be stable.

The recommended way of installing CrowdSec in your Kubernetes cluster is by using their official [helm chart](https://github.com/crowdsecurity/helm-charts). You will find a tutorial [here](https://crowdsec.net/blog/kubernetes-crowdsec-integration/) for more information. By doing so, a syslog service is no more mandatory because agents will forward BunkerWeb logs to the CS API.

The first step is to add the CrowdSec chart repository :
Expand Down Expand Up @@ -316,8 +273,6 @@ metadata:
| `CROWDSEC_EXCLUDE_LOCATION` | | global | no | The locations to exclude while bouncing. It is a list of location, separated by commas. |
| `CROWDSEC_CACHE_EXPIRATION` | `1` | global | no | The cache expiration, in second, for IPs that the bouncer store in cache in live mode. |
| `CROWDSEC_UPDATE_FREQUENCY` | `10` | global | no | The frequency of update, in second, to pull new/old IPs from the CrowdSec local API. |
| `CROWDSEC_REDIRECT_LOCATION` | | global | no | The location to redirect the user when there is a ban. |
| `CROWDSEC_RET_CODE` | `403` | global | no | The HTTP code to return for IPs that trigger a ban remediation. (default: 403) |
| `CROWDSEC_APPSEC_URL` | `http://crowdsec:7422` | global | no | URL of the Application Security Component. |
| `CROWDSEC_APPSEC_FAILURE_ACTION` | `passthrough` | global | no | Behavior when the AppSec Component return a 500. Can let the request passthrough or deny it. |
| `CROWDSEC_APPSEC_CONNECT_TIMEOUT` | `100` | global | no | The timeout in milliseconds of the connection between the remediation component and AppSec Component. |
Expand Down
21 changes: 5 additions & 16 deletions crowdsec/crowdsec.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ local ngx = ngx
local ERR = ngx.ERR
local HTTP_INTERNAL_SERVER_ERROR = ngx.HTTP_INTERNAL_SERVER_ERROR
local HTTP_OK = ngx.HTTP_OK
local has_variable = utils.has_variable
local get_deny_status = utils.get_deny_status
local cs_init = cs.init
local cs_allow = cs.Allow
Expand All @@ -21,16 +20,11 @@ end

function crowdsec:init()
-- Check if init is needed
local init_needed, err = has_variable("USE_CROWDSEC", "yes")
if init_needed == nil then
return self:ret(false, "can't check USE_CROWDSEC variable : " .. err)
end
if not init_needed or self.is_loading then
if self.variables["USE_CROWDSEC"] ~= "yes" or self.is_loading then
return self:ret(true, "init not needed")
end
-- Init CS
local ok
ok, err = cs_init("/var/cache/bunkerweb/crowdsec/crowdsec.conf", "crowdsec-bunkerweb-bouncer/v1.6")
local ok, err = cs_init("/var/cache/bunkerweb/crowdsec/crowdsec.conf", "crowdsec-bunkerweb-bouncer/v2.0")
if not ok then
self.logger:log(ERR, "error while initializing bouncer : " .. err)
return self:ret(false, err)
Expand Down Expand Up @@ -58,17 +52,12 @@ end
function crowdsec:api()
if self.ctx.bw.uri == "/crowdsec/ping" and self.ctx.bw.request_method == "POST" then
-- Check crowdsec connection
local check, err = has_variable("USE_CROWDSEC", "yes")
if check == nil then
return self:ret(true, "error while checking variable USE_CROWDSEC (" .. err .. ")")
end
if not check then
return self:ret(true, "Crowdsec plugin not enabled")
if self.variables["USE_CROWDSEC"] ~= "yes" then
return self:ret(true, "CrowdSec plugin is not enabled", HTTP_OK)
end

-- Do the check
local ok
ok, err = cs_allow("127.0.0.1")
local ok, err = cs_allow("127.0.0.1")
if not ok then
return self:ret(true, "Error while executing CrowdSec bouncer : " .. err, HTTP_INTERNAL_SERVER_ERROR)
end
Expand Down
6 changes: 2 additions & 4 deletions crowdsec/jobs/crowdsec-conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,13 @@
CROWDSEC_EXCLUDE_LOCATION=getenv("CROWDSEC_EXCLUDE_LOCATION", ""),
CROWDSEC_CACHE_EXPIRATION=getenv("CROWDSEC_CACHE_EXPIRATION", "1"),
CROWDSEC_UPDATE_FREQUENCY=getenv("CROWDSEC_UPDATE_FREQUENCY", "10"),
CROWDSEC_REDIRECT_LOCATION=getenv("CROWDSEC_REDIRECT_LOCATION", ""),
CROWDSEC_RET_CODE=getenv("CROWDSEC_RET_CODE", "403"),
CROWDSEC_APPSEC_URL=getenv("CROWDSEC_APPSEC_URL", "http://crowdsec:7422"),
CROWDSEC_APPSEC_FAILURE_ACTION=getenv("CROWDSEC_APPSEC_FAILURE_ACTION", "passthrough"),
CROWDSEC_APPSEC_CONNECT_TIMEOUT=getenv("CROWDSEC_APPSEC_CONNECT_TIMEOUT", "100"),
CROWDSEC_APPSEC_SEND_TIMEOUT=getenv("CROWDSEC_APPSEC_SEND_TIMEOUT", "100"),
CROWDSEC_APPSEC_PROCESS_TIMEOUT=getenv("CROWDSEC_APPSEC_PROCESS_TIMEOUT", "500"),
CROWDSEC_ALWAYS_SEND_TO_APPSEC="true" if getenv("CROWDSEC_ALWAYS_SEND_TO_APPSEC", "no") == "yes" else "false",
CROWDSEC_APPSEC_SSL_VERIFY="true" if getenv("CROWDSEC_APPSEC_SSL_VERIFY", "no") == "yes" else "false",
CROWDSEC_ALWAYS_SEND_TO_APPSEC=("true" if getenv("CROWDSEC_ALWAYS_SEND_TO_APPSEC", "no") == "yes" else "false"),
CROWDSEC_APPSEC_SSL_VERIFY=("true" if getenv("CROWDSEC_APPSEC_SSL_VERIFY", "no") == "yes" else "false"),
)
.encode()
)
Expand Down
22 changes: 11 additions & 11 deletions crowdsec/lib/bouncer.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ function csmod.init(configFile, userAgent)
local captcha_ok = true
local err = captcha.New(runtime.conf["SITE_KEY"], runtime.conf["SECRET_KEY"], runtime.conf["CAPTCHA_TEMPLATE_PATH"], runtime.conf["CAPTCHA_PROVIDER"])
if err ~= nil then
ngx.log(ngx.ERR, "error loading captcha plugin: " .. err)
-- ngx.log(ngx.ERR, "error loading captcha plugin: " .. err)
captcha_ok = false
end
local succ, err, forcible = runtime.cache:set("captcha_ok", captcha_ok)
Expand All @@ -71,10 +71,10 @@ function csmod.init(configFile, userAgent)
end


local err = ban.new(runtime.conf["BAN_TEMPLATE_PATH"], runtime.conf["REDIRECT_LOCATION"], runtime.conf["RET_CODE"])
if err ~= nil then
ngx.log(ngx.ERR, "error loading ban plugins: " .. err)
end
-- local err = ban.new(runtime.conf["BAN_TEMPLATE_PATH"], runtime.conf["REDIRECT_LOCATION"], runtime.conf["RET_CODE"])
-- if err ~= nil then
-- ngx.log(ngx.ERR, "error loading ban plugins: " .. err)
-- end

if runtime.conf["REDIRECT_LOCATION"] ~= "" then
table.insert(runtime.conf["EXCLUDE_LOCATION"], runtime.conf["REDIRECT_LOCATION"])
Expand Down Expand Up @@ -605,11 +605,11 @@ end

function csmod.Allow(ip)
if runtime.conf["ENABLED"] == "false" then
ngx.exit(ngx.DECLINED)
return true, "disabled"
end

if ngx.req.is_internal() then
ngx.exit(ngx.DECLINED)
return true, "internal"
end

local remediationSource = flag.BOUNCER_SOURCE
Expand All @@ -619,7 +619,7 @@ function csmod.Allow(ip)
for k, v in pairs(runtime.conf["EXCLUDE_LOCATION"]) do
if ngx.var.uri == v then
ngx.log(ngx.ERR, "whitelisted location: " .. v)
ngx.exit(ngx.DECLINED)
return true, "whitelisted " .. v
end
local uri_to_check = v
if utils.ends_with(uri_to_check, "/") == false then
Expand Down Expand Up @@ -716,8 +716,8 @@ function csmod.Allow(ip)
if not ok then
if remediation == "ban" then
ngx.log(ngx.ALERT, "[Crowdsec] denied '" .. ip .. "' with '"..remediation.."' (by " .. flag.Flags[remediationSource] .. ")")
ban.apply(ret_code)
return
-- ban.apply(ret_code)
return true, "denied", true
end
-- if the remediation is a captcha and captcha is well configured
if remediation == "captcha" and captcha_ok and ngx.var.uri ~= "/favicon.ico" then
Expand Down Expand Up @@ -750,7 +750,7 @@ function csmod.Allow(ip)
end
end
end
ngx.exit(ngx.DECLINED)
return true, "allow"
end


Expand Down
4 changes: 2 additions & 2 deletions crowdsec/misc/crowdsec.conf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ UPDATE_FREQUENCY={{ CROWDSEC_UPDATE_FREQUENCY }}
#those apply for "ban" action
# /!\ REDIRECT_LOCATION and BAN_TEMPLATE_PATH/RET_CODE can't be used together. REDIRECT_LOCATION take priority over RET_CODE AND BAN_TEMPLATE_PATH
BAN_TEMPLATE_PATH=/var/lib/crowdsec/lua/templates/ban.html
REDIRECT_LOCATION={{ CROWDSEC_REDIRECT_LOCATION }}
RET_CODE={{ CROWDSEC_RET_CODE }}
REDIRECT_LOCATION=
RET_CODE=
#those apply for "captcha" action
#valid providers are recaptcha, hcaptcha, turnstile
CAPTCHA_PROVIDER=
Expand Down
18 changes: 0 additions & 18 deletions crowdsec/plugin.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,24 +78,6 @@
"regex": "^\\d+$",
"type": "text"
},
"CROWDSEC_REDIRECT_LOCATION": {
"context": "global",
"default": "",
"help": "The location to redirect the user when there is a ban.",
"id": "crowdsec-redirect-location",
"label": "CrowdSec redirect location",
"regex": "^.*$",
"type": "text"
},
"CROWDSEC_RET_CODE": {
"context": "global",
"default": "403",
"help": "The HTTP code to return for IPs that trigger a ban remediation. (default: 403)",
"id": "crowdsec-ret-code",
"label": "CrowdSec return code",
"regex": "^\\d+$",
"type": "text"
},
"CROWDSEC_APPSEC_URL": {
"context": "global",
"default": "http://crowdsec:7422",
Expand Down

0 comments on commit 6097cd7

Please sign in to comment.