ACME fails with IPv6 only ACME server #4170
Assignees
Milestone
Comments
|
Try adding contact, and I also do this ###
### ejabberd configuration file
###
### The parameters used in this configuration file are explained at
###
### https://docs.ejabberd.im/admin/configuration
###
### The configuration file is written in YAML.
### *******************************************************
### ******* !!! WARNING !!! *******
### ******* YAML IS INDENTATION SENSITIVE *******
### ******* MAKE SURE YOU INDENT SECTIONS CORRECTLY *******
### *******************************************************
### Refer to http://en.wikipedia.org/wiki/YAML for the brief description.
###
hosts:
- jabber.doman.ru
loglevel: error
ca_file: /opt/ejabberd/conf/cacert.pem
certfiles:
- /opt/ejabberd/conf/server.pem
## If you already have certificates, list them here
# certfiles:
# - /opt/ejabberd/certs/jabber.doman.ru.fullchain.pem
# - /opt/ejabberd/certs/jabber.doman.ru.privkey.pem
# - /opt/ejabberd/certs/upload.jabber.doman.ru.fullchain.pem
# - /opt/ejabberd/certs/upload.jabber.doman.ru.privkey.pem
# - /opt/ejabberd/certs/conference.jabber.doman.ru.fullchain.pem
# - /opt/ejabberd/certs/conference.jabber.doman.ru.privkey.pem
# - /opt/ejabberd/certs/pubsub.jabber.doman.ru.fullchain.pem
# - /opt/ejabberd/certs/pubsub.jabber.doman.ru.privkey.pem
# - /opt/ejabberd/certs/proxy.jabber.doman.ru.fullchain.pem
# - /opt/ejabberd/certs/proxy.jabber.doman.ru.privkey.pem
listen:
-
port: 5222
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5223
ip: "::"
tls: true
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5269
ip: "::"
module: ejabberd_s2s_in
max_stanza_size: 524288
-
port: 5443
ip: "::"
module: ejabberd_http
tls: true
request_handlers:
# /admin: ejabberd_web_admin
/api: mod_http_api
/bosh: mod_bosh
# /captcha: ejabberd_captcha
/upload: mod_http_upload
/ws: ejabberd_http_ws
-
port: 5280
ip: "::"
module: ejabberd_http
tls: false
request_handlers:
# /admin: ejabberd_web_admin
/.well-known/acme-challenge: ejabberd_acme
-
port: 3478
transport: udp
module: ejabberd_stun
use_turn: true
turn_min_port: 49152
turn_max_port: 65535
## The server's public IPv4 address:
turn_ipv4_address: 188.225.72.69
## The server's public IPv6 address:
# turn_ipv6_address: "2001:db8::3"
-
port: 5349
transport: tcp
module: ejabberd_stun
use_turn: true
tls: true
turn_min_port: 49152
turn_max_port: 65535
ip: "::"
## The server's public IPv4 address:
turn_ipv4_address: 188.225.72.69
# -
# port: 1883
# ip: "::"
# module: mod_mqtt
# backlog: 1000
acme:
## Staging environment
contact: mailto:post@vivaldi.net
# ca_url: https://acme-staging-v02.api.letsencrypt.org/directory
# auto: false
## Production environment (the default):
ca_url: https://acme-v02.api.letsencrypt.org/directory
auto: false
# ca_url: https://acme-v02.api.letsencrypt.org
s2s_use_starttls: required
acl:
admin:
- user: user@jabber.doman.ru
local:
user_regexp: ""
loopback:
ip:
- 127.0.0.0/8
- ::1/128
- ::FFFF:127.0.0.1/128
access_rules:
local:
allow: local
c2s:
deny: blocked
allow: all
announce:
allow: admin
configure:
allow: admin
muc_create:
allow: local
pubsub_createnode:
allow: local
trusted_network:
allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
access:
allow:
- acl: loopback
- acl: admin
oauth:
scope: "ejabberd:admin"
access:
allow:
- acl: loopback
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
ip: 127.0.0.1/8
what:
- status
- connected_users_number
shaper:
normal:
rate: 3000
burst_size: 20000
fast: 100000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
5000: admin
100: all
c2s_shaper:
none: admin
normal: all
s2s_shaper: fast
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco:
server_info:
-
modules: all
name: "abuse-addresses"
urls: ["mailto:post@vivaldi.net"]
# -
# modules:
# - mod_disco
# - mod_vcard
# name: admin-addresses
# urls:
# - https://example.com
# - mailto:admin@example.com
# - xmpp:admin@example.com
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: https://@HOST@:5443/upload
max_size: 262144000 #250mb
custom_headers:
"Access-Control-Allow-Origin": "https://@HOST@"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
mod_http_upload_quota:
max_days: 90
mod_last: {}
mod_mam:
## Mnesia is limited to 2GB, better to use an SQL backend
## For small servers SQLite is a good fit and is very easy
## to configure. Uncomment this when you have SQL configured:
## db_type: sql
assume_mam_usage: true
default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
access_mam:
- allow
default_room_options:
allow_subscription: true
persistent: true
mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
# mod_proxy65:
# port: 2053
# ip: 31.129.105.100
# access: local
# max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- flat
- pep
force_node_config:
## Avoid buggy clients to make their bookmarks public
storage:bookmarks:
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
## Only accept registration requests from the "trusted"
## network (see access_rules section above).
## Think twice before enabling registration from any
## address. See the Jabber SPAM Manifesto for details:
## https://github.com/ge0rg/jabber-spam-fighting-manifesto
ip_access: trusted_network
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_stun_disco:
credentials_lifetime: 12h
services:
-
host: 188.225.72.69
port: 3478
type: stun
transport: udp
restricted: false
-
host: 188.225.72.69
port: 3478
type: turn
transport: udp
restricted: true
-
host: jabber.doman.ru
port: 5349
type: stuns
transport: tcp
restricted: false
-
host: jabber.doman.ru
port: 5349
type: turns
transport: tcp
restricted: true
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false
#update_sql_schema: true
default_db: sql
auth_method: sql
#MariaDB with ejabberd
sql_type: mysql
sql_server: localhost
sql_database: admin
sql_username: admin
sql_password: admin
## If you want to specify the port:
sql_port: 3306
language: ru
### Local Variables:
### mode: yaml
### End:
### vim: set filetype=yaml tabstop=8 |
|
I added contact but this doesn't fix it. You don't have this issue, assuming the configuration file you linked is what you are using, you are using Let's Encrypt's endpoints which provide access over IPv4, the internal ACME server of mine does not have an IPv4 endpoint and fails to connect. Contact missing would not cause nxdomain. |
|
I hate doing this but bump this, I think the fix is practically just setting |
But if this is used when the server is IPv4-only, the query will fail, right? Looking at the documentation, I found that the
In Erlang/OTP 26 and 27.0-rc2 it is documented: Using Erlang 27.0-rc2: In Erlang/OTP 25.0 Release Candidate 3 that option was not yet documented: However, looking at the Erlang source code, I suspect that this option is supported since many more versions ago (at least since Erlang 20.3), even if it was not documented: Assuming that the ipv6fb4 is available and works correctly since at least Erlang 20 (this is the oldest version supported by ejabberd and p1_acme), then a patch like this could be safely added to p1_acme: diff --git a/src/p1_acme.erl b/src/p1_acme.erl
index b80f539..8e53502 100644
--- a/src/p1_acme.erl
+++ b/src/p1_acme.erl
@@ -598,6 +598,7 @@ http_request(State, ReqFun, RetryTimeout) ->
Timeout ->
{Method, URL} = Request = ReqFun(State),
?DEBUG("HTTP request: ~p", [Request]),
+ httpc:set_options([{ipfamily, inet6fb4}]),
case httpc:request(Method, URL,
[{timeout, infinity},
{connect_timeout, infinity}], |
|
A better way to test if
I would suggest this (untested) which only sets the options once and doesn't interfere with global/default httpc profile: diff --git a/src/p1_acme.erl b/src/p1_acme.erl
index b80f539..a648bea 100644
--- a/src/p1_acme.erl
+++ b/src/p1_acme.erl
@@ -113,6 +113,9 @@
%%% API
%%%===================================================================
start() ->
+ application:start(inets),
+ inets:start(httpc, [{profile, p1_acme}]),
+ httpc:set_options([{ip_family, inet6fb4}], p1_acme),
case application:ensure_all_started(?MODULE) of
{ok, _} -> ok;
Err -> Err
@@ -602,7 +605,7 @@ http_request(State, ReqFun, RetryTimeout) ->
[{timeout, infinity},
{connect_timeout, infinity}],
[{body_format, binary},
- {sync, false}]) of
+ {sync, false}], p1_acme) of
{ok, Ref} ->
ReqTimeout = min(timer:seconds(10), Timeout),
receive
@@ -612,7 +615,7 @@ http_request(State, ReqFun, RetryTimeout) ->
ReqFun, Response, State, RetryTimeout)
after ReqTimeout ->
?DEBUG("HTTP request timeout", []),
- httpc:cancel_request(Ref),
+ httpc:cancel_request(Ref, p1_acme),
http_request(State, ReqFun, RetryTimeout)
end;
{error, WTF} -> |
badlop
added a commit
to processone/p1_acme
that referenced
this issue
Apr 22, 2024
badlop
added a commit
that referenced
this issue
Apr 22, 2024
|
Thanks! I've tested your patch and seems to work, so I've applied it to p1_acme, and updated ejabberd to use it. It would be great if somebody else can test the feature and report the results, @regalialong ;) There are binary installers and the container image. |
|
I pulled the deb package from the actions run you have linked and ejabberd_acme appears to crash for me: ###
### ejabberd configuration file
###
### The parameters used in this configuration file are explained at
###
### https://docs.ejabberd.im/admin/configuration
###
### The configuration file is written in YAML.
### *******************************************************
### ******* !!! WARNING !!! *******
### ******* YAML IS INDENTATION SENSITIVE *******
### ******* MAKE SURE YOU INDENT SECTIONS CORRECTLY *******
### *******************************************************
### Refer to http://en.wikipedia.org/wiki/YAML for the brief description.
###
hosts:
- xmpp.demik
loglevel: info
ca_file: /usr/local/share/ca-certificates/DEMik/rootCA.crt
certfiles:
- /opt/ejabberd/conf/server.pem
## If you already have certificates, list them here
# certfiles:
# - /etc/letsencrypt/live/domain.tld/fullchain.pem
# - /etc/letsencrypt/live/domain.tld/privkey.pem
acme:
ca_url: https://pki.demik/v1/pki_int/acme/directory
auto: true
listen:
-
port: 5222
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
starttls_required: true
-
port: 5223
ip: "::"
module: ejabberd_c2s
max_stanza_size: 262144
shaper: c2s_shaper
access: c2s
tls: true
-
port: 5269
ip: "::"
module: ejabberd_s2s_in
max_stanza_size: 524288
shaper: s2s_shaper
-
port: 5443
ip: "::"
module: ejabberd_http
tls: true
request_handlers:
/admin: ejabberd_web_admin
/api: mod_http_api
/bosh: mod_bosh
/captcha: ejabberd_captcha
/upload: mod_http_upload
/ws: ejabberd_http_ws
-
port: 80
ip: "::"
module: ejabberd_http
request_handlers:
/admin: ejabberd_web_admin
/.well-known/acme-challenge: ejabberd_acme
-
port: 3478
ip: "::"
transport: udp
module: ejabberd_stun
use_turn: true
## The server's public IPv4 address:
# turn_ipv4_address: "203.0.113.3"
## The server's public IPv6 address:
# turn_ipv6_address: "2001:db8::3"
-
port: 1883
ip: "::"
module: mod_mqtt
backlog: 1000
s2s_use_starttls: optional
acl:
local:
user_regexp: ""
loopback:
ip:
- 127.0.0.0/8
- ::1/128
access_rules:
local:
allow: local
c2s:
deny: blocked
allow: all
announce:
allow: admin
configure:
allow: admin
muc_create:
allow: local
pubsub_createnode:
allow: local
trusted_network:
allow: loopback
api_permissions:
"console commands":
from:
- ejabberd_ctl
who: all
what: "*"
"admin access":
who:
access:
allow:
- acl: loopback
- acl: admin
oauth:
scope: "ejabberd:admin"
access:
allow:
- acl: loopback
- acl: admin
what:
- "*"
- "!stop"
- "!start"
"public commands":
who:
ip: 127.0.0.1/8
what:
- status
- connected_users_number
shaper:
normal:
rate: 3000
burst_size: 20000
fast: 100000
shaper_rules:
max_user_sessions: 10
max_user_offline_messages:
5000: admin
100: all
c2s_shaper:
none: admin
normal: all
s2s_shaper: fast
modules:
mod_adhoc: {}
mod_admin_extra: {}
mod_announce:
access: announce
mod_avatar: {}
mod_blocking: {}
mod_bosh: {}
mod_caps: {}
mod_carboncopy: {}
mod_client_state: {}
mod_configure: {}
mod_disco: {}
mod_fail2ban: {}
mod_http_api: {}
mod_http_upload:
put_url: https://@HOST@:5443/upload
custom_headers:
"Access-Control-Allow-Origin": "https://@HOST@"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
mod_last: {}
mod_mam:
## Mnesia is limited to 2GB, better to use an SQL backend
## For small servers SQLite is a good fit and is very easy
## to configure. Uncomment this when you have SQL configured:
## db_type: sql
assume_mam_usage: true
default: always
mod_mqtt: {}
mod_muc:
access:
- allow
access_admin:
- allow: admin
access_create: muc_create
access_persistent: muc_create
access_mam:
- allow
default_room_options:
mam: true
mod_muc_admin: {}
mod_offline:
access_max_user_messages: max_user_offline_messages
mod_ping: {}
mod_privacy: {}
mod_private: {}
mod_proxy65:
access: local
max_connections: 5
mod_pubsub:
access_createnode: pubsub_createnode
plugins:
- flat
- pep
force_node_config:
## Avoid buggy clients to make their bookmarks public
storage:bookmarks:
access_model: whitelist
mod_push: {}
mod_push_keepalive: {}
mod_register:
## Only accept registration requests from the "trusted"
## network (see access_rules section above).
## Think twice before enabling registration from any
## address. See the Jabber SPAM Manifesto for details:
## https://github.com/ge0rg/jabber-spam-fighting-manifesto
ip_access: trusted_network
mod_roster:
versioning: true
mod_s2s_dialback: {}
mod_shared_roster: {}
mod_stream_mgmt:
resend_on_timeout: if_offline
mod_stun_disco: {}
mod_vcard: {}
mod_vcard_xupdate: {}
mod_version:
show_os: false
### Local Variables:
### mode: yaml
### End:
### vim: set filetype=yaml tabstop=8
Everything beyond what I sent dumps what looks like entire certificate store. The system does trust our CA certificate (curl and others work), same happens no matter if I specify ca_file or comment it out. I also ran certbot on the same machine to verify that ACME works and that renews successfully as well. |
|
Aha! I can reproduce the problem, ejabberd crashes on start. The problem is that the function p1_acme:start is not called, so the httpc_p1_acme process is not started, and that one is required since the recent patch to perform the HTTP queries, as it contains the specific HTTP options. I've applied an additional patch to p1_acme, and built installers in my ejabberd fork: Now ejabberd starts for me. Can you try that? Once you confirm it works perfectly, I'll merge into ejabberd upstream. |
|
|
|
Ok, ejabberd already points to the latest p1_acme, the next version will include it. Thanks all for the report, patches and testing! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environment
Configuration (only if needed): grep -Ev '^$|^\s*#' ejabberd.yml
Errors from error.log/crash.log
Bug description
I have a IPv6-only VPN with internal TLD and I wanted to try out ejabberd. I have an ACME server at
https://pki.demikwhich I want to get the certificates for the XMPP server for but ejabberd fails to connect to it withnon-existing domainwhich doesn't make sense because the rest of the system resolves and connects fine.Checking logs on debug loglevel shows the connection to the ACME server fails:
Trying to diagnose this in the debug shell,
inet_res:getbyname("pki.demik", aaaa).fails which is fixed adding{inet6, true}.to inetrc.ACME still fails with non-existing domain though.
I think this is happening because httpc is at the default of
inetforipfamilyinstead ofinet6, connections to ipv6.google.com similarly fail with nxdomain.Setting inet6 makes the same request work:
And also makes the connection to the ACME server work with
ejabberdctl request_certificate all:Is this an oversight or is this just an unsupported usecase?
The text was updated successfully, but these errors were encountered: