[Git][debian-proftpd-team/proftpd-mod-proxy][upstream] New upstream version 0.9.4
Hilmar Preuße (@hilmar-guest)
gitlab at salsa.debian.org
Thu Mar 28 22:26:14 GMT 2024
Hilmar Preuße pushed to branch upstream at Debian ProFTPD Team / proftpd-mod-proxy
Commits:
fde85fec by Hilmar Preuße at 2024-03-28T23:09:24+01:00
New upstream version 0.9.4
- - - - -
8 changed files:
- .github/workflows/ci.yml
- .github/workflows/codeql.yml
- .github/workflows/regressions.yml
- Makefile.in
- doc/NOTES.health-checks
- lib/proxy/db.c
- mod_proxy.h.in
- mod_proxy.html
Changes:
=====================================
.github/workflows/ci.yml
=====================================
@@ -4,6 +4,9 @@ on:
push:
branches:
- master
+ paths-ignore:
+ - '*.html'
+ - '*.md'
pull_request:
branches:
- master
=====================================
.github/workflows/codeql.yml
=====================================
@@ -5,6 +5,7 @@ on:
branches:
- master
paths-ignore:
+ - '*.html'
- '**/*.md'
- '**/doc/*'
pull_request:
=====================================
.github/workflows/regressions.yml
=====================================
@@ -4,6 +4,9 @@ on:
push:
branches:
- master
+ paths-ignore:
+ - '*.html'
+ - '*.md'
pull_request:
branches:
- master
=====================================
Makefile.in
=====================================
@@ -74,6 +74,7 @@ SHARED_MODULE_OBJS=mod_proxy.lo \
lib/proxy/ssh/redis.lo \
lib/proxy/tls.lo \
lib/proxy/tls/db.lo \
+ lib/proxy/tls/redis.lo \
lib/proxy/uri.lo \
lib/proxy/forward.lo \
lib/proxy/reverse.lo \
@@ -119,7 +120,7 @@ LDFLAGS=-L../../lib @LIBDIRS@
$(LIBTOOL) --mode=compile --tag=CC $(CC) $(CPPFLAGS) $(CFLAGS) $(SHARED_CFLAGS) -c $< -o $@
shared: $(SHARED_MODULE_OBJS)
- $(LIBTOOL) --mode=link --tag=CC $(CC) -o $(MODULE_NAME).la $(SHARED_MODULE_OBJS) -rpath $(LIBEXECDIR) $(LDFLAGS) $(SHARED_LDFLAGS) $(SHARED_MODULE_LIBS) `cat $(MODULE_NAME).c | grep '$$Libraries:' | sed -e 's/^.*\$$Libraries: \(.*\)\\$$/\1/'`
+ $(LIBTOOL) --mode=link --tag=CC $(CC) -o $(MODULE_NAME).la $(SHARED_MODULE_OBJS) -rpath $(LIBEXECDIR) $(LDFLAGS) $(SHARED_LDFLAGS) $(MODULE_LIBS) $(SHARED_MODULE_LIBS) `cat $(MODULE_NAME).c | grep '$$Libraries:' | sed -e 's/^.*\$$Libraries: \(.*\)\\$$/\1/'`
static: $(MODULE_OBJS)
test -z "$(MODULE_LIBS)" || echo "$(MODULE_LIBS)" >> $(MODULE_LIBS_FILE)
=====================================
doc/NOTES.health-checks
=====================================
@@ -73,14 +73,176 @@ is unhealthy? How long to skip over unhealthy backends? What happens when
all backends are unhealthy?
Types of errors:
+ DNS resolution errors
TCP connect errors
TLS handshake errors
FTP connect/login errors (ignoring bad credentials!)
+ SSH banner error (e.g. ill-formed SSH banner/version)
Specifically, we want to track errors that indicate that that server is
-unavailable for service. Thus probably NOT TLS handshake errors, or
-FTP data transfer errors. That leaves TCP connect errors, and FTP non-200
-responses on connect.
+unavailable for service. Thus probably NOT FTP data transfer errors.
+
+ Configuration:
+ use passive health tracking: yes/no
+ number of failures before unhealthy ("down") [default: 2-3]
+ number of successes before healthy ("up") [default: 1]
+ success implies active probes/health checks; this is NOT that
+ unhealthy timeout (before unhealthy status expires) [default: 10s]
+
+ ProxyReverseHealthPolicy
+ none/off
+
+ PassiveChecks
+ failures N (2)
+ expires time (30s)
+
+ ProxyReverseHealthPolicy PassiveChecks failures 2 expires 30s
+
+ ActiveChecks (not implemented)
+
+ Retries:
+ depth-first (retry same target multiple times _first_), or
+ breadth-first (retry next target _first_, cycling through list until max retries count reached)
+
+ NOTE: We only want to call _index_used() if we WANT to move to the next
+ backend. If we want to retry THIS backend (due to
+ transient/ignorable/non-fatal error), then we explicitly do NOT call
+ _index_used. Subtle. Need to capture this in comments for my future self.
+
+ int (*policy_used_backend)(pool *p, void *dsh, int policy_id,
+ unsigned int vhost_id, int backend_id);
+ int (*policy_update_backend)(pool *p, void *dsh, int policy_id,
+ unsigned int vhost_id, int backend_id, int conn_incr, long connect_ms);
+
+Maybe we need now:
+
+ int (*policy_unhealthy_backend)(pool *p, void *dsh, int health_policy_id,
+ unsigned int vhost_id, int backend_id, int unhealthy_incr, long unhealthy_ms, const char *unhealthy_reason)
+
+ Unhealthy Errors
+ DNS
+ Maybe log message should include hint for "Trace dns:20" for more
+ info
+
+ gai_strerror:
+ EAI_AGAIN (ignore)
+ EAI_FAIL
+ EAI_SYSTEM (see errno)
+ EAI_NONAME
+ mapped to ENOENT
+ EAI_FAMILY
+ mapped to EAFNOSUPPORT
+
+ h_errno:
+ HOST_NOT_FOUND
+ TRY_AGAIN (ignore)
+ NO_RECOVERY
+ NO_DATA
+
+ TCP
+ EADDRINUSE (local error, ignore?)
+ EADDRNOTAVAIL (local error, ignore?)
+ ENETDOWN
+ ENETUNREACH
+ EHOSTUNREACH
+ ENETRESET (ignore?)
+ ECONNABORTED
+ ECONNRESET
+ ECONNREFUSED
+ ETIMEDOUT
+
+ TLS
+ any ignorable?
+
+ Maybe log message should include hint for "Trace tls:20" for more
+ info
+
+ FTP
+ non-220 greeting
+ banner_ok = FALSE (non-2xx) in reverse_try_connect()
+
+ Maybe log message should include hint for "Trace proxy.response:20"
+ for more info
+
+ SSH
+ illegal SSH version/banner
+
+ bad_proto = TRUE in lib/proxy/ssh.c#ssh_get_server_version
+
+ Maybe log message should include hint for
+ "Trace proxy.ssh2:20 ssh2:20" for more info
+
+ note that lib/proxy/ssh.c will NOT have the db index as seen
+ in lib/proxy/reverse.c; may need a way to get it. That said,
+ lib/proxy/ssh.c#ssh_ssh2_auth_completed_ev() DOES call
+ proxy_reverse_connect() and thus all of the above. So the lack
+ of treatment of illegal SSH banner as "unhealthy" is the result;
+ I think that's OK for now. We can handle this case as unhealthy
+ in a later pass.
+
+ Remember that _index_used is what advances the index, for _next_backend.
+ lib/proxy/reverse/db.c schema does NOT currently have columns for
+ unhealthy status; would need to bump schema version!
+
+ unhealthy_count INT
+ unhealthy_ms BIGINT
+ unhealthy_reason TEXT
+
+ need reverse_connect_index_unhealthy() function for recording "down"
+ backends, it increments unheathy_count, updates _ms, records last
+ _reason (e.g. "dns: host unknown", "tcp: connection refused")
+
+ and in _next per-policy db functions, need to see if selected backend
+ is unhealthy, or not. If unhealthy, LOG IT. If down status expired,
+ log expiry (and clear unhealthy columns) and use selected entry.
+ Otherwise, select NEXT backend.
+
+ This is where the health policy failure count is honored/implemented,
+ by examining unhealthy_count value for exceeding threshold.
+
+ If all backend addresses discovered are marked as "unavailable", should
+ mod_proxy try connecting to one of them anyway?
+
+ These _next per-policy db functions thus need to handle this scenario,
+ where all are unhealthy, none expired; watch for "wrapping" around the
+ list of targets! This, in turn, means that callers of _next need
+ to handle NULL/ENOENT returns.
+
+ Some policies, like Shuffle or Random, may require a "read-then-write"
+ approach. Hmm. These are currently implemented as
+ "get index into list of conns", and idx is a single value, not any
+ metadata associated with that backend/idx (such as unhealthy fields).
+
+ Maybe, in policy_next_backend, it should be:
+
+ get index per policy
+
+ get backend metadata for that index
+ exposing/adding a policy_get_backend feels like surfacing this
+ unnecessarily; it should be an impl detail of policy_next_backend.
+ However, if implemented at the datastore layer, then the datastore
+ layer needs accessors to the configured health policy parameters.
+ I guess this is where having an include/proxy/reverse/health.h API
+ could come in handy.
+
+ int proxy_reverse_health_sess_init(pool *p);
+ does lookup of ProxyReverseHealthPolicy directive
+ called by proxy_reverse_sess_init
+
+ int proxy_reverse_health_is_healthy(unsigned int unhealthy_count, unsigned long unhealthy_ms, const char *unhealthy_reason);
+
+ This way, the Health API knows which policy is configured, knows
+ its parameters. It logs the fields, handles expiration, etc.
+ If the result is TRUE, then we clear any unhealthy fields.
+
+ check backend metadata for health
+
+ if unhealthy
+ mark index as used
+ goto top
+
+ TRACK starting index, to catch looping around to same index value
+ again due to ALL backends being unhealthy.
Consider also the case where a configured backend server is a DNS name/URL,
which resolves to multiple IP addresses/ports. These resolved addresses/ports
@@ -105,3 +267,17 @@ in the db. Hmm.
https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-health-check/#fine-tuning-tcp-health-checks
Defaults: interval=5s, passes=1, fails=1
+
+Tests:
+
+ No regressions (default health policy: none)
+ health policy: passive checks
+ DNS errors
+ TCP errors
+ TLS errors
+ FTP errors
+
+ count exceeded, not exceeded
+ expired, not expired
+
+ all backends unhealthy for a connect policy + vhost
=====================================
lib/proxy/db.c
=====================================
@@ -1,6 +1,6 @@
/*
* ProFTPD - mod_proxy database implementation
- * Copyright (c) 2015-2022 TJ Saunders
+ * Copyright (c) 2015-2024 TJ Saunders
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -38,8 +38,8 @@ static const char *current_schema = NULL;
static const char *trace_channel = "proxy.db";
-#define PROXY_DB_SQLITE_MAX_RETRY_COUNT 4
-#define PROXY_DB_SQLITE_MAX_RETRY_DELAY_MS 250
+#define PROXY_DB_SQLITE_MAX_RETRY_COUNT 20
+#define PROXY_DB_SQLITE_MAX_RETRY_DELAY_MS 100
#define PROXY_DB_SQLITE_TRACE_LEVEL 17
@@ -64,12 +64,12 @@ static int db_busy(void *user_data, int busy_count) {
/* If we're busy, then sleep for a short while, on the assumption that the
* other process will finish its business with our tables.
*/
- (void) pr_timer_usleep(PROXY_DB_SQLITE_MAX_RETRY_DELAY_MS);
+ (void) pr_timer_usleep(PROXY_DB_SQLITE_MAX_RETRY_DELAY_MS * 1000);
return retry;
}
-#ifdef SQLITE_CONFIG_LOG
+#if defined(SQLITE_CONFIG_LOG)
static void db_err(void *user_data, int err_code, const char *err_msg) {
if (current_schema != NULL) {
pr_trace_msg(trace_channel, 1, "(sqlite3): schema '%s': [error %d] %s",
@@ -82,7 +82,7 @@ static void db_err(void *user_data, int err_code, const char *err_msg) {
}
#endif /* SQLITE_CONFIG_LOG */
-#ifdef SQLITE_CONFIG_SQLLOG
+#if defined(SQLITE_CONFIG_SQLLOG)
static void db_sql(void *user_data, sqlite3 *db, const char *info,
int event_type) {
switch (event_type) {
@@ -570,7 +570,7 @@ array_header *proxy_db_exec_prepared_stmt(pool *p, struct proxy_dbh *dbh,
current_schema = dbh->schema;
/* The sqlit3_stmt_readonly() function first appeared in SQLite 3.7.x. */
-#ifdef HAVE_SQLITE3_STMT_READONLY
+#if defined(HAVE_SQLITE3_STMT_READONLY)
readonly = sqlite3_stmt_readonly(pstmt);
#else
readonly = FALSE;
@@ -695,7 +695,7 @@ struct proxy_dbh *proxy_db_open(pool *p, const char *table_path,
schema_name, table_path);
flags = SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE;
-#ifdef SQLITE_OPEN_PRIVATECACHE
+#if defined(SQLITE_OPEN_PRIVATECACHE)
/* By default, disable the shared cache mode. */
flags |= SQLITE_OPEN_PRIVATECACHE;
#endif
=====================================
mod_proxy.h.in
=====================================
@@ -1,6 +1,6 @@
/*
* ProFTPD - mod_proxy
- * Copyright (c) 2012-2023 TJ Saunders
+ * Copyright (c) 2012-2024 TJ Saunders
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
@@ -88,7 +88,7 @@
/* Define if you have the strnstr(3) function. */
#undef HAVE_STRNSTR
-#define MOD_PROXY_VERSION "mod_proxy/0.9.3"
+#define MOD_PROXY_VERSION "mod_proxy/0.9.4"
/* Make sure the version of proftpd is as necessary. */
#if PROFTPD_VERSION_NUMBER < 0x0001030706
=====================================
mod_proxy.html
=====================================
@@ -681,7 +681,7 @@ The currently supported policies are:
<li><code>Shuffle</code>
<p>
Similar to the <code>Random</code> policy, except the selection happens
- from the not-yet-chosed backend servers. This means that <b>all</b>
+ from the not-yet-chosen backend servers. This means that <b>all</b>
backend servers will eventually be used evenly, just in a random order.
</li>
</ul>
@@ -2280,6 +2280,21 @@ custom <code>LogFormats</code> and/or custom SQL statements:
<li><code>%{note:mod_proxy.backend-url}</code>: URL to the backend/proxied server
</ul>
+<p><a name="SELinux"></a>
+<b>SELinux</b><br>
+If using the <code>mod_proxy</code> module on an SELinux-enabled system, you
+may need the following to allow for proper operations of proxying. For example,
+using the following <code>mod_proxy</code> configuration:
+<pre>
+ ProxyTables /var/ftp/proxy
+</pre>
+may require that you run:
+<pre>
+ $ semanage fcontext --add --type public_content_rw_t '/var/ftp/proxy(/.*)?'
+ $ setsebool -P ftpd_anon_write=1
+ $ setsebool -P nis_enabled=1
+</pre>
+
<p><a name="Wishlist"></a>
<b>Suggested Future Features</b><br>
The following lists the features I hope to add to <code>mod_proxy</code>,
@@ -2531,7 +2546,7 @@ development library/header files <b>must</b> be installed for building
<hr>
<font size=2><b><i>
-© Copyright 2015-2023 TJ Saunders<br>
+© Copyright 2015-2024 TJ Saunders<br>
All Rights Reserved<br>
</i></b></font>
View it on GitLab: https://salsa.debian.org/debian-proftpd-team/proftpd-mod-proxy/-/commit/fde85fec3f20a2eb3ff9d3d125049bc2bb42b778
--
View it on GitLab: https://salsa.debian.org/debian-proftpd-team/proftpd-mod-proxy/-/commit/fde85fec3f20a2eb3ff9d3d125049bc2bb42b778
You're receiving this email because of your account on salsa.debian.org.
More information about the Pkg-proftpd-maintainers
mailing list