- Email: office@safebyte.io
- Bucharest, ROU
Article by: Teodor Lupan
At Safebyte Consulting we often integrate Linux systems into enterprise VPN infrastructures as part of our security assignments like security audits or pentests. During a recent project connecting Kali Linux (strongSwan) to FortiGate gateways that enforce PSK + XAuth + TOTP (2FA), we implemented a minimal strongSwan patch to handle FortiGate’s two-step XAuth flow (username/password, then TOTP).
While stabilizing authentication and MTU/offload issues, we discovered a secondary — and operationally important — behaviour: Windows FortiClient configurations commonly push a default route through the VPN, but with our strongSwan setup we could instead configure client-side routing to achieve split-tunnel behaviour. This revealed a practical attack surface and an operational risk for organizations relying on strict full-tunnel enforcement from the gateway.
This article documents the full engineering story: diagnosis, design rationale, the patch to xauth_generic.c, build and test steps, OTP automation, routing discovery and risk analysis, and mitigations. Annexes contain the exact patch, helper scripts, and sample configs.
Most FortiGate VPNs used in enterprise environments rely on IKEv1 with XAuth for user authentication.
When 2FA is enabled, the authentication sequence becomes:
Unfortunately, stock strongSwan expects either:
As a result, attempts to connect a Linux VM via strongSwan fail after the first phase:
<22> charon: 12[IKE] XAuth request received: XAUTH_USER_NAME, XAUTH_USER_PASSWORD <22> charon: 12[IKE] XAuth authentication started <22> charon: 12[IKE] XAuth prompt: TOTP: <22> charon: 12[IKE] no credentials found for TOTP prompt <22> charon: 12[IKE] XAuth authentication failed
How FortiGate’s two-step XAuth works and why strongSwan failed
[IKE] XAuth request: XAUTH_USER_NAME, XAUTH_USER_PASSWORD
[IKE] XAuth request: XAUTH_USER_PASSWORD (prompt: "TOTP:")
Important points:
XAUTH_USER_PASSWORD) for the second phase but changes the prompt to indicate TOTP.Stock xauth_generic lacks logic to detect and respond to this second prompt non-interactively — it typically replies once with the stored password and then fails.
Before debugging authentication, we verified the network path between our test VM and the FortiGate gateway.
Fragmented IKE or ESP packets can cause erratic behaviour.
We disabled all offloads and fixed MTU:
ethtool -K eth0 tso off gso off gro off
ip link set dev eth0 mtu 1400
FortiGate’s XAuth dialogue looks like this (captured from charon debug):
<IKE> XAuth request received:
XAUTH_USER_NAME: 'Username:'
XAUTH_USER_PASSWORD: 'Password:'
<IKE> XAuth request received:
XAUTH_USER_PASSWORD: 'TOTP:'
That second message is the crucial difference: FortiGate reuses the same attribute type (XAUTH_USER_PASSWORD) but changes the prompt string to “TOTP:”.
If the client simply replies again with the OTP digits, authentication succeeds.
The stock xauth_generic plugin has no logic to detect this — it tries to reuse the stored password, leading to failure.
Before writing the patch, we tested every “official” or “clever” way to make strongSwan handle FortiGate’s two-step XAuth + TOTP flow without modifying source code.
All of them failed in headless mode for structural reasons.
At first glance, this seems simple: when FortiGate asks for “TOTP:”, strongSwan could just prompt the user.
However, the reality is that when using the normal ipsec starter / charon daemon model, there is no TTY for the process to ask on.ipsec up <conn> only sends commands to the daemon via the control socket; the daemon cannot prompt on stdin.
We tried several combinations, including launching ipsec up directly from a shell and expecting an OTP prompt — none appeared.
That behavior is expected: the only strongSwan components capable of prompting a human are:
In other words, the interactive approach only works for desktop users, not for automated or headless deployments.
expect, oathtool, or pre-writing /etc/ipsec.otp)Another idea was to run strongSwan under expect, catch the “Password:” or “TOTP:” prompts, and feed them automatically.
That works for programs which actually print to stdout and read from stdin — but charon doesn’t.
The strongSwan starter talks to charon via control sockets (stroke/VICI), not via the terminal, so there is no prompt to intercept.
Even if an expect script tried to read from the process, it would see nothing; the exchange happens internally inside the daemon.
We also tested the trick of writing the OTP into a file before the second phase (e.g., /etc/ipsec.otp), hoping the plugin might read it.
Stock xauth_generic ignores any external file — it simply reuses the static password string from configuration, leading to authentication failure.
Other theoretical options, like xauth_pam or using the VICI API to inject attributes mid-exchange, do not apply either:
xauth_pam is used when strongSwan acts as the XAuth server, not as the client.XAUTH_USER_PASSWORD attribute in IKEv1; it cannot push an OTP dynamically.Because none of the above could supply a second-phase credential non-interactively, the only clean and reliable solution was to extend the existing plugin itself.
By adding logic in xauth_generic to detect "TOTP:" and fetch the code from either a file or a helper script, we kept the entire exchange inside charon’s normal state machine — no external I/O, no race conditions, and full headless support.
We extended process_request() insrc/libcharon/plugins/xauth_generic/xauth_generic.c.
Detection:
If the prompt string contains "TOTP", the plugin executes a helper to obtain the OTP and sets that as the response value for the current attribute.
OTP source:
/etc/ipsec.otp — static file (can be written by automation when OTP received via SMS), or/usr/local/sbin/ipsec-totp.sh — dynamic generator using oathtool.Security:
Result:
Transparent two-step XAuth authentication on Linux.
We inserted detection logic insidesrc/libcharon/plugins/xauth_generic/xauth_generic.c → process_request().
Flow:
XAUTH_USER_PASSWORD, check the prompt string."TOTP", fetch the OTP (file or script).Two sources supported:
/etc/ipsec.otp (static, e.g. SMS-based)/usr/local/sbin/ipsec-totp.sh (dynamic, uses oathtool)Security: files are root:root 0600; OTP lives only in memory briefly; debug never prints secrets.
apt install build-essential libgmp-dev libssl-dev libsystemd-dev \
flex bison git autoconf libtool pkg-config oathtool
git clone https://github.com/strongswan/strongswan.git
cd strongswan
git checkout 5.9.13
git apply /path/to/0001-xauth-totp.patch
./autogen.sh
./configure --prefix=/usr --sysconfdir=/etc \
--enable-unity --enable-xauth-generic \
--enable-openssl --enable-systemd
make -j$(nproc)
make install
Use the examples in Annex B–C, then reload ipsec:
ipsec restartipsec up fortigate-base
[IKE] XAuth request: Username/Password
[IKE] credentials accepted
[IKE] XAuth request: TOTP:
[xauth-totp] sent OTP in response to TOTP challenge
[IKE] XAuth authentication of 'user1' successful
[CFG] CHILD_SA fortigate-internal{1} established/etc/ipsec.conf (simplified):
config setup
charondebug="ike 2, knl 2, cfg 2"
uniqueids=yes
conn %default
keyexchange=ikev1
authby=xauthpsk
xauth=client
xauth_identity=user1
left=%defaultroute
leftsourceip=%config
right=<fortigate_public_ip>
rightid=@FGT
ike=aes256-sha256-modp1024!
esp=aes256-sha256!
ikelifetime=8h
keylife=1h
aggressive=yes
modecfgpull=yes
dpdaction=restart
dpddelay=30
dpdtimeout=120
conn fortigate-internal
also=%default
rightsubnet=10.0.0.0/8
auto=add
/etc/strongswan.conf:
charon {
install_routes = yes
load_modular = yes
plugins {
xauth-generic {
enable = yes
}
}
filelog {
/var/log/charon.log {
time_format = %b %e %T
ike_name = yes
append = yes
default = 2
flush_line = yes
}
}
}
/usr/local/sbin/ipsec-totp.sh:
#!/usr/bin/env bash
# Safebyte Consulting - TOTP helper for strongSwan
set -euo pipefail
CONN="fortigate-base"
OTP_FILE="/etc/ipsec.otp"
SECRET_FILE="/root/.totp_base32"
IPSEC="/usr/local/sbin/ipsec"
# 1) verificări rapide
if ! command -v oathtool >/dev/null 2>&1; then
echo "Error: oathtool is not installed. Install it with: sudo apt-get install -y oathtool" >&2
exit 1
fi
if [[ ! -r "$SECRET_FILE" ]]; then
echo "Error: missing $SECRET_FILE (key TOTP base32). Create it with 600 permissions." >&2
exit 2
fi
# 2) OTP curent (fără newline)
SECRET=$(tr -d '\r\n ' < "$SECRET_FILE")
OTP=$(oathtool --totp -b "$SECRET")
printf "%s" "$OTP" | sudo tee "$OTP_FILE" >/dev/null
sudo chmod 600 "$OTP_FILE"
# 3) inițiază conexiunea
$IPSEC up "$CONN" || { echo "ipsec up failed"; exit 3; }
# 4) mică așteptare + verificare
sleep 3
if $IPSEC statusall | grep -qE "$CONN\{[0-9]+\}.*ESTABLISHED|$CONN.*CHILD_SA.*established"; then
echo "✅ $CONN is up. Cleaning OTP."
sudo shred -u "$OTP_FILE" || sudo rm -f "$OTP_FILE"
exit 0
else
echo "⚠️ $CONN is not up yet. Leaving OTP for rapid retry (expiring in ~30s)." >&2
exit 4
fi
Permissions:
chmod 700 /usr/local/sbin/ipsec-totp.sh
chmod 600 /etc/ipsec.secret.totp
If FortiGate sends OTP via SMS, write it to /etc/ipsec.otp before running ipsec up.
Troubleshooting checklist
| Symptom | Likely cause | Fix |
|---|---|---|
XAuth authentication failed after TOTP | wrong OTP/time skew | verify date sync and compare with phone |
NO_PROPOSAL_CHOSEN | mismatched IKE/ESP ciphers | align ike= and esp= with FortiGate settings |
| Tunnel up but no traffic | routes not installed | set install_routes=yes |
| IKE timeout / fragments | MTU too high | lower to 1400, disable offloads |
| ESP packets blocked | NAT-T or firewall | check UDP 4500 open, use tcpdump -n -i eth0 esp or port 4500 |
Example statusall after success:
Status of IKE charon daemon (strongSwan 5.9.13):
IKEv1 Connections:
fortigate-internal[1]: ESTABLISHED, IKEv1
IKE SPIs: 07cda3fa8d...
XAuth authentication of 'user1' successful
CHILD_SA fortigate-internal{1} INSTALLED
10.0.0.0/8 === 192.168.10.0/24
While validating routing we compared clients:
0.0.0.0/0 as default route → all traffic through VPN (full tunnel).rightsubnet=. Local routes remain → split-tunnel.If policy assumes full tunneling, a user or attacker could:
/etc/ipsec.secret.totp and /etc/ipsec.otp must be root:root 0600.charondebug at low levels in production to avoid leaking prompts.The patch enables strongSwan to interoperate fully with FortiGate’s two-step XAuth + TOTP flow — non-interactive, secure, and automation-friendly.
Equally important, our routing analysis revealed that strongSwan clients can unintentionally operate in split-tunnel mode even when FortiGate expects full-tunnel enforcement — a subtle but serious policy gap.
Key takeaways:
Annex A — xauth_generic.c — Safebyte TOTP variant
/*
* Copyright (C) 2011 Tobias Brunner
* HSR Hochschule fuer Technik Rapperswil
*ee
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the License, or (at your
* option) any later version. See <http://www.fsf.org/copyleft/gpl.txt>.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
* or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
* for more details.
* xauth_generic.c — Safebyte TOTP variant (verbatim)
*/
#include "xauth_generic.h"
#include <daemon.h>
#include <library.h>
#include <stdio.h>
#include <sys/stat.h>
/* citește OTP din /etc/ipsec.otp, fără newline */
static int read_otp_file(const char *path, char *buf, size_t buflen)
{
FILE *f = fopen(path, "r");
if (!f) return -1;
size_t r = fread(buf, 1, buflen - 1, f);
fclose(f);
while (r > 0 && (buf[r-1] == '\n' || buf[r-1] == '\r')) r--;
buf[r] = '\0';
return (int)r;
}
typedef struct private_xauth_generic_t private_xauth_generic_t;
/**
* Private data of an xauth_generic_t object.
*/
struct private_xauth_generic_t {
/**
* Public interface.
*/
xauth_generic_t public;
/**
* ID of the server
*/
identification_t *server;
/**
* ID of the peer
*/
identification_t *peer;
};
METHOD(xauth_method_t, initiate_peer, status_t,
private_xauth_generic_t *this, cp_payload_t **out)
{
/* peer never initiates */
return FAILED;
}
METHOD(xauth_method_t, process_peer, status_t,
private_xauth_generic_t *this, cp_payload_t *in, cp_payload_t **out)
{
configuration_attribute_t *attr;
enumerator_t *enumerator;
shared_key_t *shared;
cp_payload_t *cp;
chunk_t msg;
bool totp_challenge = FALSE;
char otpbuf[128] = {0};
enumerator = in->create_attribute_enumerator(in);
while (enumerator->enumerate(enumerator, &attr))
{
if (attr->get_type(attr) == XAUTH_MESSAGE)
{
chunk_printable(attr->get_chunk(attr), &msg, '?');
DBG1(DBG_CFG, "XAuth message: %.*s", (int)msg.len, msg.ptr);
/* Detectează challenge TOTP: */
if (msg.len >= 4)
{
/* compară prefixul "TOTP" case-insensitive */
size_t n = msg.len > 4 ? 4 : msg.len;
char head[5] = {0};
memcpy(head, msg.ptr, n);
for (size_t i = 0; i < n; i++)
{
if (head[i] >= 'a' && head[i] <= 'z') head[i] -= 32;
}
if (memcmp(head, "TOTP", 4) == 0)
{
if (read_otp_file("/etc/ipsec.otp", otpbuf, sizeof(otpbuf)) > 0)
{
totp_challenge = TRUE;
DBG1(DBG_IKE, "xauth-totp: OTP loaded from /etc/ipsec.otp");
}
else
{
DBG1(DBG_IKE, "xauth-totp: failed to read /etc/ipsec.otp");
}
}
}
free(msg.ptr);
}
}
enumerator->destroy(enumerator);
cp = cp_payload_create_type(PLV1_CONFIGURATION, CFG_REPLY);
enumerator = in->create_attribute_enumerator(in);
while (enumerator->enumerate(enumerator, &attr))
{
shared_key_type_t type = SHARED_EAP;
switch (attr->get_type(attr))
{
case XAUTH_USER_NAME:
cp->add_attribute(cp, configuration_attribute_create_chunk(
PLV1_CONFIGURATION_ATTRIBUTE, XAUTH_USER_NAME,
this->peer->get_encoding(this->peer)));
break;
case XAUTH_NEXT_PIN:
type = SHARED_PIN;
/* FALL */
case XAUTH_USER_PASSWORD:
if (totp_challenge && otpbuf[0] != '\0')
{
chunk_t otp = chunk_create(otpbuf, strlen(otpbuf));
cp->add_attribute(cp, configuration_attribute_create_chunk(
PLV1_CONFIGURATION_ATTRIBUTE, XAUTH_USER_PASSWORD, otp));
DBG1(DBG_IKE, "xauth-totp: injected OTP into XAUTH_USER_PASSWORD");
}
else
{
shared = lib->credmgr->get_shared(lib->credmgr, type,
this->peer, this->server);
if (!shared)
{
DBG1(DBG_IKE, "no XAuth %s found for '%Y' - '%Y'",
type == SHARED_EAP ? "password" : "PIN",
this->peer, this->server);
enumerator->destroy(enumerator);
cp->destroy(cp);
return FAILED;
}
cp->add_attribute(cp, configuration_attribute_create_chunk(
PLV1_CONFIGURATION_ATTRIBUTE, attr->get_type(attr),
shared->get_key(shared)));
shared->destroy(shared);
}
break;
default:
break;
}
}
enumerator->destroy(enumerator);
*out = cp;
return NEED_MORE;
}
METHOD(xauth_method_t, initiate_server, status_t,
private_xauth_generic_t *this, cp_payload_t **out)
{
cp_payload_t *cp;
cp = cp_payload_create_type(PLV1_CONFIGURATION, CFG_REQUEST);
cp->add_attribute(cp, configuration_attribute_create_chunk(
PLV1_CONFIGURATION_ATTRIBUTE, XAUTH_USER_NAME, chunk_empty));
cp->add_attribute(cp, configuration_attribute_create_chunk(
PLV1_CONFIGURATION_ATTRIBUTE, XAUTH_USER_PASSWORD, chunk_empty));
*out = cp;
return NEED_MORE;
}
METHOD(xauth_method_t, process_server, status_t,
private_xauth_generic_t *this, cp_payload_t *in, cp_payload_t **out)
{
configuration_attribute_t *attr;
enumerator_t *enumerator;
shared_key_t *shared;
identification_t *id;
chunk_t user = chunk_empty, pass = chunk_empty;
status_t status = FAILED;
int tried = 0;
enumerator = in->create_attribute_enumerator(in);
while (enumerator->enumerate(enumerator, &attr))
{
switch (attr->get_type(attr))
{
case XAUTH_USER_NAME:
user = attr->get_chunk(attr);
break;
case XAUTH_USER_PASSWORD:
pass = attr->get_chunk(attr);
break;
default:
break;
}
}
enumerator->destroy(enumerator);
if (!user.ptr || !pass.ptr)
{
DBG1(DBG_IKE, "peer did not respond to our XAuth request");
return FAILED;
}
if (user.len)
{
id = identification_create_from_data(user);
if (!id)
{
DBG1(DBG_IKE, "failed to parse provided XAuth username");
return FAILED;
}
this->peer->destroy(this->peer);
this->peer = id;
}
if (pass.len && pass.ptr[pass.len - 1] == 0)
{ /* fix null-terminated passwords (Android etc.) */
pass.len -= 1;
}
enumerator = lib->credmgr->create_shared_enumerator(lib->credmgr,
SHARED_EAP, this->server, this->peer);
while (enumerator->enumerate(enumerator, &shared, NULL, NULL))
{
if (chunk_equals_const(shared->get_key(shared), pass))
{
status = SUCCESS;
break;
}
tried++;
}
enumerator->destroy(enumerator);
if (status != SUCCESS)
{
if (!tried)
{
DBG1(DBG_IKE, "no XAuth secret found for '%Y' - '%Y'",
this->server, this->peer);
}
else
{
DBG1(DBG_IKE, "none of %d found XAuth secrets for '%Y' - '%Y' "
"matched", tried, this->server, this->peer);
}
}
return status;
}
METHOD(xauth_method_t, get_identity, identification_t*,
private_xauth_generic_t *this)
{
return this->peer;
}
METHOD(xauth_method_t, destroy, void,
private_xauth_generic_t *this)
{
this->server->destroy(this->server);
this->peer->destroy(this->peer);
free(this);
}
/*
* Described in header.
*/
xauth_generic_t *xauth_generic_create_peer(identification_t *server,
identification_t *peer,
char *profile)
{
private_xauth_generic_t *this;
INIT(this,
.public = {
.xauth_method = {
.initiate = _initiate_peer,
.process = _process_peer,
.get_identity = _get_identity,
.destroy = _destroy,
},
},
.server = server->clone(server),
.peer = peer->clone(peer),
);
return &this->public;
}
/*
* Described in header.
*/
xauth_generic_t *xauth_generic_create_server(identification_t *server,
identification_t *peer,
char *profile)
{
private_xauth_generic_t *this;
INIT(this,
.public = {
.xauth_method = {
.initiate = _initiate_server,
.process = _process_server,
.get_identity = _get_identity,
.destroy = _destroy,
},
},
.server = server->clone(server),
.peer = peer->clone(peer),
);
return &this->public;
}
When you don’t want a full default route but target a split tunnel setup instead, define additional child connections that reuse a base IKE SA:
conn fortigate-base
keyexchange=ikev1
authby=xauthpsk
xauth=client
xauth_identity=user1
left=%defaultroute
leftsourceip=%config
right=<REDACTED_FGT_IP>
rightid=@FGT
ike=aes256-sha256-modp1024!
esp=aes256-sha256!
aggressive=yes
modecfgpull=yes
dpdaction=restart
auto=add
# Internal networks
conn fortigate-10-0-0-0_8
also=fortigate-base
rightsubnet=10.0.0.0/8
auto=add
conn fortigate-172-16-0-0_12
also=fortigate-base
rightsubnet=172.16.0.0/12
auto=add
conn fortigate-192-168-100-0_24
also=fortigate-base
rightsubnet=192.168.100.0/24
auto=add
Tip: keep charon { install_routes = yes } in strongswan.conf so the kernel routes are installed automatically for each CHILD_SA.
Compiling latest strongswan 5.9.x on a Debian bullseye works out of the box – but on GNU gcc 15 (kali rolling at the date of this article) is a different challenge – if someone wants full patches contact me at teodor.lupan[at]safebyte.io .
Recent Comments