[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnurl] 299/411: hsts: add support for Strict-Transport-Security
|
From: |
gnunet |
|
Subject: |
[gnurl] 299/411: hsts: add support for Strict-Transport-Security |
|
Date: |
Wed, 13 Jan 2021 01:21:54 +0100 |
This is an automated email from the git hooks/post-receive script.
nikita pushed a commit to branch master
in repository gnurl.
commit 7385610d0c74c6a254fea5e4cd6e1d559d848c8c
Author: Daniel Stenberg <daniel@haxx.se>
AuthorDate: Mon Nov 2 23:17:01 2020 +0100
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle
Closes #5896
---
configure.ac | 28 +++
docs/EXPERIMENTAL.md | 2 +
docs/HSTS.md | 44 ++++
docs/Makefile.am | 1 +
docs/cmdline-opts/Makefile.inc | 1 +
docs/cmdline-opts/hsts.d | 18 ++
docs/cmdline-opts/version.d | 2 +
docs/libcurl/curl_easy_setopt.3 | 4 +
docs/libcurl/curl_version_info.3 | 3 +
docs/libcurl/opts/CURLOPT_HSTS.3 | 66 ++++++
docs/libcurl/opts/CURLOPT_HSTS_CTRL.3 | 73 ++++++
docs/libcurl/opts/Makefile.inc | 2 +
docs/libcurl/symbols-in-versions | 5 +
docs/options-in-versions | 1 +
include/curl/curl.h | 10 +
lib/Makefile.inc | 4 +-
lib/curl_get_line.c | 3 +-
lib/easy.c | 11 +
lib/easyoptions.c | 4 +-
lib/hsts.c | 430 ++++++++++++++++++++++++++++++++++
lib/hsts.h | 60 +++++
lib/http.c | 18 ++
lib/setopt.c | 28 +++
lib/url.c | 21 +-
lib/urldata.h | 14 +-
lib/version.c | 3 +
src/tool_cfgable.c | 1 +
src/tool_cfgable.h | 1 +
src/tool_getparam.c | 7 +
src/tool_help.c | 4 +
src/tool_operate.c | 3 +
src/tool_setopt.c | 5 +
src/tool_setopt.h | 2 +
tests/data/Makefile.inc | 1 +
tests/data/test1660 | 81 +++++++
tests/runtests.pl | 5 +
tests/unit/Makefile.inc | 5 +-
tests/unit/unit1660.c | 172 ++++++++++++++
38 files changed, 1122 insertions(+), 21 deletions(-)
diff --git a/configure.ac b/configure.ac
index a81d96a39..d60ccc14e 100755
--- a/configure.ac
+++ b/configure.ac
@@ -4882,6 +4882,31 @@ AC_HELP_STRING([--disable-alt-svc],[Disable alt-svc
support]),
AC_MSG_RESULT(no)
)
+dnl ************************************************************
+dnl switch on/off hsts
+dnl
+curl_hsts_msg="no (--enable-hsts)";
+AC_MSG_CHECKING([whether to support HSTS])
+AC_ARG_ENABLE(hsts,
+AC_HELP_STRING([--enable-hsts],[Enable HSTS support])
+AC_HELP_STRING([--disable-hsts],[Disable HSTS support]),
+[ case "$enableval" in
+ no)
+ AC_MSG_RESULT(no)
+ ;;
+ *) AC_MSG_RESULT(yes)
+ curl_hsts_msg="enabled";
+ enable_hsts="yes"
+ ;;
+ esac ],
+ AC_MSG_RESULT(no)
+)
+
+if test "$enable_hsts" = "yes"; then
+ AC_DEFINE(USE_HSTS, 1, [to enable HSTS])
+ experimental="$experimental HSTS"
+fi
+
dnl *************************************************************
dnl check whether ECH support, if desired, is actually available
dnl
@@ -4998,6 +5023,9 @@ fi
if test "x$enable_altsvc" = "xyes"; then
SUPPORT_FEATURES="$SUPPORT_FEATURES alt-svc"
fi
+if test "x$enable_hsts" = "xyes"; then
+ SUPPORT_FEATURES="$SUPPORT_FEATURES HSTS"
+fi
if test "x$CURL_DISABLE_CRYPTO_AUTH" != "x1" -a \
\( "x$HAVE_GSSAPI" = "x1" -o "x$USE_WINDOWS_SSPI" = "x1" \); then
diff --git a/docs/EXPERIMENTAL.md b/docs/EXPERIMENTAL.md
index ee5898944..5b2d36c14 100644
--- a/docs/EXPERIMENTAL.md
+++ b/docs/EXPERIMENTAL.md
@@ -20,3 +20,5 @@ Experimental support in curl means:
- HTTP/3 support and options
- CURLSSLOPT_NATIVE_CA (No configure option, feature built in when supported)
+ - HSTS support and options
+
diff --git a/docs/HSTS.md b/docs/HSTS.md
new file mode 100644
index 000000000..c3f08393c
--- /dev/null
+++ b/docs/HSTS.md
@@ -0,0 +1,44 @@
+# HSTS support
+
+curl features **EXPERIMENTAL** support for the Strict-Transport-Security: HTTP
+header. Added in curl 7.74.0
+
+## Standard
+
+[HTTP Strict Transport Security](https://tools.ietf.org/html/rfc6797)
+
+## Behavior
+
+libcurl features an in-memory cache for HSTS hosts, so that subsequent
+HTTP-only requests to a host name present in the cache will get internally
+"redirected" to the HTTPS version.
+
+## `curl_easy_setopt()` options:
+
+ - `CURLOPT_HSTS_CTRL` - enable HSTS for this easy handle
+ - `CURLOPT_HSTS` - specify file name where to store the HSTS cache on close
+ (and possibly read from at startup)
+
+## curl cmdline options
+
+ - `--hsts [filename]` - enable HSTS, use the file as HSTS cache. If filename
+ is `""` (no length) then no file will be used, only in-memory cache.
+
+## HSTS cache file format
+
+Lines starting with `#` are ignored.
+
+For each hsts entry:
+
+ [host name] "YYYYMMDD HH:MM:SS"
+
+The `[host name]` is dot-prefixed if it is a includeSubDomain.
+
+The time stamp is when the entry expires.
+
+I considered using wget's file format for the HSTS cache. However, they store
the time stamp as the epoch (number of seconds since 1970) and I strongly
disagree with using that format. Instead I opted to use a format similar to the
curl alt-svc cache file format.
+
+## Possible future additions
+
+ - `CURLOPT_HSTS_PRELOAD` - provide a set of preloaded HSTS host names
+ - ability to save to something else than a file
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 9d67084d9..4a1984010 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -63,6 +63,7 @@ EXTRA_DIST = \
GOVERNANCE.md \
HELP-US.md \
HISTORY.md \
+ HSTS.md \
HTTP-COOKIES.md \
HTTP2.md \
HTTP3.md \
diff --git a/docs/cmdline-opts/Makefile.inc b/docs/cmdline-opts/Makefile.inc
index 792cadb3c..7e8529c1a 100644
--- a/docs/cmdline-opts/Makefile.inc
+++ b/docs/cmdline-opts/Makefile.inc
@@ -85,6 +85,7 @@ DPAGES = \
head.d header.d \
help.d \
hostpubmd5.d \
+ hsts.d \
http0.9.d \
http1.0.d \
http1.1.d http2.d \
diff --git a/docs/cmdline-opts/hsts.d b/docs/cmdline-opts/hsts.d
new file mode 100644
index 000000000..2399084bf
--- /dev/null
+++ b/docs/cmdline-opts/hsts.d
@@ -0,0 +1,18 @@
+Long: hsts
+Arg: <file name>
+Protocols: HTTPS
+Help: Enable HSTS with this cache file
+Added: 7.74.0
+Category: http
+---
+WARNING: this option is experimental. Do not use in production.
+
+This option enables HSTS for the transfer. If the file name points to an
+existing HSTS cache file, that will be used. After a completed transfer, the
+cache will be saved to the file name again if it has been modified.
+
+Specify a "" file name (zero length) to avoid loading/saving and make curl
+just handle HSTS in memory.
+
+If this option is used several times, curl will load contents from all the
+files but the last one will be used for saving.
diff --git a/docs/cmdline-opts/version.d b/docs/cmdline-opts/version.d
index 52c29f177..f6c091707 100644
--- a/docs/cmdline-opts/version.d
+++ b/docs/cmdline-opts/version.d
@@ -28,6 +28,8 @@ This curl uses a libcurl built with Debug. This enables more
error-tracking
and memory debugging etc. For curl-developers only!
.IP "GSS-API"
GSS-API is supported.
+.IP "HSTS"
+HSTS support is present.
.IP "HTTP2"
HTTP/2 support has been built-in.
.IP "HTTP3"
diff --git a/docs/libcurl/curl_easy_setopt.3 b/docs/libcurl/curl_easy_setopt.3
index 362cfef0a..3434b158c 100644
--- a/docs/libcurl/curl_easy_setopt.3
+++ b/docs/libcurl/curl_easy_setopt.3
@@ -319,6 +319,10 @@ Add or control cookies. See \fICURLOPT_COOKIELIST(3)\fP
Specify the Alt-Svc: cache file name. See \fICURLOPT_ALTSVC(3)\fP
.IP CURLOPT_ALTSVC_CTRL
Enable and configure Alt-Svc: treatment. See \fICURLOPT_ALTSVC_CTRL(3)\fP
+.IP CURLOPT_HSTS
+Set HSTS cache file. See \fICURLOPT_HSTS(3)\fP
+.IP CURLOPT_HSTS_CTRL
+Enable HSTS. See \fICURLOPT_HSTS_CTRL(3)\fP
.IP CURLOPT_HTTPGET
Do an HTTP GET request. See \fICURLOPT_HTTPGET(3)\fP
.IP CURLOPT_REQUEST_TARGET
diff --git a/docs/libcurl/curl_version_info.3 b/docs/libcurl/curl_version_info.3
index 5c5f16a3d..a6fa5e3b9 100644
--- a/docs/libcurl/curl_version_info.3
+++ b/docs/libcurl/curl_version_info.3
@@ -143,6 +143,9 @@ to use the current user credentials without the app having
to pass them on.
(Added in 7.38.0)
.IP CURL_VERSION_GSSNEGOTIATE
supports HTTP GSS-Negotiate (added in 7.10.6)
+.IP CURL_VERSION_HSTS
+libcurl was built with support for HSTS (HTTP Strict Transport Security)
+(Added in 7.74.0)
.IP CURL_VERSION_HTTPS_PROXY
libcurl was built with support for HTTPS-proxy.
(Added in 7.52.0)
diff --git a/docs/libcurl/opts/CURLOPT_HSTS.3 b/docs/libcurl/opts/CURLOPT_HSTS.3
new file mode 100644
index 000000000..b90dbff8b
--- /dev/null
+++ b/docs/libcurl/opts/CURLOPT_HSTS.3
@@ -0,0 +1,66 @@
+.\" **************************************************************************
+.\" * _ _ ____ _
+.\" * Project ___| | | | _ \| |
+.\" * / __| | | | |_) | |
+.\" * | (__| |_| | _ <| |___
+.\" * \___|\___/|_| \_\_____|
+.\" *
+.\" * Copyright (C) 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
+.\" *
+.\" * This software is licensed as described in the file COPYING, which
+.\" * you should have received as part of this distribution. The terms
+.\" * are also available at https://curl.haxx.se/docs/copyright.html.
+.\" *
+.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+.\" * copies of the Software, and permit persons to whom the Software is
+.\" * furnished to do so, under the terms of the COPYING file.
+.\" *
+.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+.\" * KIND, either express or implied.
+.\" *
+.\" **************************************************************************
+.\"
+.TH CURLOPT_HSTS 3 "5 Feb 2019" "libcurl 7.74.0" "curl_easy_setopt options"
+.SH NAME
+CURLOPT_HSTS \- set HSTS cache file name
+.SH SYNOPSIS
+.nf
+#include <curl/curl.h>
+
+CURLcode curl_easy_setopt(CURL *handle, CURLOPT_HSTS, char *filename);
+.fi
+.SH EXPERIMENTAL
+Warning: this feature is early code and is marked as experimental. It can only
+be enabled by explicitly telling configure with \fB--enable-hsts\fP. You are
+advised to not ship this in production before the experimental label is
+removed.
+.SH DESCRIPTION
+Make the \fIfilename\fP point to a file name to load an existing HSTS cache
+from, and to store the cache in when the easy handle is closed. Setting a file
+name with this option will also enable HSTS for this handle (the equivalent of
+setting \fICURLHSTS_ENABLE\fP with \fICURLOPT_HSTS_CTRL(3)\fP).
+
+If the given file does not exist or contains no HSTS entries at startup, the
+HSTS cache will simply start empty. Setting the file name to NULL or "" will
+only enable HSTS without reading from or writing to any file.
+
+If this option is set multiple times, libcurl will load cache entries from
+each given file but will only store the last used name for later writing.
+.SH DEFAULT
+NULL, no file name
+.SH PROTOCOLS
+HTTPS and HTTP
+.SH EXAMPLE
+.nf
+CURL *curl = curl_easy_init();
+if(curl) {
+ curl_easy_setopt(curl, CURLOPT_HSTS, "/home/user/.hsts-cache");
+ curl_easy_perform(curl);
+}
+.fi
+.SH AVAILABILITY
+Added in 7.74.0
+.SH RETURN VALUE
+Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if not.
+.SH "SEE ALSO"
+.BR CURLOPT_HSTS_CTRL "(3), " CURLOPT_ALTSVC "(3), " CURLOPT_RESOLVE "(3), "
diff --git a/docs/libcurl/opts/CURLOPT_HSTS_CTRL.3
b/docs/libcurl/opts/CURLOPT_HSTS_CTRL.3
new file mode 100644
index 000000000..ffdfee93f
--- /dev/null
+++ b/docs/libcurl/opts/CURLOPT_HSTS_CTRL.3
@@ -0,0 +1,73 @@
+.\" **************************************************************************
+.\" * _ _ ____ _
+.\" * Project ___| | | | _ \| |
+.\" * / __| | | | |_) | |
+.\" * | (__| |_| | _ <| |___
+.\" * \___|\___/|_| \_\_____|
+.\" *
+.\" * Copyright (C) 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
+.\" *
+.\" * This software is licensed as described in the file COPYING, which
+.\" * you should have received as part of this distribution. The terms
+.\" * are also available at https://curl.haxx.se/docs/copyright.html.
+.\" *
+.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+.\" * copies of the Software, and permit persons to whom the Software is
+.\" * furnished to do so, under the terms of the COPYING file.
+.\" *
+.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+.\" * KIND, either express or implied.
+.\" *
+.\" **************************************************************************
+.\"
+.TH CURLOPT_HSTS_CTRL 3 "4 Sep 2020" "libcurl 7.74.0" "curl_easy_setopt
options"
+.SH NAME
+CURLOPT_HSTS_CTRL \- control HSTS behavior
+.SH SYNOPSIS
+.nf
+#include <curl/curl.h>
+
+#define CURLHSTS_ENABLE (1<<0)
+#define CURLHSTS_READONLYFILE (1<<1)
+
+CURLcode curl_easy_setopt(CURL *handle, CURLOPT_HSTS_CTRL, long bitmask);
+.fi
+.SH EXPERIMENTAL
+Warning: this feature is early code and is marked as experimental. It can only
+be enabled by explicitly telling configure with \fB--enable-hsts\fP. You are
+advised to not ship this in production before the experimental label is
+removed.
+.SH DESCRIPTION
+HSTS (HTTP Strict Transport Security) means that an HTTPS server can instruct
+the client to not contact it again over clear-text HTTP for a certain period
+into the future. libcurl will then automatically redirect HTTP attempts to
+such hosts to instead use HTTPS. This is done by libcurl retaining this
+knowledge in an in-memory cache.
+
+Populate the long \fIbitmask\fP with the correct set of features to instruct
+libcurl how to handle HSTS for the transfers using this handle.
+.SH BITS
+.IP "CURLHSTS_ENABLE"
+Enable the in-memory HSTS cache for this handle.
+.IP "CURLHSTS_READONLYFILE"
+Make the HSTS file (if specified) read-only - makes libcurl not save the cache
+to the file when closing the handle.
+.SH DEFAULT
+0. HSTS is disabled by default.
+.SH PROTOCOLS
+HTTPS and HTTP
+.SH EXAMPLE
+.nf
+CURL *curl = curl_easy_init();
+if(curl) {
+ curl_easy_setopt(curl, CURLOPT_HSTS_CTRL, CURLHSTS_ENABLE);
+ curl_easy_perform(curl);
+}
+.fi
+.SH AVAILABILITY
+Added in 7.74.0
+.SH RETURN VALUE
+Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if not.
+.SH "SEE ALSO"
+.BR CURLOPT_HSTS "(3), " CURLOPT_CONNECT_TO "(3), " CURLOPT_RESOLVE "(3), "
+.BR CURLOPT_ALTSVC "(3), "
diff --git a/docs/libcurl/opts/Makefile.inc b/docs/libcurl/opts/Makefile.inc
index fe4177579..9d1eb2bba 100644
--- a/docs/libcurl/opts/Makefile.inc
+++ b/docs/libcurl/opts/Makefile.inc
@@ -180,6 +180,8 @@ man_MANS = \
CURLOPT_HEADERDATA.3 \
CURLOPT_HEADERFUNCTION.3 \
CURLOPT_HEADEROPT.3 \
+ CURLOPT_HSTS.3 \
+ CURLOPT_HSTS_CTRL.3 \
CURLOPT_HTTP09_ALLOWED.3 \
CURLOPT_HTTP200ALIASES.3 \
CURLOPT_HTTPAUTH.3 \
diff --git a/docs/libcurl/symbols-in-versions b/docs/libcurl/symbols-in-versions
index 1b37d13d1..cc35fc57b 100644
--- a/docs/libcurl/symbols-in-versions
+++ b/docs/libcurl/symbols-in-versions
@@ -213,6 +213,8 @@ CURLGSSAPI_DELEGATION_NONE 7.22.0
CURLGSSAPI_DELEGATION_POLICY_FLAG 7.22.0
CURLHEADER_SEPARATE 7.37.0
CURLHEADER_UNIFIED 7.37.0
+CURLHSTS_ENABLE 7.74.0
+CURLHSTS_READONLYFILE 7.74.0
CURLINFO_ACTIVESOCKET 7.45.0
CURLINFO_APPCONNECT_TIME 7.19.0
CURLINFO_APPCONNECT_TIME_T 7.61.0
@@ -443,6 +445,8 @@ CURLOPT_HEADER 7.1
CURLOPT_HEADERDATA 7.10
CURLOPT_HEADERFUNCTION 7.7.2
CURLOPT_HEADEROPT 7.37.0
+CURLOPT_HSTS 7.74.0
+CURLOPT_HSTS_CTRL 7.74.0
CURLOPT_HTTP09_ALLOWED 7.64.0
CURLOPT_HTTP200ALIASES 7.10.3
CURLOPT_HTTPAUTH 7.10.6
@@ -1001,6 +1005,7 @@ CURL_VERSION_CURLDEBUG 7.19.6
CURL_VERSION_DEBUG 7.10.6
CURL_VERSION_GSSAPI 7.38.0
CURL_VERSION_GSSNEGOTIATE 7.10.6 7.38.0
+CURL_VERSION_HSTS 7.74.0
CURL_VERSION_HTTP2 7.33.0
CURL_VERSION_HTTP3 7.66.0
CURL_VERSION_HTTPS_PROXY 7.52.0
diff --git a/docs/options-in-versions b/docs/options-in-versions
index 683363239..97c416d83 100644
--- a/docs/options-in-versions
+++ b/docs/options-in-versions
@@ -79,6 +79,7 @@
--header (-H) 5.0
--help (-h) 4.0
--hostpubmd5 7.17.1
+--hsts 7.74.0
--http0.9 7.64.0
--http1.0 (-0) 7.9.1
--http1.1 7.33.0
diff --git a/include/curl/curl.h b/include/curl/curl.h
index ad1a2097b..568408942 100644
--- a/include/curl/curl.h
+++ b/include/curl/curl.h
@@ -954,6 +954,10 @@ typedef enum {
#define CURLALTSVC_H2 (1<<4)
#define CURLALTSVC_H3 (1<<5)
+/* CURLHSTS_* are bits for the CURLOPT_HSTS option */
+#define CURLHSTS_ENABLE (long)(1<<0)
+#define CURLHSTS_READONLYFILE (long)(1<<1)
+
/* CURLPROTO_ defines are for the CURLOPT_*PROTOCOLS options */
#define CURLPROTO_HTTP (1<<0)
#define CURLPROTO_HTTPS (1<<1)
@@ -2029,6 +2033,11 @@ typedef enum {
*/
CURLOPT(CURLOPT_SSL_EC_CURVES, CURLOPTTYPE_STRINGPOINT, 298),
+ /* HSTS bitmask */
+ CURLOPT(CURLOPT_HSTS_CTRL, CURLOPTTYPE_LONG, 299),
+ /* HSTS file name */
+ CURLOPT(CURLOPT_HSTS, CURLOPTTYPE_STRINGPOINT, 300),
+
CURLOPT_LASTENTRY /* the last unused */
} CURLoption;
@@ -2900,6 +2909,7 @@ typedef struct curl_version_info_data
curl_version_info_data;
#define CURL_VERSION_HTTP3 (1<<25) /* HTTP3 support built-in */
#define CURL_VERSION_ZSTD (1<<26) /* zstd features are present */
#define CURL_VERSION_UNICODE (1<<27) /* Unicode support on Windows */
+#define CURL_VERSION_HSTS (1<<28) /* HSTS is supported */
/*
* NAME curl_version_info()
diff --git a/lib/Makefile.inc b/lib/Makefile.inc
index a2fd57a83..ea7a37496 100644
--- a/lib/Makefile.inc
+++ b/lib/Makefile.inc
@@ -61,7 +61,7 @@ LIB_CFILES = altsvc.c amigaos.c asyn-ares.c asyn-thread.c
base64.c \
socks_gssapi.c socks_sspi.c speedcheck.c splay.c strcase.c strdup.c \
strerror.c strtok.c strtoofft.c system_win32.c telnet.c tftp.c timeval.c \
transfer.c urlapi.c version.c warnless.c wildcard.c x509asn1.c dynbuf.c \
- version_win32.c easyoptions.c easygetopt.c
+ version_win32.c easyoptions.c easygetopt.c hsts.c
LIB_HFILES = altsvc.h amigaos.h arpa_telnet.h asyn.h conncache.h connect.h \
content_encoding.h cookie.h curl_addrinfo.h curl_base64.h curl_ctype.h \
@@ -80,7 +80,7 @@ LIB_HFILES = altsvc.h amigaos.h arpa_telnet.h asyn.h
conncache.h connect.h \
smb.h smtp.h sockaddr.h socketpair.h socks.h speedcheck.h splay.h strcase.h \
strdup.h strerror.h strtok.h strtoofft.h system_win32.h telnet.h tftp.h \
timeval.h transfer.h urlapi-int.h urldata.h warnless.h wildcard.h \
- x509asn1.h dynbuf.h version_win32.h easyoptions.h
+ x509asn1.h dynbuf.h version_win32.h easyoptions.h hsts.h
LIB_RCFILES = libcurl.rc
diff --git a/lib/curl_get_line.c b/lib/curl_get_line.c
index ffe4256ba..aa524d8fe 100644
--- a/lib/curl_get_line.c
+++ b/lib/curl_get_line.c
@@ -22,7 +22,8 @@
#include "curl_setup.h"
-#if !defined(CURL_DISABLE_COOKIES) || !defined(CURL_DISABLE_ALTSVC)
+#if !defined(CURL_DISABLE_COOKIES) || !defined(CURL_DISABLE_ALTSVC)) || \
+ defined(USE_HSTS)
#include "curl_get_line.h"
#include "curl_memory.h"
diff --git a/lib/easy.c b/lib/easy.c
index 60e2befd7..ca1117a46 100644
--- a/lib/easy.c
+++ b/lib/easy.c
@@ -79,6 +79,7 @@
#include "http2.h"
#include "dynbuf.h"
#include "altsvc.h"
+#include "hsts.h"
/* The last 3 #include files should be in this order */
#include "curl_printf.h"
@@ -880,6 +881,15 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy
*data)
if(outcurl->set.str[STRING_ALTSVC])
(void)Curl_altsvc_load(outcurl->asi, outcurl->set.str[STRING_ALTSVC]);
}
+#endif
+#ifdef USE_HSTS
+ if(data->hsts) {
+ outcurl->hsts = Curl_hsts_init();
+ if(!outcurl->hsts)
+ goto fail;
+ if(outcurl->set.str[STRING_HSTS])
+ (void)Curl_hsts_load(outcurl->hsts, outcurl->set.str[STRING_HSTS]);
+ }
#endif
/* Clone the resolver handle, if present, for the new handle */
if(Curl_resolver_duphandle(outcurl,
@@ -929,6 +939,7 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy
*data)
Curl_safefree(outcurl->change.url);
Curl_safefree(outcurl->change.referer);
Curl_altsvc_cleanup(&outcurl->asi);
+ Curl_hsts_cleanup(&outcurl->hsts);
Curl_freeset(outcurl);
free(outcurl);
}
diff --git a/lib/easyoptions.c b/lib/easyoptions.c
index 0ab6a3fc6..e5b9ffb70 100644
--- a/lib/easyoptions.c
+++ b/lib/easyoptions.c
@@ -115,6 +115,8 @@ struct curl_easyoption Curl_easyopts[] = {
{"HEADERDATA", CURLOPT_HEADERDATA, CURLOT_CBPTR, 0},
{"HEADERFUNCTION", CURLOPT_HEADERFUNCTION, CURLOT_FUNCTION, 0},
{"HEADEROPT", CURLOPT_HEADEROPT, CURLOT_VALUES, 0},
+ {"HSTS", CURLOPT_HSTS, CURLOT_STRING, 0},
+ {"HSTS_CTRL", CURLOPT_HSTS_CTRL, CURLOT_LONG, 0},
{"HTTP09_ALLOWED", CURLOPT_HTTP09_ALLOWED, CURLOT_LONG, 0},
{"HTTP200ALIASES", CURLOPT_HTTP200ALIASES, CURLOT_SLIST, 0},
{"HTTPAUTH", CURLOPT_HTTPAUTH, CURLOT_VALUES, 0},
@@ -342,6 +344,6 @@ struct curl_easyoption Curl_easyopts[] = {
*/
int Curl_easyopts_check(void)
{
- return (CURLOPT_LASTENTRY != (298 + 1));
+ return (CURLOPT_LASTENTRY != (300 + 1));
}
#endif
diff --git a/lib/hsts.c b/lib/hsts.c
new file mode 100644
index 000000000..7eb3cda03
--- /dev/null
+++ b/lib/hsts.c
@@ -0,0 +1,430 @@
+/***************************************************************************
+ * _ _ ____ _
+ * Project ___| | | | _ \| |
+ * / __| | | | |_) | |
+ * | (__| |_| | _ <| |___
+ * \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+/*
+ * The Strict-Transport-Security header is defined in RFC 6797:
+ * https://tools.ietf.org/html/rfc6797
+ */
+#include "curl_setup.h"
+
+#if !defined(CURL_DISABLE_HTTP) && defined(USE_HSTS)
+#include <curl/curl.h>
+#include "urldata.h"
+#include "llist.h"
+#include "hsts.h"
+#include "curl_get_line.h"
+#include "strcase.h"
+#include "sendf.h"
+#include "strtoofft.h"
+#include "parsedate.h"
+#include "rand.h"
+#include "rename.h"
+
+/* The last 3 #include files should be in this order */
+#include "curl_printf.h"
+#include "curl_memory.h"
+#include "memdebug.h"
+
+#define MAX_HSTS_LINE 4095
+#define MAX_HSTS_HOSTLEN 256
+#define MAX_HSTS_HOSTLENSTR "256"
+#define MAX_HSTS_SUBLEN 4
+#define MAX_HSTS_SUBLENSTR "4"
+#define MAX_HSTS_DATELEN 64
+#define MAX_HSTS_DATELENSTR "64"
+
+#ifdef DEBUGBUILD
+/* to play well with debug builds, we can *set* a fixed time this will
+ return */
+time_t deltatime; /* allow for "adjustments" for unit test purposes */
+static time_t debugtime(void *unused)
+{
+ char *timestr = getenv("CURL_TIME");
+ (void)unused;
+ if(timestr) {
+ unsigned long val = strtol(timestr, NULL, 10) + deltatime;
+ return (time_t)val;
+ }
+ return time(NULL);
+}
+#define time(x) debugtime(x)
+#endif
+
+struct hsts *Curl_hsts_init(void)
+{
+ struct hsts *h = calloc(sizeof(struct hsts), 1);
+ if(h) {
+ Curl_llist_init(&h->list, NULL);
+ }
+ return h;
+}
+
+static void hsts_free(struct stsentry *e)
+{
+ free((char *)e->host);
+ free(e);
+}
+
+void Curl_hsts_cleanup(struct hsts **hp)
+{
+ struct hsts *h = *hp;
+ if(h) {
+ struct Curl_llist_element *e;
+ struct Curl_llist_element *n;
+ for(e = h->list.head; e; e = n) {
+ struct stsentry *sts = e->ptr;
+ n = e->next;
+ hsts_free(sts);
+ }
+ free(h->filename);
+ free(h);
+ *hp = NULL;
+ }
+}
+
+static struct stsentry *hsts_entry(void)
+{
+ return calloc(sizeof(struct stsentry), 1);
+}
+
+static CURLcode hsts_create(struct hsts *h,
+ const char *hostname,
+ bool subdomains,
+ curl_off_t expires)
+{
+ struct stsentry *sts = hsts_entry();
+ if(!sts)
+ return CURLE_OUT_OF_MEMORY;
+
+ sts->expires = expires;
+ sts->includeSubDomains = subdomains;
+ sts->host = strdup(hostname);
+ if(!sts->host) {
+ free(sts);
+ return CURLE_OUT_OF_MEMORY;
+ }
+ fprintf(stderr, "*** Add %s %s %d\n", hostname,
+ subdomains?"SUB":"-", (int)expires);
+ Curl_llist_insert_next(&h->list, h->list.tail, sts, &sts->node);
+ return CURLE_OK;
+}
+
+CURLcode Curl_hsts_parse(struct hsts *h, const char *hostname,
+ const char *header)
+{
+ const char *p = header;
+ curl_off_t expires = 0;
+ bool gotma = FALSE;
+ bool gotinc = FALSE;
+ bool subdomains = FALSE;
+ struct stsentry *sts;
+ time_t now = time(NULL);
+
+ do {
+ while(*p && ISSPACE(*p))
+ p++;
+ if(Curl_strncasecompare("max-age=", p, 8)) {
+ bool quoted = FALSE;
+ CURLofft offt;
+ char *endp;
+
+ if(gotma)
+ return CURLE_BAD_FUNCTION_ARGUMENT;
+
+ p += 8;
+ while(*p && ISSPACE(*p))
+ p++;
+ if(*p == '\"') {
+ p++;
+ quoted = TRUE;
+ }
+ offt = curlx_strtoofft(p, &endp, 10, &expires);
+ if(offt == CURL_OFFT_FLOW)
+ expires = CURL_OFF_T_MAX;
+ else if(offt)
+ /* invalid max-age */
+ return CURLE_BAD_FUNCTION_ARGUMENT;
+ p = endp;
+ if(quoted) {
+ if(*p != '\"')
+ return CURLE_BAD_FUNCTION_ARGUMENT;
+ p++;
+ }
+ gotma = TRUE;
+ }
+ else if(Curl_strncasecompare("includesubdomains", p, 17)) {
+ if(gotinc)
+ return CURLE_BAD_FUNCTION_ARGUMENT;
+ subdomains = TRUE;
+ p += 17;
+ gotinc = TRUE;
+ }
+ else {
+ /* unknown directive, do a lame attempt to skip */
+ while(*p && (*p != ';'))
+ p++;
+ }
+
+ while(*p && ISSPACE(*p))
+ p++;
+ if(*p == ';')
+ p++;
+ } while (*p);
+
+ if(!gotma)
+ /* max-age is mandatory */
+ return CURLE_BAD_FUNCTION_ARGUMENT;
+
+ if(!expires) {
+ /* remove the entry if present verbatim (without subdomain match) */
+ sts = Curl_hsts(h, hostname, FALSE);
+ if(sts) {
+ Curl_llist_remove(&h->list, &sts->node, NULL);
+ hsts_free(sts);
+ }
+ return CURLE_OK;
+ }
+
+ if(CURL_OFF_T_MAX - now < expires)
+ /* would overflow, use maximum value */
+ expires = CURL_OFF_T_MAX;
+ else
+ expires += now;
+
+ /* check if it already exists */
+ sts = Curl_hsts(h, hostname, FALSE);
+ if(sts) {
+ /* just update these fields */
+ sts->expires = expires;
+ sts->includeSubDomains = subdomains;
+ }
+ else
+ return hsts_create(h, hostname, subdomains, expires);
+
+ return CURLE_OK;
+}
+
+/*
+ * Return TRUE if the given host name is currently an HSTS one.
+ *
+ * The 'subdomain' argument tells the function if subdomain matching should be
+ * attempted.
+ */
+struct stsentry *Curl_hsts(struct hsts *h, const char *hostname,
+ bool subdomain)
+{
+ if(h) {
+ time_t now = time(NULL);
+ size_t hlen = strlen(hostname);
+ struct Curl_llist_element *e;
+ struct Curl_llist_element *n;
+ for(e = h->list.head; e; e = n) {
+ struct stsentry *sts = e->ptr;
+ n = e->next;
+ if(sts->expires <= now) {
+ /* remove expired entries */
+ Curl_llist_remove(&h->list, &sts->node, NULL);
+ hsts_free(sts);
+ continue;
+ }
+ if(subdomain && sts->includeSubDomains) {
+ size_t ntail = strlen(sts->host);
+ if(ntail < hlen) {
+ size_t offs = hlen - ntail;
+ if((hostname[offs-1] == '.') &&
+ Curl_strncasecompare(&hostname[offs], sts->host, ntail))
+ return sts;
+ }
+ }
+ if(Curl_strcasecompare(hostname, sts->host))
+ return sts;
+ }
+ }
+ return NULL; /* no match */
+}
+
+/*
+ * Write this single hsts entry to a single output line
+ */
+static CURLcode hsts_out(struct stsentry *sts, FILE *fp)
+{
+ struct tm stamp;
+ CURLcode result = Curl_gmtime(sts->expires, &stamp);
+ if(result)
+ return result;
+
+ fprintf(fp, "%s%s \"%d%02d%02d %02d:%02d:%02d\"\n",
+ sts->includeSubDomains ? ".": "", sts->host,
+ stamp.tm_year + 1900, stamp.tm_mon + 1, stamp.tm_mday,
+ stamp.tm_hour, stamp.tm_min, stamp.tm_sec);
+ return CURLE_OK;
+}
+
+
+/*
+ * Curl_https_save() writes the HSTS cache to a file.
+ */
+CURLcode Curl_hsts_save(struct Curl_easy *data, struct hsts *h,
+ const char *file)
+{
+ struct Curl_llist_element *e;
+ struct Curl_llist_element *n;
+ CURLcode result = CURLE_OK;
+ FILE *out;
+ char *tempstore;
+ unsigned char randsuffix[9];
+
+ if(!h)
+ /* no cache activated */
+ return CURLE_OK;
+
+ /* if not new name is given, use the one we stored from the load */
+ if(!file && h->filename)
+ file = h->filename;
+
+ if((h->flags & CURLHSTS_READONLYFILE) || !file || !file[0])
+ /* marked as read-only, no file or zero length file name */
+ return CURLE_OK;
+
+ if(Curl_rand_hex(data, randsuffix, sizeof(randsuffix)))
+ return CURLE_FAILED_INIT;
+
+ tempstore = aprintf("%s.%s.tmp", file, randsuffix);
+ if(!tempstore)
+ return CURLE_OUT_OF_MEMORY;
+
+ out = fopen(tempstore, FOPEN_WRITETEXT);
+ if(!out)
+ result = CURLE_WRITE_ERROR;
+ else {
+ fputs("# Your HSTS cache. https://curl.haxx.se/docs/hsts.html\n";
+ "# This file was generated by libcurl! Edit at your own risk.\n",
+ out);
+ for(e = h->list.head; e; e = n) {
+ struct stsentry *sts = e->ptr;
+ n = e->next;
+ result = hsts_out(sts, out);
+ if(result)
+ break;
+ }
+ fclose(out);
+ if(!result && Curl_rename(tempstore, file))
+ result = CURLE_WRITE_ERROR;
+
+ if(result)
+ unlink(tempstore);
+ }
+ free(tempstore);
+ return result;
+}
+
+/* only returns SERIOUS errors */
+static CURLcode hsts_add(struct hsts *h, char *line)
+{
+ /* Example lines:
+ example.com "20191231 10:00:00"
+ .example.net "20191231 10:00:00"
+ */
+ char host[MAX_HSTS_HOSTLEN + 1];
+ char date[MAX_HSTS_DATELEN + 1];
+ int rc;
+
+ rc = sscanf(line,
+ "%" MAX_HSTS_HOSTLENSTR "s \"%" MAX_HSTS_DATELENSTR "[^\"]\"",
+ host, date);
+ if(2 == rc) {
+ time_t expires = Curl_getdate_capped(date);
+ CURLcode result;
+ char *p = host;
+ bool subdomain = FALSE;
+ if(p[0] == '.') {
+ p++;
+ subdomain = TRUE;
+ }
+ result = hsts_create(h, p, subdomain, expires);
+ if(result)
+ return result;
+ }
+
+ return CURLE_OK;
+}
+
+/*
+ * Load the HSTS cache from the given file. The text based line-oriented file
+ * format is documented here:
+ * https://github.com/curl/curl/wiki/HSTS
+ *
+ * This function only returns error on major problems that prevents hsts
+ * handling to work completely. It will ignore individual syntactical errors
+ * etc.
+ */
+static CURLcode hsts_load(struct hsts *h, const char *file)
+{
+ CURLcode result = CURLE_OK;
+ char *line = NULL;
+ FILE *fp;
+
+ /* we need a private copy of the file name so that the hsts cache file
+ name survives an easy handle reset */
+ free(h->filename);
+ h->filename = strdup(file);
+ if(!h->filename)
+ return CURLE_OUT_OF_MEMORY;
+
+ fp = fopen(file, FOPEN_READTEXT);
+ if(fp) {
+ line = malloc(MAX_HSTS_LINE);
+ if(!line)
+ goto fail;
+ while(Curl_get_line(line, MAX_HSTS_LINE, fp)) {
+ char *lineptr = line;
+ while(*lineptr && ISBLANK(*lineptr))
+ lineptr++;
+ if(*lineptr == '#')
+ /* skip commented lines */
+ continue;
+
+ hsts_add(h, lineptr);
+ }
+ free(line); /* free the line buffer */
+ fclose(fp);
+ }
+ return result;
+
+ fail:
+ Curl_safefree(h->filename);
+ free(line);
+ fclose(fp);
+ return CURLE_OUT_OF_MEMORY;
+}
+
+/*
+ * Curl_hsts_load() loads HSTS from file.
+ */
+CURLcode Curl_hsts_load(struct hsts *h, const char *file)
+{
+ CURLcode result;
+ DEBUGASSERT(h);
+ result = hsts_load(h, file);
+ return result;
+}
+
+#endif /* CURL_DISABLE_HTTP || USE_HSTS */
diff --git a/lib/hsts.h b/lib/hsts.h
new file mode 100644
index 000000000..60b3c2df7
--- /dev/null
+++ b/lib/hsts.h
@@ -0,0 +1,60 @@
+#ifndef HEADER_CURL_HSTS_H
+#define HEADER_CURL_HSTS_H
+/***************************************************************************
+ * _ _ ____ _
+ * Project ___| | | | _ \| |
+ * / __| | | | |_) | |
+ * | (__| |_| | _ <| |___
+ * \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+#include "curl_setup.h"
+
+#if !defined(CURL_DISABLE_HTTP) && defined(USE_HSTS)
+#include <curl/curl.h>
+#include "llist.h"
+
+#ifdef DEBUGBUILD
+extern time_t deltatime;
+#endif
+
+struct stsentry {
+ struct Curl_llist_element node;
+ const char *host;
+ bool includeSubDomains;
+ time_t expires; /* the timestamp of this entry's expiry */
+};
+
+/* The HSTS cache. Needs to be able to tailmatch host names. */
+struct hsts {
+ struct Curl_llist list;
+ char *filename;
+ unsigned int flags;
+};
+
+struct hsts *Curl_hsts_init(void);
+void Curl_hsts_cleanup(struct hsts **hp);
+CURLcode Curl_hsts_parse(struct hsts *h, const char *hostname,
+ const char *sts);
+struct stsentry *Curl_hsts(struct hsts *h, const char *hostname,
+ bool subdomain);
+CURLcode Curl_hsts_save(struct Curl_easy *data, struct hsts *h,
+ const char *file);
+CURLcode Curl_hsts_load(struct hsts *h, const char *file);
+#else
+#define Curl_hsts_cleanup(x)
+#endif /* CURL_DISABLE_HTTP || USE_HSTS */
+#endif /* HEADER_CURL_HSTS_H */
diff --git a/lib/http.c b/lib/http.c
index 3a0a32df4..4db1c9589 100644
--- a/lib/http.c
+++ b/lib/http.c
@@ -77,6 +77,7 @@
#include "connect.h"
#include "strdup.h"
#include "altsvc.h"
+#include "hsts.h"
/* The last 3 #include files should be in this order */
#include "curl_printf.h"
@@ -3990,6 +3991,23 @@ CURLcode Curl_http_readwrite_headers(struct Curl_easy
*data,
}
}
}
+
+#ifdef USE_HSTS
+ /* If enabled, the header is incoming and this is over HTTPS */
+ else if(data->hsts && checkprefix("Strict-Transport-Security:", headp) &&
+ (conn->handler->flags & PROTOPT_SSL)) {
+ CURLcode check =
+ Curl_hsts_parse(data->hsts, data->state.up.hostname,
+ &headp[ sizeof("Strict-Transport-Security:") -1 ]);
+ if(check)
+ infof(data, "Illegal STS header skipped\n");
+#ifdef DEBUGBUILD
+ else
+ infof(data, "Parsed STS header fine (%d entries)\n",
+ data->hsts->list.size);
+#endif
+ }
+#endif
#ifndef CURL_DISABLE_ALTSVC
/* If enabled, the header is incoming and this is over HTTPS */
else if(data->asi && checkprefix("Alt-Svc:", headp) &&
diff --git a/lib/setopt.c b/lib/setopt.c
index 3b96289c2..4aa31bb39 100644
--- a/lib/setopt.c
+++ b/lib/setopt.c
@@ -45,6 +45,7 @@
#include "setopt.h"
#include "multiif.h"
#include "altsvc.h"
+#include "hsts.h"
/* The last 3 #include files should be in this order */
#include "curl_printf.h"
@@ -2839,6 +2840,33 @@ CURLcode Curl_vsetopt(struct Curl_easy *data, CURLoption
option, va_list param)
data->set.trailer_data = va_arg(param, void *);
#endif
break;
+#ifdef USE_HSTS
+ case CURLOPT_HSTS:
+ if(!data->hsts) {
+ data->hsts = Curl_hsts_init();
+ if(!data->hsts)
+ return CURLE_OUT_OF_MEMORY;
+ }
+ argptr = va_arg(param, char *);
+ result = Curl_setstropt(&data->set.str[STRING_HSTS], argptr);
+ if(result)
+ return result;
+ if(argptr)
+ (void)Curl_hsts_load(data->hsts, argptr);
+ break;
+ case CURLOPT_HSTS_CTRL:
+ arg = va_arg(param, long);
+ if(arg & CURLHSTS_ENABLE) {
+ if(!data->hsts) {
+ data->hsts = Curl_hsts_init();
+ if(!data->hsts)
+ return CURLE_OUT_OF_MEMORY;
+ }
+ }
+ else
+ Curl_hsts_cleanup(&data->hsts);
+ break;
+#endif
#ifndef CURL_DISABLE_ALTSVC
case CURLOPT_ALTSVC:
if(!data->asi) {
diff --git a/lib/url.c b/lib/url.c
index 0176517d9..1e3f025ad 100644
--- a/lib/url.c
+++ b/lib/url.c
@@ -96,6 +96,7 @@ bool curl_win32_idn_to_ascii(const char *in, char **out);
#include "getinfo.h"
#include "urlapi-int.h"
#include "system_win32.h"
+#include "hsts.h"
/* And now for the protocols */
#include "ftp.h"
@@ -411,6 +412,7 @@ CURLcode Curl_close(struct Curl_easy **datap)
Curl_flush_cookies(data, TRUE);
Curl_altsvc_save(data, data->asi, data->set.str[STRING_ALTSVC]);
Curl_altsvc_cleanup(&data->asi);
+ Curl_hsts_cleanup(&data->hsts);
#if !defined(CURL_DISABLE_HTTP) && !defined(CURL_DISABLE_CRYPTO_AUTH)
Curl_http_auth_cleanup_digest(data);
#endif
@@ -1911,6 +1913,19 @@ static CURLcode parseurlandfillconn(struct Curl_easy
*data,
if(uc)
return Curl_uc_to_curlcode(uc);
+ uc = curl_url_get(uh, CURLUPART_HOST, &data->state.up.hostname, 0);
+ if(uc) {
+ if(!strcasecompare("file", data->state.up.scheme))
+ return CURLE_OUT_OF_MEMORY;
+ }
+
+#ifdef USE_HSTS
+ if(data->hsts && strcasecompare("http", data->state.up.scheme)) {
+ if(Curl_hsts(data->hsts, data->state.up.hostname, TRUE))
+ infof(data, "Switch from HTTP to HTTPS due to HSTS!\n");
+ }
+#endif
+
result = findprotocol(data, conn, data->state.up.scheme);
if(result)
return result;
@@ -1956,12 +1971,6 @@ static CURLcode parseurlandfillconn(struct Curl_easy
*data,
else if(uc != CURLUE_NO_OPTIONS)
return Curl_uc_to_curlcode(uc);
- uc = curl_url_get(uh, CURLUPART_HOST, &data->state.up.hostname, 0);
- if(uc) {
- if(!strcasecompare("file", data->state.up.scheme))
- return CURLE_OUT_OF_MEMORY;
- }
-
uc = curl_url_get(uh, CURLUPART_PATH, &data->state.up.path, 0);
if(uc)
return Curl_uc_to_curlcode(uc);
diff --git a/lib/urldata.h b/lib/urldata.h
index e8b54aa30..ea7060ec5 100644
--- a/lib/urldata.h
+++ b/lib/urldata.h
@@ -1531,35 +1531,26 @@ enum dupstring {
STRING_RTSP_SESSION_ID, /* Session ID to use */
STRING_RTSP_STREAM_URI, /* Stream URI for this request */
STRING_RTSP_TRANSPORT, /* Transport for this session */
-
STRING_SSH_PRIVATE_KEY, /* path to the private key file for auth */
STRING_SSH_PUBLIC_KEY, /* path to the public key file for auth */
STRING_SSH_HOST_PUBLIC_KEY_MD5, /* md5 of host public key in ascii hex */
STRING_SSH_KNOWNHOSTS, /* file name of knownhosts file */
-
STRING_PROXY_SERVICE_NAME, /* Proxy service name */
STRING_SERVICE_NAME, /* Service name */
STRING_MAIL_FROM,
STRING_MAIL_AUTH,
-
STRING_TLSAUTH_USERNAME_ORIG, /* TLS auth <username> */
STRING_TLSAUTH_USERNAME_PROXY, /* TLS auth <username> */
STRING_TLSAUTH_PASSWORD_ORIG, /* TLS auth <password> */
STRING_TLSAUTH_PASSWORD_PROXY, /* TLS auth <password> */
-
STRING_BEARER, /* <bearer>, if used */
-
STRING_UNIX_SOCKET_PATH, /* path to Unix socket, if used */
-
STRING_TARGET, /* CURLOPT_REQUEST_TARGET */
STRING_DOH, /* CURLOPT_DOH_URL */
-
STRING_ALTSVC, /* CURLOPT_ALTSVC */
-
+ STRING_HSTS, /* CURLOPT_HSTS */
STRING_SASL_AUTHZID, /* CURLOPT_SASL_AUTHZID */
-
STRING_TEMP_URL, /* temp URL storage for proxy use */
-
STRING_DNS_SERVERS,
STRING_DNS_INTERFACE,
STRING_DNS_LOCAL_IP4,
@@ -1899,6 +1890,9 @@ struct Curl_easy {
NOTE that the 'cookie' field in the
UserDefined struct defines if the "engine"
is to be used or not. */
+#ifdef USE_HSTS
+ struct hsts *hsts;
+#endif
#ifndef CURL_DISABLE_ALTSVC
struct altsvcinfo *asi; /* the alt-svc cache */
#endif
diff --git a/lib/version.c b/lib/version.c
index 70e456658..5b759be38 100644
--- a/lib/version.c
+++ b/lib/version.c
@@ -417,6 +417,9 @@ static curl_version_info_data version_info = {
#endif
#ifndef CURL_DISABLE_ALTSVC
| CURL_VERSION_ALTSVC
+#endif
+#if defined(USE_HSTS)
+ | CURL_VERSION_HSTS
#endif
,
NULL, /* ssl_version */
diff --git a/src/tool_cfgable.c b/src/tool_cfgable.c
index e99602c4f..3c0bbfa64 100644
--- a/src/tool_cfgable.c
+++ b/src/tool_cfgable.c
@@ -54,6 +54,7 @@ static void free_config_fields(struct OperationConfig *config)
Curl_safefree(config->egd_file);
Curl_safefree(config->useragent);
Curl_safefree(config->altsvc);
+ Curl_safefree(config->hsts);
Curl_safefree(config->cookie);
Curl_safefree(config->cookiejar);
Curl_safefree(config->cookiefile);
diff --git a/src/tool_cfgable.h b/src/tool_cfgable.h
index 489f9ca0e..ac4c7fadc 100644
--- a/src/tool_cfgable.h
+++ b/src/tool_cfgable.h
@@ -58,6 +58,7 @@ struct OperationConfig {
char *cookiejar; /* write to this file */
char *cookiefile; /* read from this file */
char *altsvc; /* alt-svc cache file name */
+ char *hsts; /* HSTS cache file name */
bool cookiesession; /* new session? */
bool encoding; /* Accept-Encoding please */
bool tr_encoding; /* Transfer-Encoding please */
diff --git a/src/tool_getparam.c b/src/tool_getparam.c
index 910a5a2f9..d2e4eb498 100644
--- a/src/tool_getparam.c
+++ b/src/tool_getparam.c
@@ -219,6 +219,7 @@ static const struct LongShort aliases[]= {
{"A", "user-agent", ARG_STRING},
{"b", "cookie", ARG_STRING},
{"ba", "alt-svc", ARG_STRING},
+ {"bb", "hsts", ARG_STRING},
{"B", "use-ascii", ARG_BOOL},
{"c", "cookie-jar", ARG_STRING},
{"C", "continue-at", ARG_STRING},
@@ -1291,6 +1292,12 @@ ParameterError getparameter(const char *flag, /* f or
-long-flag */
else
return PARAM_LIBCURL_DOESNT_SUPPORT;
break;
+ case 'b': /* --hsts */
+ if(curlinfo->features & CURL_VERSION_HSTS)
+ GetStr(&config->hsts, nextarg);
+ else
+ return PARAM_LIBCURL_DOESNT_SUPPORT;
+ break;
default: /* --cookie string coming up: */
if(nextarg[0] == '@') {
nextarg++;
diff --git a/src/tool_help.c b/src/tool_help.c
index 544dbbab0..0833a0d23 100644
--- a/src/tool_help.c
+++ b/src/tool_help.c
@@ -328,6 +328,9 @@ static const struct helptxt helptext[] = {
{" --hostpubmd5 <md5>",
"Acceptable MD5 hash of the host public key",
CURLHELP_SFTP | CURLHELP_SCP},
+ {" --hsts <file name>",
+ "Enable HSTS with this cache file",
+ CURLHELP_HTTP},
{" --http0.9",
"Allow HTTP 0.9 responses",
CURLHELP_HTTP},
@@ -862,6 +865,7 @@ static const struct feat feats[] = {
{"MultiSSL", CURL_VERSION_MULTI_SSL},
{"PSL", CURL_VERSION_PSL},
{"alt-svc", CURL_VERSION_ALTSVC},
+ {"HSTS", CURL_VERSION_HSTS},
};
static void print_category(curlhelp_t category)
diff --git a/src/tool_operate.c b/src/tool_operate.c
index 4ad5052ff..e0fde724b 100644
--- a/src/tool_operate.c
+++ b/src/tool_operate.c
@@ -2072,6 +2072,9 @@ static CURLcode single_transfer(struct GlobalConfig
*global,
if(config->altsvc)
my_setopt_str(curl, CURLOPT_ALTSVC, config->altsvc);
+ if(config->hsts)
+ my_setopt_bitmask(curl, CURLOPT_HSTS_CTRL, CURLHSTS_ENABLE);
+
#ifdef USE_METALINK
if(!metalink && config->use_metalink) {
outs->metalink_parser = metalink_parser_context_new();
diff --git a/src/tool_setopt.c b/src/tool_setopt.c
index 0dd7a57a2..2159db6cd 100644
--- a/src/tool_setopt.c
+++ b/src/tool_setopt.c
@@ -62,6 +62,11 @@ const struct NameValue setopt_nv_CURL_SOCKS_PROXY[] = {
NVEND,
};
+const struct NameValueUnsigned setopt_nv_CURLHSTS[] = {
+ NV(CURLHSTS_ENABLE),
+ NVEND,
+};
+
const struct NameValueUnsigned setopt_nv_CURLAUTH[] = {
NV(CURLAUTH_ANY), /* combination */
NV(CURLAUTH_ANYSAFE), /* combination */
diff --git a/src/tool_setopt.h b/src/tool_setopt.h
index 3db88c6bf..f8d3320d3 100644
--- a/src/tool_setopt.h
+++ b/src/tool_setopt.h
@@ -64,8 +64,10 @@ extern const struct NameValueUnsigned setopt_nv_CURLSSLOPT[];
extern const struct NameValue setopt_nv_CURL_NETRC[];
extern const struct NameValue setopt_nv_CURLPROTO[];
extern const struct NameValueUnsigned setopt_nv_CURLAUTH[];
+extern const struct NameValueUnsigned setopt_nv_CURLHSTS[];
/* Map options to NameValue sets */
+#define setopt_nv_CURLOPT_HSTS_CTRL setopt_nv_CURLHSTS
#define setopt_nv_CURLOPT_HTTP_VERSION setopt_nv_CURL_HTTP_VERSION
#define setopt_nv_CURLOPT_HTTPAUTH setopt_nv_CURLAUTH
#define setopt_nv_CURLOPT_SSLVERSION setopt_nv_CURL_SSLVERSION
diff --git a/tests/data/Makefile.inc b/tests/data/Makefile.inc
index 2e0c092ad..04e23c6fc 100644
--- a/tests/data/Makefile.inc
+++ b/tests/data/Makefile.inc
@@ -197,6 +197,7 @@ test1620 test1621 \
test1630 test1631 test1632 test1633 \
\
test1650 test1651 test1652 test1653 test1654 test1655 \
+test1660 \
\
test1700 test1701 test1702 \
\
diff --git a/tests/data/test1660 b/tests/data/test1660
new file mode 100644
index 000000000..f64765c3f
--- /dev/null
+++ b/tests/data/test1660
@@ -0,0 +1,81 @@
+<testcase>
+<info>
+<keywords>
+unittest
+HSTS
+</keywords>
+</info>
+
+<client>
+<server>
+none
+</server>
+<features>
+unittest
+HSTS
+</features>
+
+<file name="log/input1660">
+# Your HSTS cache. https://curl.haxx.se/docs/hsts.html
+# This file was generated by libcurl! Edit at your own risk.
+.readfrom.example "20211001 04:47:41"
+.old.example "20161001 04:47:41"
+</file>
+
+# This date is exactly "20190124 22:34:21" UTC
+<setenv>
+CURL_TIME=1548369261
+</setenv>
+<name>
+HSTS
+</name>
+<command>
+-
+</command>
+</client>
+
+<verify>
+<stdout>
+readfrom.example [readfrom.example]: 1633063661 includeSubDomains
+'old.example' is not HSTS
+'readfrom.example' is not HSTS
+example.com [example.com]: 1579905261
+example.com [example.com]: 1569905261
+example.com [example.com]: 1569905261
+example.com [example.com]: 1569905261 includeSubDomains
+example.org [example.org]: 1579905261
+Input 8: error 43
+Input 9: error 43
+this.example [this.example]: 1548400797
+'this.example' is not HSTS
+Input 12: error 43
+Input 13: error 43
+Input 14: error 43
+3.example.com [example.com]: 1569905261 includeSubDomains
+3.example.com [example.com]: 1569905261 includeSubDomains
+foo.example.com [example.com]: 1569905261 includeSubDomains
+'foo.xample.com' is not HSTS
+'forexample.net' is not HSTS
+'forexample.net' is not HSTS
+'example.net' is not HSTS
+expire.example [expire.example]: 1548369268
+Number of entries: 3
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+expire.example [expire.example]: 1548369268
+'expire.example' is not HSTS
+'expire.example' is not HSTS
+'expire.example' is not HSTS
+</stdout>
+<file name="log/hsts1660">
+# Your HSTS cache. https://curl.haxx.se/docs/hsts.html
+# This file was generated by libcurl! Edit at your own risk.
+.example.com "20191001 04:47:41"
+example.org "20200124 22:34:21"
+</file>
+</verify>
+</testcase>
diff --git a/tests/runtests.pl b/tests/runtests.pl
index 4bcf61bbf..a4e330661 100755
--- a/tests/runtests.pl
+++ b/tests/runtests.pl
@@ -256,6 +256,7 @@ my $has_cares; # set if built with c-ares
my $has_threadedres;# set if built with threaded resolver
my $has_psl; # set if libcurl is built with PSL support
my $has_altsvc; # set if libcurl is built with alt-svc support
+my $has_hsts; # set if libcurl is built with HSTS support
my $has_ldpreload; # set if curl is built for systems supporting LD_PRELOAD
my $has_multissl; # set if curl is build with MultiSSL support
my $has_manual; # set if curl is built with built-in manual
@@ -2762,6 +2763,7 @@ sub compare {
sub setupfeatures {
$feature{"alt-svc"} = $has_altsvc;
+ $feature{"HSTS"} = $has_hsts;
$feature{"brotli"} = $has_brotli;
$feature{"crypto"} = $has_crypto;
$feature{"debug"} = $debug_build;
@@ -3035,6 +3037,9 @@ sub checksystem {
# alt-svc enabled
$has_altsvc=1;
}
+ if($feat =~ /HSTS/i) {
+ $has_hsts=1;
+ }
if($feat =~ /AsynchDNS/i) {
if(!$has_cares) {
# this means threaded resolver
diff --git a/tests/unit/Makefile.inc b/tests/unit/Makefile.inc
index f63724f91..ee6816823 100644
--- a/tests/unit/Makefile.inc
+++ b/tests/unit/Makefile.inc
@@ -34,7 +34,8 @@ UNITPROGS = unit1300 unit1301 unit1302 unit1303 unit1304
unit1305 unit1307 \
unit1600 unit1601 unit1602 unit1603 unit1604 unit1605 unit1606 unit1607 \
unit1608 unit1609 unit1610 unit1611 unit1612 \
unit1620 unit1621 \
- unit1650 unit1651 unit1652 unit1653 unit1654 unit1655
+ unit1650 unit1651 unit1652 unit1653 unit1654 unit1655 \
+ unit1660
unit1300_SOURCES = unit1300.c $(UNITFILES)
unit1300_CPPFLAGS = $(AM_CPPFLAGS)
@@ -154,3 +155,5 @@ unit1654_CPPFLAGS = $(AM_CPPFLAGS)
unit1655_SOURCES = unit1655.c $(UNITFILES)
unit1655_CPPFLAGS = $(AM_CPPFLAGS)
+unit1660_SOURCES = unit1660.c $(UNITFILES)
+unit1660_CPPFLAGS = $(AM_CPPFLAGS)
diff --git a/tests/unit/unit1660.c b/tests/unit/unit1660.c
new file mode 100644
index 000000000..1687cafa1
--- /dev/null
+++ b/tests/unit/unit1660.c
@@ -0,0 +1,172 @@
+/***************************************************************************
+ * _ _ ____ _
+ * Project ___| | | | _ \| |
+ * / __| | | | |_) | |
+ * | (__| |_| | _ <| |___
+ * \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 2020, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+#include "curlcheck.h"
+
+#include "urldata.h"
+#include "hsts.h"
+
+static CURLcode
+unit_setup(void)
+{
+ return CURLE_OK;
+}
+
+static void
+unit_stop(void)
+{
+ curl_global_cleanup();
+}
+
+#if defined(CURL_DISABLE_HTTP) || !defined(USE_HSTS)
+UNITTEST_START
+{
+ return 0; /* nothing to do when HTTP or HSTS are disabled */
+}
+UNITTEST_STOP
+#else
+
+struct testit {
+ const char *host;
+ const char *chost; /* if non-NULL, use to lookup with */
+ const char *hdr; /* if NULL, just do the lookup */
+ const CURLcode result; /* parse result */
+};
+
+static const struct testit headers[] = {
+ /* two entries read from disk cache, verify first */
+ { "-", "readfrom.example", NULL, CURLE_OK},
+ { "-", "old.example", NULL, CURLE_OK},
+ /* delete the remaining one read from disk */
+ { "readfrom.example", NULL, "max-age=\"0\"", CURLE_OK},
+
+ { "example.com", NULL, "max-age=\"31536000\"\r\n", CURLE_OK },
+ { "example.com", NULL, "max-age=\"21536000\"\r\n", CURLE_OK },
+ { "example.com", NULL, "max-age=\"21536000\"; \r\n", CURLE_OK },
+ { "example.com", NULL, "max-age=\"21536000\"; includeSubDomains\r\n",
+ CURLE_OK },
+ { "example.org", NULL, "max-age=\"31536000\"\r\n", CURLE_OK },
+ { "this.example", NULL, "max=\"31536\";", CURLE_BAD_FUNCTION_ARGUMENT },
+ { "this.example", NULL, "max-age=\"31536", CURLE_BAD_FUNCTION_ARGUMENT },
+ { "this.example", NULL, "max-age=31536\"", CURLE_OK },
+ /* max-age=0 removes the entry */
+ { "this.example", NULL, "max-age=0", CURLE_OK },
+ { "another.example", NULL, "includeSubDomains; ",
+ CURLE_BAD_FUNCTION_ARGUMENT },
+
+ /* Two max-age is illegal */
+ { "example.com", NULL,
+ "max-age=\"21536000\"; includeSubDomains; max-age=\"3\";",
+ CURLE_BAD_FUNCTION_ARGUMENT },
+ /* Two includeSubDomains is illegal */
+ { "2.example.com", NULL,
+ "max-age=\"21536000\"; includeSubDomains; includeSubDomains;",
+ CURLE_BAD_FUNCTION_ARGUMENT },
+ /* use a unknown directive "include" that should be ignored */
+ { "3.example.com", NULL, "max-age=\"21536000\"; include; includeSubDomains;",
+ CURLE_OK },
+ /* remove the "3.example.com" one, should still match the example.com */
+ { "3.example.com", NULL, "max-age=\"0\"; includeSubDomains;",
+ CURLE_OK },
+ { "-", "foo.example.com", NULL, CURLE_OK},
+ { "-", "foo.xample.com", NULL, CURLE_OK},
+
+ /* should not match */
+ { "example.net", "forexample.net", "max-age=\"31536000\"\r\n", CURLE_OK },
+
+ /* should not match either, since forexample.net is not in the example.net
+ domain */
+ { "example.net", "forexample.net",
+ "max-age=\"31536000\"; includeSubDomains\r\n", CURLE_OK },
+ /* remove example.net again */
+ { "example.net", NULL, "max-age=\"0\"; includeSubDomains\r\n", CURLE_OK },
+
+ /* make this live for 7 seconds */
+ { "expire.example", NULL, "max-age=\"7\"\r\n", CURLE_OK },
+ { NULL, NULL, NULL, 0 }
+};
+
+static void showsts(struct stsentry *e, const char *chost)
+{
+ if(!e)
+ printf("'%s' is not HSTS\n", chost);
+ else {
+ printf("%s [%s]: %" CURL_FORMAT_CURL_OFF_T "%s\n",
+ chost, e->host, e->expires,
+ e->includeSubDomains ? " includeSubDomains" : "");
+ }
+}
+
+UNITTEST_START
+{
+ CURLcode result;
+ struct stsentry *e;
+ struct hsts *h = Curl_hsts_init();
+ int i;
+ const char *chost;
+ CURL *easy;
+ if(!h)
+ return 1;
+
+ Curl_hsts_load(h, "log/input1660");
+
+ for(i = 0; headers[i].host ; i++) {
+ if(headers[i].hdr) {
+ result = Curl_hsts_parse(h, headers[i].host, headers[i].hdr);
+
+ if(result != headers[i].result) {
+ fprintf(stderr, "Curl_hsts_parse(%s) failed: %d\n",
+ headers[i].hdr, result);
+ unitfail++;
+ continue;
+ }
+ else if(result) {
+ printf("Input %u: error %d\n", i, (int) result);
+ continue;
+ }
+ }
+
+ chost = headers[i].chost ? headers[i].chost : headers[i].host;
+ e = Curl_hsts(h, chost, TRUE);
+ showsts(e, chost);
+ }
+
+ printf("Number of entries: %d\n", h->list.size);
+
+ /* verify that it is exists for 7 seconds */
+ chost = "expire.example";
+ for(i = 100; i < 110; i++) {
+ e = Curl_hsts(h, chost, TRUE);
+ showsts(e, chost);
+ deltatime++; /* another second passed */
+ }
+
+ easy = curl_easy_init();
+ if(easy) {
+ (void)Curl_hsts_save(easy, h, "log/hsts1660");
+ curl_easy_cleanup(easy);
+ }
+
+ Curl_hsts_cleanup(&h);
+ return unitfail;
+}
+UNITTEST_STOP
+#endif
--
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.
- [gnurl] 375/411: test506: make it not run in c-ares builds, (continued)
- [gnurl] 375/411: test506: make it not run in c-ares builds, gnunet, 2021/01/12
- [gnurl] 306/411: rtsp: error out on empty Session ID, unified the code, gnunet, 2021/01/12
- [gnurl] 266/411: KNOWN_BUGS: mention the individual cmake issues, gnunet, 2021/01/12
- [gnurl] 362/411: KNOWN_BUGS: wakeup socket disconnect causes havoc, gnunet, 2021/01/12
- [gnurl] 317/411: FAQ: refreshed, gnunet, 2021/01/12
- [gnurl] 322/411: FAQ: refresh "Why do I get "certificate verify failed", gnunet, 2021/01/12
- [gnurl] 302/411: RELEASE-NOTES: synced, gnunet, 2021/01/12
- [gnurl] 267/411: libssh2: require version 1.0 or later, gnunet, 2021/01/12
- [gnurl] 286/411: RELEASE-NOTES: synced, gnunet, 2021/01/12
- [gnurl] 308/411: KNOWN_BUGS: LDAPS with NSS is slow, gnunet, 2021/01/12
- [gnurl] 299/411: hsts: add support for Strict-Transport-Security,
gnunet <=
- [gnurl] 300/411: hsts: add read/write callbacks, gnunet, 2021/01/12
- [gnurl] 320/411: url: make sure an HSTS upgrade updates URL and scheme correctly, gnunet, 2021/01/12
- [gnurl] 369/411: examples: update .gitignore, gnunet, 2021/01/12
- [gnurl] 328/411: examples/httpput: remove use of CURLOPT_PUT, gnunet, 2021/01/12
- [gnurl] 325/411: configure: pass -pthread to Libs.private for pkg-config, gnunet, 2021/01/12
- [gnurl] 301/411: curlver: bumped to 7.74.0, gnunet, 2021/01/12
- [gnurl] 332/411: mqttd: fclose test file when done, gnunet, 2021/01/12
- [gnurl] 309/411: KNOWN_BUGS: SMB tests fail with Python 2, gnunet, 2021/01/12
- [gnurl] 324/411: altsvc: minimize variable scope and avoid "DEAD_STORE", gnunet, 2021/01/12
- [gnurl] 282/411: libssh2: fix transport over HTTPS proxy, gnunet, 2021/01/12