diff options
| author | Elliott Hughes <enh@google.com> | 2018-04-27 16:19:43 -0700 |
|---|---|---|
| committer | Elliott Hughes <enh@google.com> | 2018-05-29 13:40:19 -0700 |
| commit | cac3980d6d12690fe9c8aa2ca6bae981c8abb508 (patch) | |
| tree | 9e228338321c29a79ed20af99d3b2eda7e84f984 /lib | |
| parent | 3685a4d3467796179daa96d4c9950b4d2cc34c2e (diff) | |
| download | external_curl-cac3980d6d12690fe9c8aa2ca6bae981c8abb508.tar.gz external_curl-cac3980d6d12690fe9c8aa2ca6bae981c8abb508.tar.bz2 external_curl-cac3980d6d12690fe9c8aa2ca6bae981c8abb508.zip | |
Update to 7.59.0 - March 14 2018.
Changes:
curl: add --proxy-pinnedpubkey
added: CURLOPT_TIMEVALUE_LARGE and CURLINFO_FILETIME_T
CURLOPT_RESOLVE: Add support for multiple IP addresses per entry
Add option CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS
Add new tool option --happy-eyeballs-timeout-ms
Add CURLOPT_RESOLVER_START_FUNCTION and CURLOPT_RESOLVER_START_DATA
Bugfixes:
openldap: check ldap_get_attribute_ber() results for NULL before using
FTP: reject path components with control codes
readwrite: make sure excess reads don't go beyond buffer end
lib555: drop text conversion and encode data as ascii codes
lib517: make variable static to avoid compiler warning
lib544: sync ascii code data with textual data
GSKit: restore pinnedpubkey functionality
darwinssl: Don't import client certificates into Keychain on macOS
parsedate: fix date parsing for systems with 32 bit long
openssl: fix pinned public key build error in FIPS mode
SChannel/WinSSL: Implement public key pinning
cookies: remove verbose "cookie size:" output
progress-bar: don't use stderr explicitly, use bar->out
Fixes for MSDOS
build: open VC15 projects with VS 2017
curl_ctype: private is*() type macros and functions
configure: set PATH_SEPARATOR to colon for PATH w/o separator
winbuild: make linker generate proper PDB
curl_easy_reset: clear digest auth state
curl/curl.h: fix comment typo for CURLOPT_DNS_LOCAL_IP6
range: commonize FTP and FILE range handling
progress-bar docs: update to match implementation
fnmatch: do not match the empty string with a character set
fnmatch: accept an alphanum to be followed by a non-alphanum in char set
build: fix termios issue on android cross-compile
getdate: return -1 for out of range
formdata: use the mime-content type function
time-cond: fix reading the file modification time on Windows
build-openssl.bat: Extend VC15 support to include Enterprise and Professional
build-wolfssl.bat: Extend VC15 support to include Enterprise and Professional
openssl: Don't add verify locations when verifypeer==0
fnmatch: optimize processing of consecutive *s and ?s pattern characters
schannel: fix compiler warnings
content_encoding: Add "none" alias to "identity"
get_posix_time: only check for overflows if they can happen
http_chunks: don't write chunks twice with CURLOPT_HTTP_TRANSFER_DECODING
README: language fix
sha256: build with OpenSSL < 0.9.8
smtp: fix processing of initial dot in data
--tlsauthtype: works only if libcurl is built with TLS-SRP support
tests: new tests for http raw mode
libcurl-security.3: man page discussion security concerns when using libcurl
curl_gssapi: make sure this file too uses our *printf()
BINDINGS: fix curb link (and remove ruby-curl-multi)
nss: use PK11_CreateManagedGenericObject() if available
travis: add build with iconv enabled
ssh: add two missing state names
CURLOPT_HEADERFUNCTION.3: mention folded headers
http: fix the max header length detection logic
header callback: don't chop headers into smaller pieces
CURLOPT_HEADER.3: clarify problems with different data sizes
curl --version: show PSL if the run-time lib has it enabled
examples/sftpuploadresume: resume upload via CURLOPT_APPEND
Return error if called recursively from within callbacks
sasl: prefer PLAIN mechanism over LOGIN
winbuild: Use CALL to run batch scripts
curl_share_setopt.3: connection cache is shared within multi handles
winbuild: Use macros for the names of some build utilities
projects/README: remove reference to dead IDN link/package
lib655: silence compiler warning
configure: Fix version check for OpenSSL 1.1.1
docs/MANUAL: formfind.pl is not accessible on the site anymore
unit1309: fix warning on Windows x64
unit1307: proper cleanup on OOM to fix torture tests
curl_ctype: fix macro redefinition warnings
build: get CFLAGS (including -werror) used for examples and tests
NO_PROXY: fix for IPv6 numericals in the URL
krb5: use nondeprecated functions
winbuild: prefer documented zlib library names
http2: mark the connection for close on GOAWAY
limit-rate: kick in even before "limit" data has been received
HTTP: allow "header;" to replace an internal header with a blank one
http2: verbose output new MAX_CONCURRENT_STREAMS values
SECURITY: distros' max embargo time is 14 days
curl tool: accept --compressed also if Brotli is enabled and zlib is not
WolfSSL: adding TLSv1.3
checksrc.pl: add -i and -m options
CURLOPT_COOKIEFILE.3: "-" as file name means stdin
Bug: http://b/78771319
Test: builds, boots, `vendor/google/tools/fake-ota on streaming` works
Change-Id: I18731dd7c91df36ca92a5dffc8631dee22785e34
Diffstat (limited to 'lib')
69 files changed, 1582 insertions, 921 deletions
diff --git a/lib/Makefile.inc b/lib/Makefile.inc index 61e80cf5..69f9b403 100644 --- a/lib/Makefile.inc +++ b/lib/Makefile.inc @@ -5,7 +5,7 @@ # | (__| |_| | _ <| |___ # \___|\___/|_| \_\_____| # -# Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. +# Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms @@ -46,7 +46,7 @@ LIB_CFILES = file.c timeval.c base64.c hostip.c progress.c formdata.c \ http_digest.c md4.c md5.c http_negotiate.c inet_pton.c strtoofft.c \ strerror.c amigaos.c hostasyn.c hostip4.c hostip6.c hostsyn.c \ inet_ntop.c parsedate.c select.c tftp.c splay.c strdup.c socks.c \ - ssh.c ssh-libssh.c curl_addrinfo.c socks_gssapi.c socks_sspi.c \ + ssh.c ssh-libssh.c curl_addrinfo.c socks_gssapi.c socks_sspi.c \ curl_sspi.c slist.c nonblock.c curl_memrchr.c imap.c pop3.c smtp.c \ pingpong.c rtsp.c curl_threads.c warnless.c hmac.c curl_rtmp.c \ openldap.c curl_gethostname.c gopher.c idn_win32.c \ @@ -54,7 +54,7 @@ LIB_CFILES = file.c timeval.c base64.c hostip.c progress.c formdata.c \ http_ntlm.c curl_ntlm_wb.c curl_ntlm_core.c curl_sasl.c rand.c \ curl_multibyte.c hostcheck.c conncache.c pipeline.c dotdot.c \ x509asn1.c http2.c smb.c curl_endian.c curl_des.c system_win32.c \ - mime.c sha256.c setopt.c curl_path.c + mime.c sha256.c setopt.c curl_path.c curl_ctype.c curl_range.c LIB_HFILES = arpa_telnet.h netrc.h file.h timeval.h hostip.h progress.h \ formdata.h cookie.h http.h sendf.h ftp.h url.h dict.h if2ip.h \ @@ -74,7 +74,7 @@ LIB_HFILES = arpa_telnet.h netrc.h file.h timeval.h hostip.h progress.h \ curl_setup_once.h multihandle.h setup-vms.h pipeline.h dotdot.h \ x509asn1.h http2.h sigpipe.h smb.h curl_endian.h curl_des.h \ curl_printf.h system_win32.h rand.h mime.h curl_sha256.h setopt.h \ - curl_path.h + curl_path.h curl_ctype.h curl_range.h LIB_RCFILES = libcurl.rc diff --git a/lib/checksrc.pl b/lib/checksrc.pl index 92af9000..c86222b2 100755 --- a/lib/checksrc.pl +++ b/lib/checksrc.pl @@ -6,7 +6,7 @@ # | (__| |_| | _ <| |___ # \___|\___/|_| \_\_____| # -# Copyright (C) 2011 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. +# Copyright (C) 2011 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. # # This software is licensed as described in the file COPYING, which # you should have received as part of this distribution. The terms @@ -26,7 +26,7 @@ my $indent = 2; my $warnings; my $errors; -my $supressed; # whitelisted problems +my $suppressed; # whitelisted problems my $file; my $dir="."; my $wlist; @@ -35,10 +35,10 @@ my $verbose; my %whitelist; my %warnings = ( - 'LONGLINE' => "Line longer than $max_column", - 'TABS' => 'TAB characters not allowed', - 'TRAILINGSPACE' => 'Trailing white space on the line', - 'CPPCOMMENTS' => '// comment detected', + 'LONGLINE' => "Line longer than $max_column", + 'TABS' => 'TAB characters not allowed', + 'TRAILINGSPACE' => 'Trailing white space on the line', + 'CPPCOMMENTS' => '// comment detected', 'SPACEBEFOREPAREN' => 'space before an open parenthesis', 'SPACEAFTERPAREN' => 'space after open parenthesis', 'SPACEBEFORECLOSE' => 'space before a close parenthesis', @@ -58,9 +58,9 @@ my %warnings = ( 'OPENCOMMENT' => 'file ended with a /* comment still "open"', 'ASTERISKSPACE' => 'pointer declared with space after asterisk', 'ASTERISKNOSPACE' => 'pointer declared without space before asterisk', - 'ASSIGNWITHINCONDITION' => 'assignment within conditional expression', + 'ASSIGNWITHINCONDITION' => 'assignment within conditional expression', 'EQUALSNOSPACE' => 'equals sign without following space', - 'NOSPACEEQUALS' => 'equals sign without preceeding space', + 'NOSPACEEQUALS' => 'equals sign without preceding space', 'SEMINOSPACE' => 'semicolon without following space', 'MULTISPACE' => 'multiple spaces used when not suitable', ); @@ -101,7 +101,7 @@ sub checkwarn { } if($nowarn) { - $supressed++; + $suppressed++; if($w) { $swarnings++; } @@ -142,6 +142,16 @@ while(1) { $file = shift @ARGV; next; } + elsif($file =~ /-i([1-9])/) { + $indent = $1 + 0; + $file = shift @ARGV; + next; + } + elsif($file =~ /-m([0-9]+)/) { + $max_column = $1 + 0; + $file = shift @ARGV; + next; + } elsif($file =~ /^(-h|--help)/) { undef $file; last; @@ -156,6 +166,8 @@ if(!$file) { print " -D[DIR] Directory to prepend file names\n"; print " -h Show help output\n"; print " -W[file] Whitelist the given file - ignore all its flaws\n"; + print " -i<n> Indent spaces. Default: 2\n"; + print " -m<n> Maximum line length. Default: 79\n"; print "\nDetects and warns for these problems:\n"; for(sort keys %warnings) { printf (" %-18s: %s\n", $_, $warnings{$_}); @@ -422,7 +434,7 @@ sub scanfile { # There is a quote here, figure out whether the comma is # within a string or '' or not. if($pref =~ /\"/) { - # withing a string + # within a string } elsif($pref =~ /\'$/) { # a single letter @@ -596,7 +608,7 @@ sub scanfile { if($errors || $warnings || $verbose) { printf "checksrc: %d errors and %d warnings\n", $errors, $warnings; - if($supressed) { + if($suppressed) { printf "checksrc: %d errors and %d warnings suppressed\n", $serrors, $swarnings; diff --git a/lib/config-win32.h b/lib/config-win32.h index 3e5567df..fdac6f9a 100644 --- a/lib/config-win32.h +++ b/lib/config-win32.h @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -708,6 +708,11 @@ Vista /* Define to use the Windows crypto library. */ #define USE_WIN32_CRYPTO +/* Define to use Unix sockets. */ +#if defined(_MSC_VER) && _MSC_VER >= 1900 +/* #define USE_UNIX_SOCKETS */ +#endif + /* ---------------------------------------------------------------- */ /* ADDITIONAL DEFINITIONS */ /* ---------------------------------------------------------------- */ diff --git a/lib/connect.c b/lib/connect.c index 3edb71eb..1a27ae13 100644 --- a/lib/connect.c +++ b/lib/connect.c @@ -619,8 +619,8 @@ void Curl_persistconninfo(struct connectdata *conn) /* retrieves ip address and port from a sockaddr structure. note it calls Curl_inet_ntop which sets errno on fail, not SOCKERRNO. */ -static bool getaddressinfo(struct sockaddr *sa, char *addr, - long *port) +bool Curl_getaddressinfo(struct sockaddr *sa, char *addr, + long *port) { unsigned short us_port; struct sockaddr_in *si = NULL; @@ -700,16 +700,16 @@ void Curl_updateconninfo(struct connectdata *conn, curl_socket_t sockfd) return; } - if(!getaddressinfo((struct sockaddr*)&ssrem, - conn->primary_ip, &conn->primary_port)) { + if(!Curl_getaddressinfo((struct sockaddr*)&ssrem, + conn->primary_ip, &conn->primary_port)) { failf(data, "ssrem inet_ntop() failed with errno %d: %s", errno, Curl_strerror(conn, errno)); return; } memcpy(conn->ip_addr_str, conn->primary_ip, MAX_IPADR_LEN); - if(!getaddressinfo((struct sockaddr*)&ssloc, - conn->local_ip, &conn->local_port)) { + if(!Curl_getaddressinfo((struct sockaddr*)&ssloc, + conn->local_ip, &conn->local_port)) { failf(data, "ssloc inet_ntop() failed with errno %d: %s", errno, Curl_strerror(conn, errno)); return; @@ -783,7 +783,8 @@ CURLcode Curl_is_connected(struct connectdata *conn, /* should we try another protocol family? */ if(i == 0 && conn->tempaddr[1] == NULL && - Curl_timediff(now, conn->connecttime) >= HAPPY_EYEBALLS_TIMEOUT) { + (Curl_timediff(now, conn->connecttime) >= + data->set.happy_eyeballs_timeout)) { trynextip(conn, sockindex, 1); } } @@ -1005,8 +1006,8 @@ static CURLcode singleipconnect(struct connectdata *conn, return CURLE_OK; /* store remote address and port used in this connection attempt */ - if(!getaddressinfo((struct sockaddr*)&addr.sa_addr, - ipaddress, &port)) { + if(!Curl_getaddressinfo((struct sockaddr*)&addr.sa_addr, + ipaddress, &port)) { /* malformed address or bug in inet_ntop, try next address */ failf(data, "sa_addr inet_ntop() failed with errno %d: %s", errno, Curl_strerror(conn, errno)); @@ -1033,9 +1034,11 @@ static CURLcode singleipconnect(struct connectdata *conn, if(data->set.fsockopt) { /* activate callback for setting socket options */ + Curl_set_in_callback(data, true); error = data->set.fsockopt(data->set.sockopt_client, sockfd, CURLSOCKTYPE_IPCXN); + Curl_set_in_callback(data, false); if(error == CURL_SOCKOPT_ALREADY_CONNECTED) isconnected = TRUE; @@ -1204,7 +1207,8 @@ CURLcode Curl_connecthost(struct connectdata *conn, /* context */ } data->info.numconnects++; /* to track the number of connections made */ - Curl_expire(conn->data, HAPPY_EYEBALLS_TIMEOUT, EXPIRE_HAPPY_EYEBALLS); + Curl_expire(conn->data, data->set.happy_eyeballs_timeout, + EXPIRE_HAPPY_EYEBALLS); return CURLE_OK; } @@ -1311,8 +1315,12 @@ int Curl_closesocket(struct connectdata *conn, status */ conn->sock_accepted[SECONDARYSOCKET] = FALSE; else { + int rc; Curl_multi_closed(conn, sock); - return conn->fclosesocket(conn->closesocket_client, sock); + Curl_set_in_callback(conn->data, true); + rc = conn->fclosesocket(conn->closesocket_client, sock); + Curl_set_in_callback(conn->data, false); + return rc; } } @@ -1363,7 +1371,7 @@ CURLcode Curl_socket(struct connectdata *conn, addr->addrlen = sizeof(struct Curl_sockaddr_storage); memcpy(&addr->sa_addr, ai->ai_addr, addr->addrlen); - if(data->set.fopensocket) + if(data->set.fopensocket) { /* * If the opensocket callback is set, all the destination address * information is passed to the callback. Depending on this information the @@ -1373,9 +1381,12 @@ CURLcode Curl_socket(struct connectdata *conn, * might have been changed and this 'new' address will actually be used * here to connect. */ + Curl_set_in_callback(data, true); *sockfd = data->set.fopensocket(data->set.opensocket_client, CURLSOCKTYPE_IPCXN, (struct curl_sockaddr *)addr); + Curl_set_in_callback(data, false); + } else /* opensocket callback not set, so simply create the socket now */ *sockfd = socket(addr->family, addr->socktype, addr->protocol); diff --git a/lib/connect.h b/lib/connect.h index 39744863..193dc639 100644 --- a/lib/connect.h +++ b/lib/connect.h @@ -41,8 +41,6 @@ timediff_t Curl_timeleft(struct Curl_easy *data, bool duringconnect); #define DEFAULT_CONNECT_TIMEOUT 300000 /* milliseconds == five minutes */ -#define HAPPY_EYEBALLS_TIMEOUT 200 /* milliseconds to wait between - IPv4/IPv6 connection attempts */ /* * Used to extract socket and connectdata struct for the most recent @@ -78,6 +76,11 @@ void Curl_persistconninfo(struct connectdata *conn); int Curl_closesocket(struct connectdata *conn, curl_socket_t sock); /* + * Get presentation format IP address and port from a sockaddr. + */ +bool Curl_getaddressinfo(struct sockaddr *sa, char *addr, long *port); + +/* * The Curl_sockaddr_ex structure is basically libcurl's external API * curl_sockaddr structure with enough space available to directly hold any * protocol-specific address structures. The variable declared here will be diff --git a/lib/content_encoding.c b/lib/content_encoding.c index 46bef0ca..2b2188b7 100644 --- a/lib/content_encoding.c +++ b/lib/content_encoding.c @@ -726,7 +726,7 @@ static void identity_close_writer(struct connectdata *conn, static const content_encoding identity_encoding = { "identity", - NULL, + "none", identity_init_writer, identity_unencode_write, identity_close_writer, diff --git a/lib/cookie.c b/lib/cookie.c index c7afc7ae..63deee16 100644 --- a/lib/cookie.c +++ b/lib/cookie.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -430,9 +430,6 @@ Curl_cookie_add(struct Curl_easy *data, size_t nlen = strlen(name); const char *endofn = &ptr[ nlen ]; - infof(data, "cookie size: name/val %d + %d bytes\n", - nlen, len); - if(nlen >= (MAX_NAME-1) || len >= (MAX_NAME-1) || ((nlen + len) > MAX_NAME)) { /* too long individual name or contents, or too long combination of diff --git a/lib/curl_addrinfo.c b/lib/curl_addrinfo.c index ec76f754..95a3f105 100644 --- a/lib/curl_addrinfo.c +++ b/lib/curl_addrinfo.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -50,6 +50,10 @@ # define in_addr_t unsigned long #endif +#if defined(WIN32) && defined(USE_UNIX_SOCKETS) +#include <afunix.h> +#endif + #include <stddef.h> #include "curl_addrinfo.h" diff --git a/lib/curl_config.h b/lib/curl_config.h index 90a2a586..e8d34704 100644 --- a/lib/curl_config.h +++ b/lib/curl_config.h @@ -143,10 +143,7 @@ /* #undef HAVE_BUILTIN_AVAILABLE */ /* Define to 1 if you have the clock_gettime function and monotonic timer. */ -#if !defined(__APPLE__) -/* CLOCK_MONOTONIC is not defined in mac when building for the host. */ #define HAVE_CLOCK_GETTIME_MONOTONIC 1 -#endif /* Define to 1 if you have the closesocket function. */ /* #undef HAVE_CLOSESOCKET */ @@ -436,9 +433,7 @@ #define HAVE_LIBZ 1 /* Define to 1 if you have the <linux/tcp.h> header file. */ -#if !defined(__APPLE__) #define HAVE_LINUX_TCP_H 1 -#endif /* if your compiler supports LL */ #define HAVE_LL 1 @@ -462,14 +457,10 @@ #define HAVE_MEMORY_H 1 /* Define to 1 if you have the memrchr function or macro. */ -#if !defined(__APPLE__) #define HAVE_MEMRCHR 1 -#endif /* Define to 1 if you have the MSG_NOSIGNAL flag. */ -#if !defined(__APPLE__) #define HAVE_MSG_NOSIGNAL 1 -#endif /* Define to 1 if you have the <netdb.h> header file. */ #define HAVE_NETDB_H 1 @@ -526,6 +517,9 @@ /* Define to 1 if you have the `pipe' function. */ #define HAVE_PIPE 1 +/* if you have the PK11_CreateManagedGenericObject function */ +/* #undef HAVE_PK11_CREATEMANAGEDGENERICOBJECT */ + /* Define to 1 if you have a working poll function. */ #define HAVE_POLL 1 @@ -739,6 +733,9 @@ /* Define to 1 if you have the <time.h> header file. */ #define HAVE_TIME_H 1 +/* Define this if time_t is unsigned */ +/* #undef HAVE_TIME_T_UNSIGNED */ + /* Define to 1 if you have the <unistd.h> header file. */ #define HAVE_UNISTD_H 1 diff --git a/lib/curl_config.h.in b/lib/curl_config.h.in index d7ed28de..0f2a8040 100644 --- a/lib/curl_config.h.in +++ b/lib/curl_config.h.in @@ -516,6 +516,9 @@ /* Define to 1 if you have the `pipe' function. */ #undef HAVE_PIPE +/* if you have the PK11_CreateManagedGenericObject function */ +#undef HAVE_PK11_CREATEMANAGEDGENERICOBJECT + /* Define to 1 if you have a working poll function. */ #undef HAVE_POLL @@ -729,6 +732,9 @@ /* Define to 1 if you have the <time.h> header file. */ #undef HAVE_TIME_H +/* Define this if time_t is unsigned */ +#undef HAVE_TIME_T_UNSIGNED + /* Define to 1 if you have the <unistd.h> header file. */ #undef HAVE_UNISTD_H diff --git a/lib/curl_ctype.c b/lib/curl_ctype.c new file mode 100644 index 00000000..4f5abc20 --- /dev/null +++ b/lib/curl_ctype.c @@ -0,0 +1,122 @@ +/*************************************************************************** + * _ _ ____ _ + * Project ___| | | | _ \| | + * / __| | | | |_) | | + * | (__| |_| | _ <| |___ + * \___|\___/|_| \_\_____| + * + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. + * + * This software is licensed as described in the file COPYING, which + * you should have received as part of this distribution. The terms + * are also available at https://curl.haxx.se/docs/copyright.html. + * + * You may opt to use, copy, modify, merge, publish, distribute and/or sell + * copies of the Software, and permit persons to whom the Software is + * furnished to do so, under the terms of the COPYING file. + * + * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY + * KIND, either express or implied. + * + ***************************************************************************/ + +#include "curl_setup.h" + +#undef _U +#define _U (1<<0) /* upper case */ +#undef _L +#define _L (1<<1) /* lower case */ +#undef _N +#define _N (1<<2) /* decimal numerical digit */ +#undef _S +#define _S (1<<3) /* space */ +#undef _P +#define _P (1<<4) /* punctuation */ +#undef _C +#define _C (1<<5) /* control */ +#undef _X +#define _X (1<<6) /* hexadecimal letter */ +#undef _B +#define _B (1<<7) /* blank */ + +static const unsigned char ascii[128] = { + _C, _C, _C, _C, _C, _C, _C, _C, + _C, _C|_S, _C|_S, _C|_S, _C|_S, _C|_S, _C, _C, + _C, _C, _C, _C, _C, _C, _C, _C, + _C, _C, _C, _C, _C, _C, _C, _C, + _S|_B, _P, _P, _P, _P, _P, _P, _P, + _P, _P, _P, _P, _P, _P, _P, _P, + _N, _N, _N, _N, _N, _N, _N, _N, + _N, _N, _P, _P, _P, _P, _P, _P, + _P, _U|_X, _U|_X, _U|_X, _U|_X, _U|_X, _U|_X, _U, + _U, _U, _U, _U, _U, _U, _U, _U, + _U, _U, _U, _U, _U, _U, _U, _U, + _U, _U, _U, _P, _P, _P, _P, _P, + _P, _L|_X, _L|_X, _L|_X, _L|_X, _L|_X, _L|_X, _L, + _L, _L, _L, _L, _L, _L, _L, _L, + _L, _L, _L, _L, _L, _L, _L, _L, + _L, _L, _L, _P, _P, _P, _P, _C +}; + +int Curl_isspace(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & _S); +} + +int Curl_isdigit(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & _N); +} + +int Curl_isalnum(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_N|_U|_L)); +} + +int Curl_isxdigit(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_N|_X)); +} + +int Curl_isgraph(int c) +{ + if((c < 0) || (c >= 0x80) || (c == ' ')) + return FALSE; + return (ascii[c] & (_N|_X|_U|_L|_P|_S)); +} + +int Curl_isprint(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_N|_X|_U|_L|_P|_S)); +} + +int Curl_isalpha(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_U|_L)); +} + +int Curl_isupper(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_U)); +} + +int Curl_islower(int c) +{ + if((c < 0) || (c >= 0x80)) + return FALSE; + return (ascii[c] & (_L)); +} diff --git a/lib/curl_ctype.h b/lib/curl_ctype.h new file mode 100644 index 00000000..da3bd95a --- /dev/null +++ b/lib/curl_ctype.h @@ -0,0 +1,48 @@ +#ifndef HEADER_CURL_CTYPE_H +#define HEADER_CURL_CTYPE_H +/*************************************************************************** + * _ _ ____ _ + * Project ___| | | | _ \| | + * / __| | | | |_) | | + * | (__| |_| | _ <| |___ + * \___|\___/|_| \_\_____| + * + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. + * + * This software is licensed as described in the file COPYING, which + * you should have received as part of this distribution. The terms + * are also available at https://curl.haxx.se/docs/copyright.html. + * + * You may opt to use, copy, modify, merge, publish, distribute and/or sell + * copies of the Software, and permit persons to whom the Software is + * furnished to do so, under the terms of the COPYING file. + * + * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY + * KIND, either express or implied. + * + ***************************************************************************/ + +int Curl_isspace(int c); +int Curl_isdigit(int c); +int Curl_isalnum(int c); +int Curl_isxdigit(int c); +int Curl_isgraph(int c); +int Curl_isprint(int c); +int Curl_isalpha(int c); +int Curl_isupper(int c); +int Curl_islower(int c); + +#define ISSPACE(x) (Curl_isspace((int) ((unsigned char)x))) +#define ISDIGIT(x) (Curl_isdigit((int) ((unsigned char)x))) +#define ISALNUM(x) (Curl_isalnum((int) ((unsigned char)x))) +#define ISXDIGIT(x) (Curl_isxdigit((int) ((unsigned char)x))) +#define ISGRAPH(x) (Curl_isgraph((int) ((unsigned char)x))) +#define ISALPHA(x) (Curl_isalpha((int) ((unsigned char)x))) +#define ISPRINT(x) (Curl_isprint((int) ((unsigned char)x))) +#define ISUPPER(x) (Curl_isupper((int) ((unsigned char)x))) +#define ISLOWER(x) (Curl_islower((int) ((unsigned char)x))) +#define ISASCII(x) (((x) >= 0) && ((x) <= 0x80)) +#define ISBLANK(x) (int)((((unsigned char)x) == ' ') || \ + (((unsigned char)x) == '\t')) + +#endif /* HEADER_CURL_CTYPE_H */ diff --git a/lib/curl_fnmatch.c b/lib/curl_fnmatch.c index f33bba1f..0179a4f7 100644 --- a/lib/curl_fnmatch.c +++ b/lib/curl_fnmatch.c @@ -47,14 +47,7 @@ #define CURLFNM_UPPER (CURLFNM_CHARSET_LEN + 10) typedef enum { - CURLFNM_LOOP_DEFAULT = 0, - CURLFNM_LOOP_BACKSLASH -} loop_state; - -typedef enum { CURLFNM_SCHS_DEFAULT = 0, - CURLFNM_SCHS_MAYRANGE, - CURLFNM_SCHS_MAYRANGE2, CURLFNM_SCHS_RIGHTBR, CURLFNM_SCHS_RIGHTBRLEFTBR } setcharset_state; @@ -64,6 +57,13 @@ typedef enum { CURLFNM_PKW_DDOT } parsekey_state; +typedef enum { + CCLASS_OTHER = 0, + CCLASS_DIGIT, + CCLASS_UPPER, + CCLASS_LOWER +} char_class; + #define SETCHARSET_OK 1 #define SETCHARSET_FAIL 0 @@ -81,12 +81,12 @@ static int parsekeyword(unsigned char **pattern, unsigned char *charset) return SETCHARSET_FAIL; switch(state) { case CURLFNM_PKW_INIT: - if(ISALPHA(c) && ISLOWER(c)) + if(ISLOWER(c)) keyword[i] = c; else if(c == ':') state = CURLFNM_PKW_DDOT; else - return 0; + return SETCHARSET_FAIL; break; case CURLFNM_PKW_DDOT: if(c == ']') @@ -123,14 +123,48 @@ static int parsekeyword(unsigned char **pattern, unsigned char *charset) return SETCHARSET_OK; } +/* Return the character class. */ +static char_class charclass(unsigned char c) +{ + if(ISUPPER(c)) + return CCLASS_UPPER; + if(ISLOWER(c)) + return CCLASS_LOWER; + if(ISDIGIT(c)) + return CCLASS_DIGIT; + return CCLASS_OTHER; +} + +/* Include a character or a range in set. */ +static void setcharorrange(unsigned char **pp, unsigned char *charset) +{ + unsigned char *p = (*pp)++; + unsigned char c = *p++; + + charset[c] = 1; + if(ISALNUM(c) && *p++ == '-') { + char_class cc = charclass(c); + unsigned char endrange = *p++; + + if(endrange == '\\') + endrange = *p++; + if(endrange >= c && charclass(endrange) == cc) { + while(c++ != endrange) + if(charclass(c) == cc) /* Chars in class may be not consecutive. */ + charset[c] = 1; + *pp = p; + } + } +} + /* returns 1 (true) if pattern is OK, 0 if is bad ("p" is pattern pointer) */ static int setcharset(unsigned char **p, unsigned char *charset) { setcharset_state state = CURLFNM_SCHS_DEFAULT; - unsigned char rangestart = 0; - unsigned char lastchar = 0; bool something_found = FALSE; unsigned char c; + + memset(charset, 0, CURLFNM_CHSET_SIZE); for(;;) { c = **p; if(!c) @@ -138,14 +172,7 @@ static int setcharset(unsigned char **p, unsigned char *charset) switch(state) { case CURLFNM_SCHS_DEFAULT: - if(ISALNUM(c)) { /* ASCII value */ - rangestart = c; - charset[c] = 1; - (*p)++; - state = CURLFNM_SCHS_MAYRANGE; - something_found = TRUE; - } - else if(c == ']') { + if(c == ']') { if(something_found) return SETCHARSET_OK; something_found = TRUE; @@ -154,26 +181,16 @@ static int setcharset(unsigned char **p, unsigned char *charset) (*p)++; } else if(c == '[') { - char c2 = *((*p) + 1); - if(c2 == ':') { /* there has to be a keyword */ - (*p) += 2; - if(parsekeyword(p, charset)) { - state = CURLFNM_SCHS_DEFAULT; - } - else - return SETCHARSET_FAIL; - } + unsigned char *pp = *p + 1; + + if(*pp++ == ':' && parsekeyword(&pp, charset)) + *p = pp; else { charset[c] = 1; (*p)++; } something_found = TRUE; } - else if(c == '?' || c == '*') { - something_found = TRUE; - charset[c] = 1; - (*p)++; - } else if(c == '^' || c == '!') { if(!something_found) { if(charset[CURLFNM_NEGATE]) { @@ -189,82 +206,17 @@ static int setcharset(unsigned char **p, unsigned char *charset) } else if(c == '\\') { c = *(++(*p)); - if(ISPRINT((c))) { - something_found = TRUE; - state = CURLFNM_SCHS_MAYRANGE; - charset[c] = 1; - rangestart = c; - (*p)++; - } + if(c) + setcharorrange(p, charset); else - return SETCHARSET_FAIL; + charset['\\'] = 1; + something_found = TRUE; } else { - charset[c] = 1; - (*p)++; + setcharorrange(p, charset); something_found = TRUE; } break; - case CURLFNM_SCHS_MAYRANGE: - if(c == '-') { - charset[c] = 1; - (*p)++; - lastchar = '-'; - state = CURLFNM_SCHS_MAYRANGE2; - } - else if(c == '[') { - state = CURLFNM_SCHS_DEFAULT; - } - else if(ISALNUM(c)) { - charset[c] = 1; - (*p)++; - } - else if(c == '\\') { - c = *(++(*p)); - if(ISPRINT(c)) { - charset[c] = 1; - (*p)++; - } - else - return SETCHARSET_FAIL; - } - else if(c == ']') { - return SETCHARSET_OK; - } - else - return SETCHARSET_FAIL; - break; - case CURLFNM_SCHS_MAYRANGE2: - if(c == ']') { - return SETCHARSET_OK; - } - else if(c == '\\') { - c = *(++(*p)); - if(ISPRINT(c)) { - charset[c] = 1; - state = CURLFNM_SCHS_DEFAULT; - (*p)++; - } - else - return SETCHARSET_FAIL; - } - else if(c >= rangestart) { - if((ISLOWER(c) && ISLOWER(rangestart)) || - (ISDIGIT(c) && ISDIGIT(rangestart)) || - (ISUPPER(c) && ISUPPER(rangestart))) { - charset[lastchar] = 0; - rangestart++; - while(rangestart++ <= c) - charset[rangestart-1] = 1; - (*p)++; - state = CURLFNM_SCHS_DEFAULT; - } - else - return SETCHARSET_FAIL; - } - else - return SETCHARSET_FAIL; - break; case CURLFNM_SCHS_RIGHTBR: if(c == '[') { state = CURLFNM_SCHS_RIGHTBRLEFTBR; @@ -286,14 +238,11 @@ static int setcharset(unsigned char **p, unsigned char *charset) goto fail; break; case CURLFNM_SCHS_RIGHTBRLEFTBR: - if(c == ']') { + if(c == ']') return SETCHARSET_OK; - } - else { - state = CURLFNM_SCHS_DEFAULT; - charset[c] = 1; - (*p)++; - } + state = CURLFNM_SCHS_DEFAULT; + charset[c] = 1; + (*p)++; break; } } @@ -304,107 +253,93 @@ fail: static int loop(const unsigned char *pattern, const unsigned char *string, int maxstars) { - loop_state state = CURLFNM_LOOP_DEFAULT; unsigned char *p = (unsigned char *)pattern; unsigned char *s = (unsigned char *)string; unsigned char charset[CURLFNM_CHSET_SIZE] = { 0 }; - int rc = 0; for(;;) { - switch(state) { - case CURLFNM_LOOP_DEFAULT: - if(*p == '*') { - if(!maxstars) - return CURL_FNMATCH_NOMATCH; - while(*(p + 1) == '*') /* eliminate multiple stars */ - p++; - if(*s == '\0' && *(p + 1) == '\0') - return CURL_FNMATCH_MATCH; - rc = loop(p + 1, s, maxstars - 1); /* *.txt matches .txt <=> - .txt matches .txt */ - if(rc == CURL_FNMATCH_MATCH) + unsigned char *pp; + + switch(*p) { + case '*': + if(!maxstars) + return CURL_FNMATCH_NOMATCH; + /* Regroup consecutive stars and question marks. This can be done because + '*?*?*' can be expressed as '??*'. */ + for(;;) { + if(*++p == '\0') return CURL_FNMATCH_MATCH; - if(*s) /* let the star eat up one character */ - s++; - else - return CURL_FNMATCH_NOMATCH; - } - else if(*p == '?') { - if(ISPRINT(*s)) { - s++; - p++; + if(*p == '?') { + if(!*s++) + return CURL_FNMATCH_NOMATCH; } - else if(*s == '\0') - return CURL_FNMATCH_NOMATCH; - else - return CURL_FNMATCH_FAIL; /* cannot deal with other character */ + else if(*p != '*') + break; } - else if(*p == '\0') { - if(*s == '\0') + /* Skip string characters until we find a match with pattern suffix. */ + for(maxstars--; *s; s++) { + if(loop(p, s, maxstars) == CURL_FNMATCH_MATCH) return CURL_FNMATCH_MATCH; - return CURL_FNMATCH_NOMATCH; } - else if(*p == '\\') { - state = CURLFNM_LOOP_BACKSLASH; + return CURL_FNMATCH_NOMATCH; + case '?': + if(!*s) + return CURL_FNMATCH_NOMATCH; + s++; + p++; + break; + case '\0': + return *s? CURL_FNMATCH_NOMATCH: CURL_FNMATCH_MATCH; + case '\\': + if(p[1]) p++; - } - else if(*p == '[') { - unsigned char *pp = p + 1; /* cannot handle with pointer to register */ - if(setcharset(&pp, charset)) { - int found = FALSE; - if(charset[(unsigned int)*s]) - found = TRUE; - else if(charset[CURLFNM_ALNUM]) - found = ISALNUM(*s); - else if(charset[CURLFNM_ALPHA]) - found = ISALPHA(*s); - else if(charset[CURLFNM_DIGIT]) - found = ISDIGIT(*s); - else if(charset[CURLFNM_XDIGIT]) - found = ISXDIGIT(*s); - else if(charset[CURLFNM_PRINT]) - found = ISPRINT(*s); - else if(charset[CURLFNM_SPACE]) - found = ISSPACE(*s); - else if(charset[CURLFNM_UPPER]) - found = ISUPPER(*s); - else if(charset[CURLFNM_LOWER]) - found = ISLOWER(*s); - else if(charset[CURLFNM_BLANK]) - found = ISBLANK(*s); - else if(charset[CURLFNM_GRAPH]) - found = ISGRAPH(*s); + if(*s++ != *p++) + return CURL_FNMATCH_NOMATCH; + break; + case '[': + pp = p + 1; /* Copy in case of syntax error in set. */ + if(setcharset(&pp, charset)) { + int found = FALSE; + if(!*s) + return CURL_FNMATCH_NOMATCH; + if(charset[(unsigned int)*s]) + found = TRUE; + else if(charset[CURLFNM_ALNUM]) + found = ISALNUM(*s); + else if(charset[CURLFNM_ALPHA]) + found = ISALPHA(*s); + else if(charset[CURLFNM_DIGIT]) + found = ISDIGIT(*s); + else if(charset[CURLFNM_XDIGIT]) + found = ISXDIGIT(*s); + else if(charset[CURLFNM_PRINT]) + found = ISPRINT(*s); + else if(charset[CURLFNM_SPACE]) + found = ISSPACE(*s); + else if(charset[CURLFNM_UPPER]) + found = ISUPPER(*s); + else if(charset[CURLFNM_LOWER]) + found = ISLOWER(*s); + else if(charset[CURLFNM_BLANK]) + found = ISBLANK(*s); + else if(charset[CURLFNM_GRAPH]) + found = ISGRAPH(*s); - if(charset[CURLFNM_NEGATE]) - found = !found; + if(charset[CURLFNM_NEGATE]) + found = !found; - if(found) { - p = pp + 1; - if(*s) - /* don't advance if we're matching on an empty string */ - s++; - memset(charset, 0, CURLFNM_CHSET_SIZE); - } - else - return CURL_FNMATCH_NOMATCH; - } - else - return CURL_FNMATCH_FAIL; - } - else { - if(*p++ != *s++) + if(!found) return CURL_FNMATCH_NOMATCH; + p = pp + 1; + s++; + break; } - break; - case CURLFNM_LOOP_BACKSLASH: - if(ISPRINT(*p)) { - if(*p++ == *s++) - state = CURLFNM_LOOP_DEFAULT; - else - return CURL_FNMATCH_NOMATCH; - } - else - return CURL_FNMATCH_FAIL; + + /* Syntax error in set: this must be taken as a regular character. */ + /* FALLTHROUGH */ + default: + if(*p++ != *s++) + return CURL_FNMATCH_NOMATCH; break; } } diff --git a/lib/curl_gssapi.c b/lib/curl_gssapi.c index 83f3fa0c..f007986c 100644 --- a/lib/curl_gssapi.c +++ b/lib/curl_gssapi.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 2011 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 2011 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -27,6 +27,11 @@ #include "curl_gssapi.h" #include "sendf.h" +/* The last 3 #include files should be in this order */ +#include "curl_printf.h" +#include "curl_memory.h" +#include "memdebug.h" + static char spnego_oid_bytes[] = "\x2b\x06\x01\x05\x05\x02"; gss_OID_desc Curl_spnego_mech_oid = { 6, &spnego_oid_bytes }; static char krb5_oid_bytes[] = "\x2a\x86\x48\x86\xf7\x12\x01\x02\x02"; diff --git a/lib/curl_ntlm_wb.c b/lib/curl_ntlm_wb.c index 03f47a3a..353a6564 100644 --- a/lib/curl_ntlm_wb.c +++ b/lib/curl_ntlm_wb.c @@ -364,7 +364,7 @@ CURLcode Curl_output_ntlm_wb(struct connectdata *conn, case NTLMSTATE_TYPE1: default: /* Use Samba's 'winbind' daemon to support NTLM authentication, - * by delegating the NTLM challenge/response protocal to a helper + * by delegating the NTLM challenge/response protocol to a helper * in ntlm_auth. * http://devel.squid-cache.org/ntlm/squid_helper_protocol.html * https://www.samba.org/samba/docs/man/manpages-3/winbindd.8.html diff --git a/lib/curl_range.c b/lib/curl_range.c new file mode 100644 index 00000000..aa3c4933 --- /dev/null +++ b/lib/curl_range.c @@ -0,0 +1,95 @@ +/*************************************************************************** + * _ _ ____ _ + * Project ___| | | | _ \| | + * / __| | | | |_) | | + * | (__| |_| | _ <| |___ + * \___|\___/|_| \_\_____| + * + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. + * + * This software is licensed as described in the file COPYING, which + * you should have received as part of this distribution. The terms + * are also available at https://curl.haxx.se/docs/copyright.html. + * + * You may opt to use, copy, modify, merge, publish, distribute and/or sell + * copies of the Software, and permit persons to whom the Software is + * furnished to do so, under the terms of the COPYING file. + * + * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY + * KIND, either express or implied. + * + ***************************************************************************/ + +#include "curl_setup.h" +#include <curl/curl.h> +#include "curl_range.h" +#include "sendf.h" +#include "strtoofft.h" + +/* Only include this function if one or more of FTP, FILE are enabled. */ +#if !defined(CURL_DISABLE_FTP) || !defined(CURL_DISABLE_FILE) + + /* + Check if this is a range download, and if so, set the internal variables + properly. + */ +CURLcode Curl_range(struct connectdata *conn) +{ + curl_off_t from, to; + char *ptr; + char *ptr2; + struct Curl_easy *data = conn->data; + + if(data->state.use_range && data->state.range) { + CURLofft from_t; + CURLofft to_t; + from_t = curlx_strtoofft(data->state.range, &ptr, 0, &from); + if(from_t == CURL_OFFT_FLOW) + return CURLE_RANGE_ERROR; + while(*ptr && (ISSPACE(*ptr) || (*ptr == '-'))) + ptr++; + to_t = curlx_strtoofft(ptr, &ptr2, 0, &to); + if(to_t == CURL_OFFT_FLOW) + return CURLE_RANGE_ERROR; + if((to_t == CURL_OFFT_INVAL) && !from_t) { + /* X - */ + data->state.resume_from = from; + DEBUGF(infof(data, "RANGE %" CURL_FORMAT_CURL_OFF_T " to end of file\n", + from)); + } + else if((from_t == CURL_OFFT_INVAL) && !to_t) { + /* -Y */ + data->req.maxdownload = to; + data->state.resume_from = -to; + DEBUGF(infof(data, "RANGE the last %" CURL_FORMAT_CURL_OFF_T " bytes\n", + to)); + } + else { + /* X-Y */ + curl_off_t totalsize; + + /* Ensure the range is sensible - to should follow from. */ + if(from > to) + return CURLE_RANGE_ERROR; + + totalsize = to - from; + if(totalsize == CURL_OFF_T_MAX) + return CURLE_RANGE_ERROR; + + data->req.maxdownload = totalsize + 1; /* include last byte */ + data->state.resume_from = from; + DEBUGF(infof(data, "RANGE from %" CURL_FORMAT_CURL_OFF_T + " getting %" CURL_FORMAT_CURL_OFF_T " bytes\n", + from, data->req.maxdownload)); + } + DEBUGF(infof(data, "range-download from %" CURL_FORMAT_CURL_OFF_T + " to %" CURL_FORMAT_CURL_OFF_T ", totally %" + CURL_FORMAT_CURL_OFF_T " bytes\n", + from, to, data->req.maxdownload)); + } + else + data->req.maxdownload = -1; + return CURLE_OK; +} + +#endif diff --git a/lib/curl_range.h b/lib/curl_range.h new file mode 100644 index 00000000..2350df99 --- /dev/null +++ b/lib/curl_range.h @@ -0,0 +1,30 @@ +#ifndef HEADER_CURL_RANGE_H +#define HEADER_CURL_RANGE_H +/*************************************************************************** + * _ _ ____ _ + * Project ___| | | | _ \| | + * / __| | | | |_) | | + * | (__| |_| | _ <| |___ + * \___|\___/|_| \_\_____| + * + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. + * + * This software is licensed as described in the file COPYING, which + * you should have received as part of this distribution. The terms + * are also available at https://curl.haxx.se/docs/copyright.html. + * + * You may opt to use, copy, modify, merge, publish, distribute and/or sell + * copies of the Software, and permit persons to whom the Software is + * furnished to do so, under the terms of the COPYING file. + * + * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY + * KIND, either express or implied. + * + ***************************************************************************/ + +#include "curl_setup.h" +#include "urldata.h" + +CURLcode Curl_range(struct connectdata *conn); + +#endif /* HEADER_CURL_RANGE_H */ diff --git a/lib/curl_sasl.c b/lib/curl_sasl.c index 550433d6..7052bd91 100644 --- a/lib/curl_sasl.c +++ b/lib/curl_sasl.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 2012 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 2012 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -361,15 +361,6 @@ CURLcode Curl_sasl_start(struct SASL *sasl, struct connectdata *conn, conn->oauth_bearer, &resp, &len); } - else if(enabledmechs & SASL_MECH_LOGIN) { - mech = SASL_MECH_STRING_LOGIN; - state1 = SASL_LOGIN; - state2 = SASL_LOGIN_PASSWD; - sasl->authused = SASL_MECH_LOGIN; - - if(force_ir || data->set.sasl_ir) - result = Curl_auth_create_login_message(data, conn->user, &resp, &len); - } else if(enabledmechs & SASL_MECH_PLAIN) { mech = SASL_MECH_STRING_PLAIN; state1 = SASL_PLAIN; @@ -379,6 +370,15 @@ CURLcode Curl_sasl_start(struct SASL *sasl, struct connectdata *conn, result = Curl_auth_create_plain_message(data, conn->user, conn->passwd, &resp, &len); } + else if(enabledmechs & SASL_MECH_LOGIN) { + mech = SASL_MECH_STRING_LOGIN; + state1 = SASL_LOGIN; + state2 = SASL_LOGIN_PASSWD; + sasl->authused = SASL_MECH_LOGIN; + + if(force_ir || data->set.sasl_ir) + result = Curl_auth_create_login_message(data, conn->user, &resp, &len); + } } if(!result && mech) { diff --git a/lib/curl_setup.h b/lib/curl_setup.h index 609ee9ea..f128696e 100644 --- a/lib/curl_setup.h +++ b/lib/curl_setup.h @@ -389,6 +389,11 @@ # define LSEEK_ERROR (off_t)-1 #endif +#ifndef SIZEOF_TIME_T +/* assume default size of time_t to be 32 bit */ +#define SIZEOF_TIME_T 4 +#endif + /* * Default sizeof(off_t) in case it hasn't been defined in config file. */ @@ -424,6 +429,24 @@ #endif #define CURL_OFF_T_MIN (-CURL_OFF_T_MAX - CURL_OFF_T_C(1)) +#if (SIZEOF_TIME_T == 4) +# ifdef HAVE_TIME_T_UNSIGNED +# define TIME_T_MAX UINT_MAX +# define TIME_T_MIN 0 +# else +# define TIME_T_MAX INT_MAX +# define TIME_T_MIN INT_MIN +# endif +#else +# ifdef HAVE_TIME_T_UNSIGNED +# define TIME_T_MAX 0xFFFFFFFFFFFFFFFF +# define TIME_T_MIN 0 +# else +# define TIME_T_MAX 0x7FFFFFFFFFFFFFFF +# define TIME_T_MIN (-TIME_T_MAX - 1) +# endif +#endif + /* * Arg 2 type for gethostname in case it hasn't been defined in config file. */ @@ -607,11 +630,6 @@ int netware_init(void); #error "Both libidn2 and WinIDN are enabled, choose one." #endif -#ifndef SIZEOF_TIME_T -/* assume default size of time_t to be 32 bit */ -#define SIZEOF_TIME_T 4 -#endif - #define LIBIDN_REQUIRED_VERSION "0.4.1" #if defined(USE_GNUTLS) || defined(USE_OPENSSL) || defined(USE_NSS) || \ @@ -751,11 +769,11 @@ endings either CRLF or LF so 't' is appropriate. # if defined(WIN32) || defined(__CYGWIN__) # define USE_RECV_BEFORE_SEND_WORKAROUND # endif -#else /* DONT_USE_RECV_BEFORE_SEND_WORKAROUNDS */ +#else /* DONT_USE_RECV_BEFORE_SEND_WORKAROUND */ # ifdef USE_RECV_BEFORE_SEND_WORKAROUND # undef USE_RECV_BEFORE_SEND_WORKAROUND # endif -#endif /* DONT_USE_RECV_BEFORE_SEND_WORKAROUNDS */ +#endif /* DONT_USE_RECV_BEFORE_SEND_WORKAROUND */ /* Detect Windows App environment which has a restricted access * to the Win32 APIs. */ diff --git a/lib/curl_setup_once.h b/lib/curl_setup_once.h index a5b542c6..6d01ea15 100644 --- a/lib/curl_setup_once.h +++ b/lib/curl_setup_once.h @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2013, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -101,7 +101,6 @@ # endif #endif - /* * Definition of timeval struct for platforms that don't have it. */ @@ -274,25 +273,6 @@ struct timeval { # define sfcntl fcntl #endif -/* - * Uppercase macro versions of ANSI/ISO is*() functions/macros which - * avoid negative number inputs with argument byte codes > 127. - */ - -#define ISSPACE(x) (isspace((int) ((unsigned char)x))) -#define ISDIGIT(x) (isdigit((int) ((unsigned char)x))) -#define ISALNUM(x) (isalnum((int) ((unsigned char)x))) -#define ISXDIGIT(x) (isxdigit((int) ((unsigned char)x))) -#define ISGRAPH(x) (isgraph((int) ((unsigned char)x))) -#define ISALPHA(x) (isalpha((int) ((unsigned char)x))) -#define ISPRINT(x) (isprint((int) ((unsigned char)x))) -#define ISUPPER(x) (isupper((int) ((unsigned char)x))) -#define ISLOWER(x) (islower((int) ((unsigned char)x))) -#define ISASCII(x) (isascii((int) ((unsigned char)x))) - -#define ISBLANK(x) (int)((((unsigned char)x) == ' ') || \ - (((unsigned char)x) == '\t')) - #define TOLOWER(x) (tolower((int) ((unsigned char)x))) @@ -347,6 +327,7 @@ struct timeval { #define FALSE false #endif +#include "curl_ctype.h" /* * Macro WHILE_FALSE may be used to build single-iteration do-while loops, @@ -61,6 +61,7 @@ #include "strdup.h" #include "progress.h" #include "easyif.h" +#include "multiif.h" #include "select.h" #include "sendf.h" /* for failf function prototype */ #include "connect.h" /* for Curl_getconnectinfo */ @@ -73,6 +74,7 @@ #include "sigpipe.h" #include "ssh.h" #include "setopt.h" +#include "http_digest.h" /* The last 3 #include files should be in this order */ #include "curl_printf.h" @@ -760,6 +762,9 @@ static CURLcode easy_perform(struct Curl_easy *data, bool events) data->multi_easy = multi; } + if(multi->in_callback) + return CURLE_RECURSIVE_API_CALL; + /* Copy the MAXCONNECTS option to the multi handle */ curl_multi_setopt(multi, CURLMOPT_MAXCONNECTS, data->set.maxconnects); @@ -1017,6 +1022,7 @@ void curl_easy_reset(struct Curl_easy *data) /* zero out authentication data: */ memset(&data->state.authhost, 0, sizeof(struct auth)); memset(&data->state.authproxy, 0, sizeof(struct auth)); + Curl_digest_cleanup(data); } /* @@ -1028,6 +1034,9 @@ void curl_easy_reset(struct Curl_easy *data) * the pausing, you may get your write callback called at this point. * * Action is a bitmask consisting of CURLPAUSE_* bits in curl/curl.h + * + * NOTE: This is one of few API functions that are allowed to be called from + * within a callback. */ CURLcode curl_easy_pause(struct Curl_easy *data, int action) { @@ -1070,10 +1079,8 @@ CURLcode curl_easy_pause(struct Curl_easy *data, int action) /* even if one function returns error, this loops through and frees all buffers */ if(!result) - result = Curl_client_chop_write(conn, - writebuf[i].type, - writebuf[i].buf, - writebuf[i].len); + result = Curl_client_write(conn, writebuf[i].type, writebuf[i].buf, + writebuf[i].len); free(writebuf[i].buf); } @@ -1132,6 +1139,9 @@ CURLcode curl_easy_recv(struct Curl_easy *data, void *buffer, size_t buflen, ssize_t n1; struct connectdata *c; + if(Curl_is_in_callback(data)) + return CURLE_RECURSIVE_API_CALL; + result = easy_connection(data, &sfd, &c); if(result) return result; @@ -1159,6 +1169,9 @@ CURLcode curl_easy_send(struct Curl_easy *data, const void *buffer, ssize_t n1; struct connectdata *c = NULL; + if(Curl_is_in_callback(data)) + return CURLE_RECURSIVE_API_CALL; + result = easy_connection(data, &sfd, &c); if(result) return result; @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -61,6 +61,7 @@ #include "url.h" #include "parsedate.h" /* for the week day and month names */ #include "warnless.h" +#include "curl_range.h" /* The last 3 #include files should be in this order */ #include "curl_printf.h" #include "curl_memory.h" @@ -125,65 +126,6 @@ static CURLcode file_setup_connection(struct connectdata *conn) return CURLE_OK; } - /* - Check if this is a range download, and if so, set the internal variables - properly. This code is copied from the FTP implementation and might as - well be factored out. - */ -static CURLcode file_range(struct connectdata *conn) -{ - curl_off_t from, to; - curl_off_t totalsize = -1; - char *ptr; - char *ptr2; - struct Curl_easy *data = conn->data; - - if(data->state.use_range && data->state.range) { - CURLofft from_t; - CURLofft to_t; - from_t = curlx_strtoofft(data->state.range, &ptr, 0, &from); - if(from_t == CURL_OFFT_FLOW) - return CURLE_RANGE_ERROR; - while(*ptr && (ISSPACE(*ptr) || (*ptr == '-'))) - ptr++; - to_t = curlx_strtoofft(ptr, &ptr2, 0, &to); - if(to_t == CURL_OFFT_FLOW) - return CURLE_RANGE_ERROR; - if((to_t == CURL_OFFT_INVAL) && !from_t) { - /* X - */ - data->state.resume_from = from; - DEBUGF(infof(data, "RANGE %" CURL_FORMAT_CURL_OFF_T " to end of file\n", - from)); - } - else if((from_t == CURL_OFFT_INVAL) && !to_t) { - /* -Y */ - data->req.maxdownload = to; - data->state.resume_from = -to; - DEBUGF(infof(data, "RANGE the last %" CURL_FORMAT_CURL_OFF_T " bytes\n", - to)); - } - else { - /* X-Y */ - totalsize = to-from; - if(totalsize == CURL_OFF_T_MAX) - /* this is too big to increase, so bail out */ - return CURLE_RANGE_ERROR; - data->req.maxdownload = totalsize + 1; /* include last byte */ - data->state.resume_from = from; - DEBUGF(infof(data, "RANGE from %" CURL_FORMAT_CURL_OFF_T - " getting %" CURL_FORMAT_CURL_OFF_T " bytes\n", - from, data->req.maxdownload)); - } - DEBUGF(infof(data, "range-download from %" CURL_FORMAT_CURL_OFF_T - " to %" CURL_FORMAT_CURL_OFF_T ", totally %" - CURL_FORMAT_CURL_OFF_T " bytes\n", - from, to, data->req.maxdownload)); - } - else - data->req.maxdownload = -1; - return CURLE_OK; -} - /* * file_connect() gets called from Curl_protocol_connect() to allow us to * do protocol-specific actions at connect-time. We emulate a @@ -461,12 +403,12 @@ static CURLcode file_do(struct connectdata *conn, bool *done) /* we could stat it, then read out the size */ expected_size = statbuf.st_size; /* and store the modification time */ - data->info.filetime = (long)statbuf.st_mtime; + data->info.filetime = statbuf.st_mtime; fstated = TRUE; } if(fstated && !data->state.range && data->set.timecondition) { - if(!Curl_meets_timecondition(data, (time_t)data->info.filetime)) { + if(!Curl_meets_timecondition(data, data->info.filetime)) { *done = TRUE; return CURLE_OK; } @@ -514,7 +456,9 @@ static CURLcode file_do(struct connectdata *conn, bool *done) } /* Check whether file range has been specified */ - file_range(conn); + result = Curl_range(conn); + if(result) + return result; /* Adjust the start offset in case we want to get the N last bytes * of the stream iff the filesize could be determined */ diff --git a/lib/formdata.c b/lib/formdata.c index d0579c52..f9124105 100644 --- a/lib/formdata.c +++ b/lib/formdata.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -155,60 +155,6 @@ static FormInfo * AddFormInfo(char *value, /*************************************************************************** * - * ContentTypeForFilename() - * - * Provides content type for filename if one of the known types (else - * (either the prevtype or the default is returned). - * - * Returns some valid contenttype for filename. - * - ***************************************************************************/ -static const char *ContentTypeForFilename(const char *filename, - const char *prevtype) -{ - const char *contenttype = NULL; - unsigned int i; - /* - * No type was specified, we scan through a few well-known - * extensions and pick the first we match! - */ - struct ContentType { - const char *extension; - const char *type; - }; - static const struct ContentType ctts[]={ - {".gif", "image/gif"}, - {".jpg", "image/jpeg"}, - {".jpeg", "image/jpeg"}, - {".txt", "text/plain"}, - {".html", "text/html"}, - {".xml", "application/xml"} - }; - - if(prevtype) - /* default to the previously set/used! */ - contenttype = prevtype; - else - contenttype = HTTPPOST_CONTENTTYPE_DEFAULT; - - if(filename) { /* in case a NULL was passed in */ - for(i = 0; i<sizeof(ctts)/sizeof(ctts[0]); i++) { - if(strlen(filename) >= strlen(ctts[i].extension)) { - if(strcasecompare(filename + - strlen(filename) - strlen(ctts[i].extension), - ctts[i].extension)) { - contenttype = ctts[i].type; - break; - } - } - } - } - /* we have a contenttype by now */ - return contenttype; -} - -/*************************************************************************** - * * FormAdd() * * Stores a formpost parameter and builds the appropriate linked list. @@ -627,9 +573,15 @@ CURLFORMcode FormAdd(struct curl_httppost **httppost, !form->contenttype) { char *f = form->flags & HTTPPOST_BUFFER? form->showfilename : form->value; + char const *type; + type = Curl_mime_contenttype(f); + if(!type) + type = prevtype; + if(!type) + type = FILE_CONTENTTYPE_DEFAULT; /* our contenttype is missing */ - form->contenttype = strdup(ContentTypeForFilename(f, prevtype)); + form->contenttype = strdup(type); if(!form->contenttype) { return_value = CURL_FORMADD_MEMORY; break; @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -59,6 +59,7 @@ #include "ftp.h" #include "fileinfo.h" #include "ftplistparser.h" +#include "curl_range.h" #include "curl_sec.h" #include "strtoofft.h" #include "strcase.h" @@ -310,9 +311,11 @@ static CURLcode AcceptServerConnect(struct connectdata *conn) int error = 0; /* activate callback for setting socket options */ + Curl_set_in_callback(data, true); error = data->set.fsockopt(data->set.sockopt_client, s, CURLSOCKTYPE_ACCEPT); + Curl_set_in_callback(data, false); if(error) { close_secondarysocket(conn); @@ -1471,7 +1474,7 @@ static CURLcode ftp_state_list(struct connectdata *conn) slashPos = strrchr(inpath, '/'); n = slashPos - inpath; } - result = Curl_urldecode(data, inpath, n, &lstArg, NULL, FALSE); + result = Curl_urldecode(data, inpath, n, &lstArg, NULL, TRUE); if(result) return result; } @@ -1535,7 +1538,7 @@ static CURLcode ftp_state_type(struct connectdata *conn) date. */ if(data->set.opt_no_body && ftpc->file && ftp_need_type(conn, data->set.prefer_ascii)) { - /* The SIZE command is _not_ RFC 959 specified, and therefor many servers + /* The SIZE command is _not_ RFC 959 specified, and therefore many servers may not support it! It is however the only way we have to get a file's size! */ @@ -1615,8 +1618,10 @@ static CURLcode ftp_state_ul_setup(struct connectdata *conn, /* Let's read off the proper amount of bytes from the input. */ if(conn->seek_func) { + Curl_set_in_callback(data, true); seekerr = conn->seek_func(conn->seek_client, data->state.resume_from, SEEK_SET); + Curl_set_in_callback(data, true); } if(seekerr != CURL_SEEKFUNC_OK) { @@ -2060,7 +2065,7 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn, "%04d%02d%02d %02d:%02d:%02d GMT", year, month, day, hour, minute, second); /* now, convert this into a time() value: */ - data->info.filetime = (long)curl_getdate(timebuf, &secs); + data->info.filetime = curl_getdate(timebuf, &secs); } #ifdef CURL_FTP_HTTPSTYLE_HEAD @@ -2072,7 +2077,7 @@ static CURLcode ftp_state_mdtm_resp(struct connectdata *conn, data->set.get_filetime && (data->info.filetime >= 0) ) { char headerbuf[128]; - time_t filetime = (time_t)data->info.filetime; + time_t filetime = data->info.filetime; struct tm buffer; const struct tm *tm = &buffer; @@ -3180,14 +3185,16 @@ static CURLcode ftp_done(struct connectdata *conn, CURLcode status, if(data->state.wildcardmatch) { if(data->set.chunk_end && ftpc->file) { + Curl_set_in_callback(data, true); data->set.chunk_end(data->wildcard.customptr); + Curl_set_in_callback(data, false); } ftpc->known_filesize = -1; } if(!result) /* get the "raw" path */ - result = Curl_urldecode(data, path_to_use, 0, &path, NULL, FALSE); + result = Curl_urldecode(data, path_to_use, 0, &path, NULL, TRUE); if(result) { /* We can limp along anyway (and should try to since we may already be in * the error path) */ @@ -3463,62 +3470,6 @@ ftp_pasv_verbose(struct connectdata *conn, #endif /* - Check if this is a range download, and if so, set the internal variables - properly. - */ - -static CURLcode ftp_range(struct connectdata *conn) -{ - curl_off_t from, to; - char *ptr; - struct Curl_easy *data = conn->data; - struct ftp_conn *ftpc = &conn->proto.ftpc; - - if(data->state.use_range && data->state.range) { - CURLofft from_t; - CURLofft to_t; - from_t = curlx_strtoofft(data->state.range, &ptr, 0, &from); - if(from_t == CURL_OFFT_FLOW) - return CURLE_RANGE_ERROR; - while(*ptr && (ISSPACE(*ptr) || (*ptr == '-'))) - ptr++; - to_t = curlx_strtoofft(ptr, NULL, 0, &to); - if(to_t == CURL_OFFT_FLOW) - return CURLE_RANGE_ERROR; - if((to_t == CURL_OFFT_INVAL) && !from_t) { - /* X - */ - data->state.resume_from = from; - DEBUGF(infof(conn->data, "FTP RANGE %" CURL_FORMAT_CURL_OFF_T - " to end of file\n", from)); - } - else if(!to_t && (from_t == CURL_OFFT_INVAL)) { - /* -Y */ - data->req.maxdownload = to; - data->state.resume_from = -to; - DEBUGF(infof(conn->data, "FTP RANGE the last %" CURL_FORMAT_CURL_OFF_T - " bytes\n", to)); - } - else { - /* X-Y */ - data->req.maxdownload = (to - from) + 1; /* include last byte */ - data->state.resume_from = from; - DEBUGF(infof(conn->data, "FTP RANGE from %" CURL_FORMAT_CURL_OFF_T - " getting %" CURL_FORMAT_CURL_OFF_T " bytes\n", - from, data->req.maxdownload)); - } - DEBUGF(infof(conn->data, "range-download from %" CURL_FORMAT_CURL_OFF_T - " to %" CURL_FORMAT_CURL_OFF_T ", totally %" - CURL_FORMAT_CURL_OFF_T " bytes\n", - from, to, data->req.maxdownload)); - ftpc->dont_check = TRUE; /* don't check for successful transfer */ - } - else - data->req.maxdownload = -1; - return CURLE_OK; -} - - -/* * ftp_do_more() * * This function shall be called when the second FTP (data) connection is @@ -3640,7 +3591,13 @@ static CURLcode ftp_do_more(struct connectdata *conn, int *completep) /* download */ ftp->downloadsize = -1; /* unknown as of yet */ - result = ftp_range(conn); + result = Curl_range(conn); + + if(result == CURLE_OK && data->req.maxdownload >= 0) { + /* Don't check for successful transfer */ + ftpc->dont_check = TRUE; + } + if(result) ; else if(data->set.ftp_list_only || !ftpc->file) { @@ -3883,8 +3840,11 @@ static CURLcode wc_statemach(struct connectdata *conn) infof(conn->data, "Wildcard - START of \"%s\"\n", finfo->filename); if(conn->data->set.chunk_bgn) { - long userresponse = conn->data->set.chunk_bgn( + long userresponse; + Curl_set_in_callback(conn->data, true); + userresponse = conn->data->set.chunk_bgn( finfo, wildcard->customptr, (int)wildcard->filelist.size); + Curl_set_in_callback(conn->data, false); switch(userresponse) { case CURL_CHUNK_BGN_FUNC_SKIP: infof(conn->data, "Wildcard - \"%s\" skipped by user\n", @@ -3920,8 +3880,11 @@ static CURLcode wc_statemach(struct connectdata *conn) } break; case CURLWC_SKIP: { - if(conn->data->set.chunk_end) + if(conn->data->set.chunk_end) { + Curl_set_in_callback(conn->data, true); conn->data->set.chunk_end(conn->data->wildcard.customptr); + Curl_set_in_callback(conn->data, false); + } Curl_llist_remove(&wildcard->filelist, wildcard->filelist.head, NULL); wildcard->state = (wildcard->filelist.size == 0) ? CURLWC_CLEAN : CURLWC_DOWNLOADING; @@ -4192,7 +4155,7 @@ CURLcode ftp_parse_url_path(struct connectdata *conn) result = Curl_urldecode(conn->data, slash_pos ? cur_pos : "/", slash_pos ? dirlen : 1, &ftpc->dirs[0], NULL, - FALSE); + TRUE); if(result) { freedirs(ftpc); return result; @@ -4299,7 +4262,7 @@ CURLcode ftp_parse_url_path(struct connectdata *conn) size_t dlen; char *path; CURLcode result = - Curl_urldecode(conn->data, data->state.path, 0, &path, &dlen, FALSE); + Curl_urldecode(conn->data, data->state.path, 0, &path, &dlen, TRUE); if(result) { freedirs(ftpc); return result; diff --git a/lib/ftplistparser.c b/lib/ftplistparser.c index 262ac030..7668ea8c 100644 --- a/lib/ftplistparser.c +++ b/lib/ftplistparser.c @@ -49,6 +49,7 @@ #include "ftplistparser.h" #include "curl_fnmatch.h" #include "curl_memory.h" +#include "multiif.h" /* The last #include file should be: */ #include "memdebug.h" @@ -294,6 +295,7 @@ static CURLcode ftp_pl_insert_finfo(struct connectdata *conn, compare = Curl_fnmatch; /* filter pattern-corresponding filenames */ + Curl_set_in_callback(conn->data, true); if(compare(conn->data->set.fnmatch_data, wc->pattern, finfo->filename) == 0) { /* discard symlink which is containing multiple " -> " */ @@ -305,6 +307,7 @@ static CURLcode ftp_pl_insert_finfo(struct connectdata *conn, else { add = FALSE; } + Curl_set_in_callback(conn->data, false); if(add) { Curl_llist_insert_next(llist, llist->tail, finfo, &infop->list); diff --git a/lib/getinfo.c b/lib/getinfo.c index 862ced01..d280eebf 100644 --- a/lib/getinfo.c +++ b/lib/getinfo.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -156,7 +156,12 @@ static CURLcode getinfo_long(struct Curl_easy *data, CURLINFO info, *param_longp = data->info.httpproxycode; break; case CURLINFO_FILETIME: - *param_longp = data->info.filetime; + if(data->info.filetime > LONG_MAX) + *param_longp = LONG_MAX; + else if(data->info.filetime < LONG_MIN) + *param_longp = LONG_MIN; + else + *param_longp = (long)data->info.filetime; break; case CURLINFO_HEADER_SIZE: *param_longp = data->info.header_size; @@ -253,6 +258,9 @@ static CURLcode getinfo_offt(struct Curl_easy *data, CURLINFO info, curl_off_t *param_offt) { switch(info) { + case CURLINFO_FILETIME_T: + *param_offt = (curl_off_t)data->info.filetime; + break; case CURLINFO_SIZE_UPLOAD_T: *param_offt = data->progress.uploaded; break; diff --git a/lib/hostip.c b/lib/hostip.c index 886aeec4..8554d39d 100644 --- a/lib/hostip.c +++ b/lib/hostip.c @@ -58,6 +58,7 @@ #include "strerror.h" #include "url.h" #include "inet_ntop.h" +#include "multiif.h" #include "warnless.h" /* The last 3 #include files should be in this order */ #include "curl_printf.h" @@ -481,6 +482,17 @@ int Curl_resolv(struct connectdata *conn, if(!Curl_ipvalid(conn)) return CURLRESOLV_ERROR; + /* notify the resolver start callback */ + if(data->set.resolver_start) { + int st; + Curl_set_in_callback(data, true); + st = data->set.resolver_start(data->state.resolver, NULL, + data->set.resolver_start_client); + Curl_set_in_callback(data, false); + if(st) + return CURLRESOLV_ERROR; + } + /* If Curl_getaddrinfo() returns NULL, 'respwait' might be set to a non-zero value indicating that we need to wait for the response to the resolve call */ @@ -781,7 +793,7 @@ CURLcode Curl_loadhostpairs(struct Curl_easy *data) { struct curl_slist *hostp; char hostname[256]; - int port; + int port = 0; for(hostp = data->change.resolve; hostp; hostp = hostp->next) { if(!hostp->data) @@ -819,32 +831,95 @@ CURLcode Curl_loadhostpairs(struct Curl_easy *data) } else { struct Curl_dns_entry *dns; - Curl_addrinfo *addr; + Curl_addrinfo *head = NULL, *tail = NULL; char *entry_id; size_t entry_len; - char buffer[256]; - char *address = &buffer[0]; + char address[64]; + char *addresses = NULL; + char *addr_begin; + char *addr_end; + char *port_ptr; + char *end_ptr; + char *host_end; + unsigned long tmp_port; + bool error = true; + + host_end = strchr(hostp->data, ':'); + if(!host_end || + ((host_end - hostp->data) >= (ptrdiff_t)sizeof(hostname))) + goto err; + + memcpy(hostname, hostp->data, host_end - hostp->data); + hostname[host_end - hostp->data] = '\0'; + + port_ptr = host_end + 1; + tmp_port = strtoul(port_ptr, &end_ptr, 10); + if(tmp_port > USHRT_MAX || end_ptr == port_ptr || *end_ptr != ':') + goto err; + + port = (int)tmp_port; + addresses = end_ptr + 1; + + while(*end_ptr) { + size_t alen; + Curl_addrinfo *ai; + + addr_begin = end_ptr + 1; + addr_end = strchr(addr_begin, ','); + if(!addr_end) + addr_end = addr_begin + strlen(addr_begin); + end_ptr = addr_end; + + /* allow IP(v6) address within [brackets] */ + if(*addr_begin == '[') { + if(addr_end == addr_begin || *(addr_end - 1) != ']') + goto err; + ++addr_begin; + --addr_end; + } - if(3 != sscanf(hostp->data, "%255[^:]:%d:%255s", hostname, &port, - address)) { - infof(data, "Couldn't parse CURLOPT_RESOLVE entry '%s'!\n", - hostp->data); - continue; - } + alen = addr_end - addr_begin; + if(!alen) + continue; + + if(alen >= sizeof(address)) + goto err; - /* allow IP(v6) address within [brackets] */ - if(address[0] == '[') { - size_t alen = strlen(address); - if(address[alen-1] != ']') - /* it needs to also end with ] to be valid */ + memcpy(address, addr_begin, alen); + address[alen] = '\0'; + +#ifndef ENABLE_IPV6 + if(strchr(address, ':')) { + infof(data, "Ignoring resolve address '%s', missing IPv6 support.\n", + address); continue; - address[alen-1] = 0; /* zero terminate there */ - address++; /* pass the open bracket */ + } +#endif + + ai = Curl_str2addr(address, port); + if(!ai) { + infof(data, "Resolve address '%s' found illegal!\n", address); + goto err; + } + + if(tail) { + tail->ai_next = ai; + tail = tail->ai_next; + } + else { + head = tail = ai; + } } - addr = Curl_str2addr(address, port); - if(!addr) { - infof(data, "Address in '%s' found illegal!\n", hostp->data); + if(!head) + goto err; + + error = false; + err: + if(error) { + infof(data, "Couldn't parse CURLOPT_RESOLVE entry '%s'!\n", + hostp->data); + Curl_freeaddrinfo(head); continue; } @@ -852,10 +927,9 @@ CURLcode Curl_loadhostpairs(struct Curl_easy *data) entry_id = create_hostcache_id(hostname, port); /* If we can't create the entry id, fail */ if(!entry_id) { - Curl_freeaddrinfo(addr); + Curl_freeaddrinfo(head); return CURLE_OUT_OF_MEMORY; } - entry_len = strlen(entry_id); if(data->share) @@ -869,7 +943,7 @@ CURLcode Curl_loadhostpairs(struct Curl_easy *data) if(!dns) { /* if not in the cache already, put this host in the cache */ - dns = Curl_cache_addr(data, addr, hostname, port); + dns = Curl_cache_addr(data, head, hostname, port); if(dns) { dns->timestamp = 0; /* mark as added by CURLOPT_RESOLVE */ /* release the returned reference; the cache itself will keep the @@ -880,19 +954,19 @@ CURLcode Curl_loadhostpairs(struct Curl_easy *data) else { /* this is a duplicate, free it again */ infof(data, "RESOLVE %s:%d is already cached, %s not stored!\n", - hostname, port, address); - Curl_freeaddrinfo(addr); + hostname, port, addresses); + Curl_freeaddrinfo(head); } if(data->share) Curl_share_unlock(data, CURL_LOCK_DATA_DNS); if(!dns) { - Curl_freeaddrinfo(addr); + Curl_freeaddrinfo(head); return CURLE_OUT_OF_MEMORY; } infof(data, "Added %s:%d:%s to DNS cache\n", - hostname, port, address); + hostname, port, addresses); } } data->change.resolve = NULL; /* dealt with now */ @@ -177,9 +177,9 @@ CURLcode Curl_http_setup_conn(struct connectdata *conn) * if proxy headers are not available, then it will lookup into http header * link list * - * It takes a connectdata struct as input instead of the Curl_easy simply - * to know if this is a proxy request or not, as it then might check a - * different header list. + * It takes a connectdata struct as input instead of the Curl_easy simply to + * know if this is a proxy request or not, as it then might check a different + * header list. Provide the header prefix without colon!. */ char *Curl_checkProxyheaders(const struct connectdata *conn, const char *thisheader) @@ -191,7 +191,8 @@ char *Curl_checkProxyheaders(const struct connectdata *conn, for(head = (conn->bits.proxy && data->set.sep_headers) ? data->set.proxyheaders : data->set.headers; head; head = head->next) { - if(strncasecompare(head->data, thisheader, thislen)) + if(strncasecompare(head->data, thisheader, thislen) && + Curl_headersep(head->data[thislen])) return head->data; } @@ -614,9 +615,9 @@ output_auth_headers(struct connectdata *conn, if(authstatus->picked == CURLAUTH_BASIC) { /* Basic */ if((proxy && conn->bits.proxy_user_passwd && - !Curl_checkProxyheaders(conn, "Proxy-authorization:")) || + !Curl_checkProxyheaders(conn, "Proxy-authorization")) || (!proxy && conn->bits.user_passwd && - !Curl_checkheaders(conn, "Authorization:"))) { + !Curl_checkheaders(conn, "Authorization"))) { auth = "Basic"; result = http_output_basic(conn, proxy); if(result) @@ -1533,7 +1534,7 @@ static CURLcode expect100(struct Curl_easy *data, /* if not doing HTTP 1.0 or version 2, or disabled explicitly, we add an Expect: 100-continue to the headers which actually speeds up post operations (as there is one packet coming back from the web server) */ - ptr = Curl_checkheaders(conn, "Expect:"); + ptr = Curl_checkheaders(conn, "Expect"); if(ptr) { data->state.expect100header = Curl_compareheader(ptr, "Expect:", "100-continue"); @@ -1598,7 +1599,32 @@ CURLcode Curl_add_custom_headers(struct connectdata *conn, headers = h[i]; while(headers) { + char *semicolonp = NULL; ptr = strchr(headers->data, ':'); + if(!ptr) { + char *optr; + /* no colon, semicolon? */ + ptr = strchr(headers->data, ';'); + if(ptr) { + optr = ptr; + ptr++; /* pass the semicolon */ + while(*ptr && ISSPACE(*ptr)) + ptr++; + + if(*ptr) { + /* this may be used for something else in the future */ + optr = NULL; + } + else { + if(*(--ptr) == ';') { + /* send no-value custom header if terminated by semicolon */ + *ptr = ':'; + semicolonp = ptr; + } + } + ptr = optr; + } + } if(ptr) { /* we require a colon for this to be a true header */ @@ -1606,8 +1632,9 @@ CURLcode Curl_add_custom_headers(struct connectdata *conn, while(*ptr && ISSPACE(*ptr)) ptr++; - if(*ptr) { - /* only send this if the contents was non-blank */ + if(*ptr || semicolonp) { + /* only send this if the contents was non-blank or done special */ + CURLcode result = CURLE_OK; if(conn->allocptr.host && /* a Host: header was sent already, don't pass on any custom Host: @@ -1645,40 +1672,12 @@ CURLcode Curl_add_custom_headers(struct connectdata *conn, !strcasecompare(data->state.first_host, conn->host.name))) ; else { - CURLcode result = Curl_add_bufferf(req_buffer, "%s\r\n", - headers->data); - if(result) - return result; - } - } - } - else { - ptr = strchr(headers->data, ';'); - if(ptr) { - - ptr++; /* pass the semicolon */ - while(*ptr && ISSPACE(*ptr)) - ptr++; - - if(*ptr) { - /* this may be used for something else in the future */ - } - else { - if(*(--ptr) == ';') { - CURLcode result; - - /* send no-value custom header if terminated by semicolon */ - *ptr = ':'; - result = Curl_add_bufferf(req_buffer, "%s\r\n", - headers->data); - - /* restore the previous value */ - *ptr = ';'; - - if(result) - return result; - } + result = Curl_add_bufferf(req_buffer, "%s\r\n", headers->data); } + if(semicolonp) + *semicolonp = ';'; /* put back the semicolon */ + if(result) + return result; } } headers = headers->next; @@ -1869,7 +1868,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) it might have been used in the proxy connect, but if we have got a header with the user-agent string specified, we erase the previously made string here. */ - if(Curl_checkheaders(conn, "User-Agent:")) { + if(Curl_checkheaders(conn, "User-Agent")) { free(conn->allocptr.uagent); conn->allocptr.uagent = NULL; } @@ -1890,7 +1889,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) conn->bits.authneg = FALSE; Curl_safefree(conn->allocptr.ref); - if(data->change.referer && !Curl_checkheaders(conn, "Referer:")) { + if(data->change.referer && !Curl_checkheaders(conn, "Referer")) { conn->allocptr.ref = aprintf("Referer: %s\r\n", data->change.referer); if(!conn->allocptr.ref) return CURLE_OUT_OF_MEMORY; @@ -1899,11 +1898,11 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) conn->allocptr.ref = NULL; #if !defined(CURL_DISABLE_COOKIES) - if(data->set.str[STRING_COOKIE] && !Curl_checkheaders(conn, "Cookie:")) + if(data->set.str[STRING_COOKIE] && !Curl_checkheaders(conn, "Cookie")) addcookies = data->set.str[STRING_COOKIE]; #endif - if(!Curl_checkheaders(conn, "Accept-Encoding:") && + if(!Curl_checkheaders(conn, "Accept-Encoding") && data->set.str[STRING_ENCODING]) { Curl_safefree(conn->allocptr.accept_encoding); conn->allocptr.accept_encoding = @@ -1919,22 +1918,29 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) #ifdef HAVE_LIBZ /* we only consider transfer-encoding magic if libz support is built-in */ - if(!Curl_checkheaders(conn, "TE:") && + if(!Curl_checkheaders(conn, "TE") && data->set.http_transfer_encoding) { /* When we are to insert a TE: header in the request, we must also insert TE in a Connection: header, so we need to merge the custom provided Connection: header and prevent the original to get sent. Note that if the user has inserted his/hers own TE: header we don't do this magic but then assume that the user will handle it all! */ - char *cptr = Curl_checkheaders(conn, "Connection:"); + char *cptr = Curl_checkheaders(conn, "Connection"); #define TE_HEADER "TE: gzip\r\n" Curl_safefree(conn->allocptr.te); + if(cptr) { + cptr = Curl_copy_header_value(cptr); + if(!cptr) + return CURLE_OUT_OF_MEMORY; + } + /* Create the (updated) Connection: header */ - conn->allocptr.te = cptr? aprintf("%s, TE\r\n" TE_HEADER, cptr): - strdup("Connection: TE\r\n" TE_HEADER); + conn->allocptr.te = aprintf("Connection: %s%sTE\r\n" TE_HEADER, + cptr ? cptr : "", (cptr && *cptr) ? ", ":""); + free(cptr); if(!conn->allocptr.te) return CURLE_OUT_OF_MEMORY; } @@ -1958,7 +1964,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) } if(http->sendit) { - const char *cthdr = Curl_checkheaders(conn, "Content-Type:"); + const char *cthdr = Curl_checkheaders(conn, "Content-Type"); /* Read and seek body only. */ http->sendit->flags |= MIME_BODY_ONLY; @@ -1982,7 +1988,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) http->postsize = Curl_mime_size(http->sendit); } - ptr = Curl_checkheaders(conn, "Transfer-Encoding:"); + ptr = Curl_checkheaders(conn, "Transfer-Encoding"); if(ptr) { /* Some kind of TE is requested, check if 'chunked' is chosen */ data->req.upload_chunky = @@ -2016,7 +2022,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) Curl_safefree(conn->allocptr.host); - ptr = Curl_checkheaders(conn, "Host:"); + ptr = Curl_checkheaders(conn, "Host"); if(ptr && (!data->state.this_is_a_follow || strcasecompare(data->state.first_host, conn->host.name))) { #if !defined(CURL_DISABLE_COOKIES) @@ -2055,7 +2061,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) #endif if(strcmp("Host:", ptr)) { - conn->allocptr.host = aprintf("%s\r\n", ptr); + conn->allocptr.host = aprintf("Host:%s\r\n", &ptr[5]); if(!conn->allocptr.host) return CURLE_OUT_OF_MEMORY; } @@ -2164,7 +2170,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) } #endif /* CURL_DISABLE_PROXY */ - http->p_accept = Curl_checkheaders(conn, "Accept:")?NULL:"Accept: */*\r\n"; + http->p_accept = Curl_checkheaders(conn, "Accept")?NULL:"Accept: */*\r\n"; if((HTTPREQ_POST == httpreq || HTTPREQ_PUT == httpreq) && data->state.resume_from) { @@ -2191,8 +2197,10 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) /* Now, let's read off the proper amount of bytes from the input. */ if(conn->seek_func) { + Curl_set_in_callback(data, true); seekerr = conn->seek_func(conn->seek_client, data->state.resume_from, SEEK_SET); + Curl_set_in_callback(data, false); } if(seekerr != CURL_SEEKFUNC_OK) { @@ -2243,14 +2251,14 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) * ones if any such are specified. */ if(((httpreq == HTTPREQ_GET) || (httpreq == HTTPREQ_HEAD)) && - !Curl_checkheaders(conn, "Range:")) { + !Curl_checkheaders(conn, "Range")) { /* if a line like this was already allocated, free the previous one */ free(conn->allocptr.rangeline); conn->allocptr.rangeline = aprintf("Range: bytes=%s\r\n", data->state.range); } else if((httpreq == HTTPREQ_POST || httpreq == HTTPREQ_PUT) && - !Curl_checkheaders(conn, "Content-Range:")) { + !Curl_checkheaders(conn, "Content-Range")) { /* if a line like this was already allocated, free the previous one */ free(conn->allocptr.rangeline); @@ -2352,7 +2360,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) conn->allocptr.ref:"" /* Referer: <data> */, (conn->bits.httpproxy && !conn->bits.tunnel_proxy && - !Curl_checkProxyheaders(conn, "Proxy-Connection:"))? + !Curl_checkProxyheaders(conn, "Proxy-Connection"))? "Proxy-Connection: Keep-Alive\r\n":"", te ); @@ -2453,7 +2461,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) postsize = data->state.infilesize; if((postsize != -1) && !data->req.upload_chunky && - (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length:"))) { + (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length"))) { /* only add Content-Length if not uploading chunked */ result = Curl_add_bufferf(req_buffer, "Content-Length: %" CURL_FORMAT_CURL_OFF_T @@ -2515,7 +2523,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) we don't upload data chunked, as RFC2616 forbids us to set both kinds of headers (Transfer-Encoding: chunked and Content-Length) */ if(postsize != -1 && !data->req.upload_chunky && - (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length:"))) { + (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length"))) { /* we allow replacing this header if not during auth negotiation, although it isn't very wise to actually set your own */ result = Curl_add_bufferf(req_buffer, @@ -2540,7 +2548,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) the somewhat bigger ones we allow the app to disable it. Just make sure that the expect100header is always set to the preferred value here. */ - ptr = Curl_checkheaders(conn, "Expect:"); + ptr = Curl_checkheaders(conn, "Expect"); if(ptr) { data->state.expect100header = Curl_compareheader(ptr, "Expect:", "100-continue"); @@ -2594,7 +2602,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) we don't upload data chunked, as RFC2616 forbids us to set both kinds of headers (Transfer-Encoding: chunked and Content-Length) */ if((postsize != -1) && !data->req.upload_chunky && - (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length:"))) { + (conn->bits.authneg || !Curl_checkheaders(conn, "Content-Length"))) { /* we allow replacing this header if not during auth negotiation, although it isn't very wise to actually set your own */ result = Curl_add_bufferf(req_buffer, @@ -2604,7 +2612,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) return result; } - if(!Curl_checkheaders(conn, "Content-Type:")) { + if(!Curl_checkheaders(conn, "Content-Type")) { result = Curl_add_bufferf(req_buffer, "Content-Type: application/" "x-www-form-urlencoded\r\n"); @@ -2616,7 +2624,7 @@ CURLcode Curl_http(struct connectdata *conn, bool *done) the somewhat bigger ones we allow the app to disable it. Just make sure that the expect100header is always set to the preferred value here. */ - ptr = Curl_checkheaders(conn, "Expect:"); + ptr = Curl_checkheaders(conn, "Expect"); if(ptr) { data->state.expect100header = Curl_compareheader(ptr, "Expect:", "100-continue"); @@ -2878,20 +2886,19 @@ static CURLcode header_append(struct Curl_easy *data, struct SingleRequest *k, size_t length) { - if(k->hbuflen + length >= data->state.headersize) { + size_t newsize = k->hbuflen + length; + if(newsize > CURL_MAX_HTTP_HEADER) { + /* The reason to have a max limit for this is to avoid the risk of a bad + server feeding libcurl with a never-ending header that will cause + reallocs infinitely */ + failf(data, "Rejected %zd bytes header (max is %d)!", newsize, + CURL_MAX_HTTP_HEADER); + return CURLE_OUT_OF_MEMORY; + } + if(newsize >= data->state.headersize) { /* We enlarge the header buffer as it is too small */ char *newbuff; size_t hbufp_index; - size_t newsize; - - if(k->hbuflen + length > CURL_MAX_HTTP_HEADER) { - /* The reason to have a max limit for this is to avoid the risk of a bad - server feeding libcurl with a never-ending header that will cause - reallocs infinitely */ - failf(data, "Avoided giant realloc for header (max is %d)!", - CURL_MAX_HTTP_HEADER); - return CURLE_OUT_OF_MEMORY; - } newsize = CURLMAX((k->hbuflen + length) * 3 / 2, data->state.headersize*2); hbufp_index = k->hbufp - data->state.headerbuff; @@ -3692,7 +3699,7 @@ CURLcode Curl_http_readwrite_headers(struct Curl_easy *data, k->timeofdoc = curl_getdate(k->p + strlen("Last-Modified:"), &secs); if(data->set.get_filetime) - data->info.filetime = (long)k->timeofdoc; + data->info.filetime = k->timeofdoc; } else if((checkprefix("WWW-Authenticate:", k->p) && (401 == k->httpcode)) || @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -172,8 +172,6 @@ struct HTTP { size_t pauselen; /* the number of bytes left in data */ bool closed; /* TRUE on HTTP2 stream close */ bool close_handled; /* TRUE if stream closure is handled by libcurl */ - uint32_t error_code; /* HTTP/2 error code */ - char *mem; /* points to a buffer in memory to store received data */ size_t len; /* size of the buffer 'mem' points to */ size_t memlen; /* size of data copied to mem */ @@ -226,6 +224,7 @@ struct http_conn { /* list of settings that will be sent */ nghttp2_settings_entry local_settings[3]; size_t local_settings_num; + uint32_t error_code; /* HTTP/2 error code */ #else int unused; /* prevent a compiler warning */ #endif diff --git a/lib/http2.c b/lib/http2.c index 69928794..b2c34e94 100644 --- a/lib/http2.c +++ b/lib/http2.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -65,6 +65,12 @@ #define HTTP2_HUGE_WINDOW_SIZE (1 << 30) +#ifdef DEBUG_HTTP2 +#define H2BUGF(x) x +#else +#define H2BUGF(x) do { } WHILE_FALSE +#endif + /* * Curl_http2_init_state() is called when the easy handle is created and * allows for HTTP/2 specific init of state. @@ -140,13 +146,13 @@ static CURLcode http2_disconnect(struct connectdata *conn, struct http_conn *c = &conn->proto.httpc; (void)dead_connection; - DEBUGF(infof(conn->data, "HTTP/2 DISCONNECT starts now\n")); + H2BUGF(infof(conn->data, "HTTP/2 DISCONNECT starts now\n")); nghttp2_session_del(c->h2); Curl_safefree(c->inbuf); http2_stream_free(conn->data->req.protop); - DEBUGF(infof(conn->data, "HTTP/2 DISCONNECT done\n")); + H2BUGF(infof(conn->data, "HTTP/2 DISCONNECT done\n")); return CURLE_OK; } @@ -204,7 +210,6 @@ void Curl_http2_setup_req(struct Curl_easy *data) http->status_code = -1; http->pausedata = NULL; http->pauselen = 0; - http->error_code = NGHTTP2_NO_ERROR; http->closed = FALSE; http->close_handled = FALSE; http->mem = data->state.buffer; @@ -217,6 +222,7 @@ void Curl_http2_setup_conn(struct connectdata *conn) { conn->proto.httpc.settings.max_concurrent_streams = DEFAULT_MAX_CONCURRENT_STREAMS; + conn->proto.httpc.error_code = NGHTTP2_NO_ERROR; } /* @@ -428,7 +434,7 @@ static int push_promise(struct Curl_easy *data, const nghttp2_push_promise *frame) { int rv; - DEBUGF(infof(data, "PUSH_PROMISE received, stream %u!\n", + H2BUGF(infof(data, "PUSH_PROMISE received, stream %u!\n", frame->promised_stream_id)); if(data->multi->push_cb) { struct HTTP *stream; @@ -448,7 +454,7 @@ static int push_promise(struct Curl_easy *data, heads.data = data; heads.frame = frame; /* ask the application */ - DEBUGF(infof(data, "Got PUSH_PROMISE, ask application!\n")); + H2BUGF(infof(data, "Got PUSH_PROMISE, ask application!\n")); stream = data->req.protop; if(!stream) { @@ -458,9 +464,11 @@ static int push_promise(struct Curl_easy *data, goto fail; } + Curl_set_in_callback(data, true); rv = data->multi->push_cb(data, newhandle, stream->push_headers_used, &heads, data->multi->push_userp); + Curl_set_in_callback(data, false); /* free the headers again */ for(i = 0; i<stream->push_headers_used; i++) @@ -497,7 +505,7 @@ static int push_promise(struct Curl_easy *data, frame->promised_stream_id, newhandle); } else { - DEBUGF(infof(data, "Got PUSH_PROMISE, ignore it!\n")); + H2BUGF(infof(data, "Got PUSH_PROMISE, ignore it!\n")); rv = 1; } fail: @@ -520,21 +528,22 @@ static int on_frame_recv(nghttp2_session *session, const nghttp2_frame *frame, /* stream ID zero is for connection-oriented stuff */ if(frame->hd.type == NGHTTP2_SETTINGS) { uint32_t max_conn = httpc->settings.max_concurrent_streams; - DEBUGF(infof(conn->data, "Got SETTINGS\n")); + H2BUGF(infof(conn->data, "Got SETTINGS\n")); httpc->settings.max_concurrent_streams = nghttp2_session_get_remote_settings( session, NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS); httpc->settings.enable_push = nghttp2_session_get_remote_settings( session, NGHTTP2_SETTINGS_ENABLE_PUSH); - DEBUGF(infof(conn->data, "MAX_CONCURRENT_STREAMS == %d\n", + H2BUGF(infof(conn->data, "MAX_CONCURRENT_STREAMS == %d\n", httpc->settings.max_concurrent_streams)); - DEBUGF(infof(conn->data, "ENABLE_PUSH == %s\n", + H2BUGF(infof(conn->data, "ENABLE_PUSH == %s\n", httpc->settings.enable_push?"TRUE":"false")); if(max_conn != httpc->settings.max_concurrent_streams) { /* only signal change if the value actually changed */ infof(conn->data, - "Connection state changed (MAX_CONCURRENT_STREAMS updated)!\n"); + "Connection state changed (MAX_CONCURRENT_STREAMS == %d)!\n", + httpc->settings.max_concurrent_streams); Curl_multi_connchanged(conn->data->multi); } } @@ -545,7 +554,7 @@ static int on_frame_recv(nghttp2_session *session, const nghttp2_frame *frame, lastStream = stream_id; } if(!data_s) { - DEBUGF(infof(conn->data, + H2BUGF(infof(conn->data, "No Curl_easy associated with stream: %x\n", stream_id)); return 0; @@ -553,12 +562,12 @@ static int on_frame_recv(nghttp2_session *session, const nghttp2_frame *frame, stream = data_s->req.protop; if(!stream) { - DEBUGF(infof(conn->data, "No proto pointer for stream: %x\n", + H2BUGF(infof(conn->data, "No proto pointer for stream: %x\n", stream_id)); return NGHTTP2_ERR_CALLBACK_FAILURE; } - DEBUGF(infof(data_s, "on_frame_recv() header %x stream %x\n", + H2BUGF(infof(data_s, "on_frame_recv() header %x stream %x\n", frame->hd.type, stream_id)); switch(frame->hd.type) { @@ -600,7 +609,7 @@ static int on_frame_recv(nghttp2_session *session, const nghttp2_frame *frame, ncopy); stream->nread_header_recvbuf += ncopy; - DEBUGF(infof(data_s, "Store %zu bytes headers from stream %u at %p\n", + H2BUGF(infof(data_s, "Store %zu bytes headers from stream %u at %p\n", ncopy, stream_id, stream->mem)); stream->len -= ncopy; @@ -629,7 +638,7 @@ static int on_frame_recv(nghttp2_session *session, const nghttp2_frame *frame, } break; default: - DEBUGF(infof(conn->data, "Got frame type %x for stream %u!\n", + H2BUGF(infof(conn->data, "Got frame type %x for stream %u!\n", frame->hd.type, stream_id)); break; } @@ -642,13 +651,13 @@ static int on_invalid_frame_recv(nghttp2_session *session, { struct Curl_easy *data_s = NULL; (void)userp; -#if !defined(DEBUGBUILD) || defined(CURL_DISABLE_VERBOSE_STRINGS) +#if !defined(DEBUG_HTTP2) || defined(CURL_DISABLE_VERBOSE_STRINGS) (void)lib_error_code; #endif data_s = nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); if(data_s) { - DEBUGF(infof(data_s, + H2BUGF(infof(data_s, "on_invalid_frame_recv() was called, error=%d:%s\n", lib_error_code, nghttp2_strerror(lib_error_code))); } @@ -693,7 +702,7 @@ static int on_data_chunk_recv(nghttp2_session *session, uint8_t flags, if(conn->data != data_s) Curl_expire(data_s, 0, EXPIRE_RUN_NOW); - DEBUGF(infof(data_s, "%zu data received for stream %u " + H2BUGF(infof(data_s, "%zu data received for stream %u " "(%zu left in buffer %p, total %zu)\n", nread, stream_id, stream->len, stream->mem, @@ -702,7 +711,7 @@ static int on_data_chunk_recv(nghttp2_session *session, uint8_t flags, if(nread < len) { stream->pausedata = data + nread; stream->pauselen = len - nread; - DEBUGF(infof(data_s, "NGHTTP2_ERR_PAUSE - %zu bytes out of buffer" + H2BUGF(infof(data_s, "NGHTTP2_ERR_PAUSE - %zu bytes out of buffer" ", stream %u\n", len - nread, stream_id)); data_s->easy_conn->proto.httpc.pause_stream_id = stream_id; @@ -730,7 +739,7 @@ static int before_frame_send(nghttp2_session *session, data_s = nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); if(data_s) { - DEBUGF(infof(data_s, "before_frame_send() was called\n")); + H2BUGF(infof(data_s, "before_frame_send() was called\n")); } return 0; @@ -744,7 +753,7 @@ static int on_frame_send(nghttp2_session *session, data_s = nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); if(data_s) { - DEBUGF(infof(data_s, "on_frame_send() was called, length = %zd\n", + H2BUGF(infof(data_s, "on_frame_send() was called, length = %zd\n", frame->hd.length)); } return 0; @@ -755,13 +764,13 @@ static int on_frame_not_send(nghttp2_session *session, { struct Curl_easy *data_s; (void)userp; -#if !defined(DEBUGBUILD) || defined(CURL_DISABLE_VERBOSE_STRINGS) +#if !defined(DEBUG_HTTP2) || defined(CURL_DISABLE_VERBOSE_STRINGS) (void)lib_error_code; #endif data_s = nghttp2_session_get_stream_user_data(session, frame->hd.stream_id); if(data_s) { - DEBUGF(infof(data_s, + H2BUGF(infof(data_s, "on_frame_not_send() was called, lib_error_code = %d\n", lib_error_code)); } @@ -777,6 +786,7 @@ static int on_stream_close(nghttp2_session *session, int32_t stream_id, (void)stream_id; if(stream_id) { + struct http_conn *httpc; /* get the stream from the hash based on Stream ID, stream ID zero is for connection-oriented stuff */ data_s = nghttp2_session_get_stream_user_data(session, stream_id); @@ -785,20 +795,21 @@ static int on_stream_close(nghttp2_session *session, int32_t stream_id, decided to reject stream (e.g., PUSH_PROMISE). */ return 0; } - DEBUGF(infof(data_s, "on_stream_close(), %s (err %d), stream %u\n", + H2BUGF(infof(data_s, "on_stream_close(), %s (err %d), stream %u\n", Curl_http2_strerror(error_code), error_code, stream_id)); stream = data_s->req.protop; if(!stream) return NGHTTP2_ERR_CALLBACK_FAILURE; - stream->error_code = error_code; stream->closed = TRUE; data_s->state.drain++; - conn->proto.httpc.drain_total++; + httpc = &conn->proto.httpc; + httpc->drain_total++; + httpc->error_code = error_code; /* remove the entry from the hash as the stream is now gone */ nghttp2_session_set_stream_user_data(session, stream_id, 0); - DEBUGF(infof(data_s, "Removed stream %u hash!\n", stream_id)); + H2BUGF(infof(data_s, "Removed stream %u hash!\n", stream_id)); } return 0; } @@ -815,7 +826,7 @@ static int on_begin_headers(nghttp2_session *session, return 0; } - DEBUGF(infof(data_s, "on_begin_headers() was called\n")); + H2BUGF(infof(data_s, "on_begin_headers() was called\n")); if(frame->hd.type != NGHTTP2_HEADERS) { return 0; @@ -827,7 +838,7 @@ static int on_begin_headers(nghttp2_session *session, } /* This is trailer HEADERS started. Allocate buffer for them. */ - DEBUGF(infof(data_s, "trailer field started\n")); + H2BUGF(infof(data_s, "trailer field started\n")); DEBUGASSERT(stream->trailer_recvbuf == NULL); @@ -928,7 +939,7 @@ static int on_header(nghttp2_session *session, const nghttp2_frame *frame, /* 4 is for ": " and "\r\n". */ uint32_t n = (uint32_t)(namelen + valuelen + 4); - DEBUGF(infof(data_s, "h2 trailer: %.*s: %.*s\n", namelen, name, valuelen, + H2BUGF(infof(data_s, "h2 trailer: %.*s: %.*s\n", namelen, name, valuelen, value)); Curl_add_buffer(stream->trailer_recvbuf, &n, sizeof(n)); @@ -956,7 +967,7 @@ static int on_header(nghttp2_session *session, const nghttp2_frame *frame, if(conn->data != data_s) Curl_expire(data_s, 0, EXPIRE_RUN_NOW); - DEBUGF(infof(data_s, "h2 status: HTTP/2 %03d (easy %p)\n", + H2BUGF(infof(data_s, "h2 status: HTTP/2 %03d (easy %p)\n", stream->status_code, data_s)); return 0; } @@ -972,7 +983,7 @@ static int on_header(nghttp2_session *session, const nghttp2_frame *frame, if(conn->data != data_s) Curl_expire(data_s, 0, EXPIRE_RUN_NOW); - DEBUGF(infof(data_s, "h2 header: %.*s: %.*s\n", namelen, name, valuelen, + H2BUGF(infof(data_s, "h2 header: %.*s: %.*s\n", namelen, name, valuelen, value)); return 0; /* 0 is successful */ @@ -1021,7 +1032,7 @@ static ssize_t data_source_read_callback(nghttp2_session *session, else if(nread == 0) return NGHTTP2_ERR_DEFERRED; - DEBUGF(infof(data_s, "data_source_read_callback: " + H2BUGF(infof(data_s, "data_source_read_callback: " "returns %zu bytes stream %u\n", nread, stream_id)); @@ -1067,7 +1078,7 @@ void Curl_http2_done(struct connectdata *conn, bool premature) struct http_conn *httpc = &conn->proto.httpc; if(http->header_recvbuf) { - DEBUGF(infof(data, "free header_recvbuf!!\n")); + H2BUGF(infof(data, "free header_recvbuf!!\n")); Curl_add_buffer_free(http->header_recvbuf); http->header_recvbuf = NULL; /* clear the pointer */ Curl_add_buffer_free(http->trailer_recvbuf); @@ -1225,13 +1236,14 @@ static int h2_session_send(struct Curl_easy *data, * This function returns 0 if it succeeds, or -1 and error code will * be assigned to *err. */ -static int h2_process_pending_input(struct Curl_easy *data, +static int h2_process_pending_input(struct connectdata *conn, struct http_conn *httpc, CURLcode *err) { ssize_t nread; char *inbuf; ssize_t rv; + struct Curl_easy *data = conn->data; nread = httpc->inbuflen - httpc->nread_inbuf; inbuf = httpc->inbuf + httpc->nread_inbuf; @@ -1246,7 +1258,7 @@ static int h2_process_pending_input(struct Curl_easy *data, } if(nread == rv) { - DEBUGF(infof(data, + H2BUGF(infof(data, "h2_process_pending_input: All data in connection buffer " "processed\n")); httpc->inbuflen = 0; @@ -1254,7 +1266,7 @@ static int h2_process_pending_input(struct Curl_easy *data, } else { httpc->nread_inbuf += rv; - DEBUGF(infof(data, + H2BUGF(infof(data, "h2_process_pending_input: %zu bytes left in connection " "buffer\n", httpc->inbuflen - httpc->nread_inbuf)); @@ -1267,9 +1279,15 @@ static int h2_process_pending_input(struct Curl_easy *data, } if(should_close_session(httpc)) { - DEBUGF(infof(data, + H2BUGF(infof(data, "h2_process_pending_input: nothing to do in this session\n")); - *err = CURLE_HTTP2; + if(httpc->error_code) + *err = CURLE_HTTP2; + else { + /* not an error per se, but should still close the connection */ + connclose(conn, "GOAWAY received"); + *err = CURLE_OK; + } return -1; } @@ -1300,7 +1318,7 @@ CURLcode Curl_http2_done_sending(struct connectdata *conn) that it can signal EOF to nghttp2 */ (void)nghttp2_session_resume_data(h2, stream->stream_id); - (void)h2_process_pending_input(conn->data, httpc, &result); + (void)h2_process_pending_input(conn, httpc, &result); } } return result; @@ -1324,7 +1342,7 @@ static ssize_t http2_handle_stream_close(struct connectdata *conn, data->state.drain = 0; if(httpc->pause_stream_id == 0) { - if(h2_process_pending_input(data, httpc, err) != 0) { + if(h2_process_pending_input(conn, httpc, err) != 0) { return -1; } } @@ -1333,10 +1351,10 @@ static ssize_t http2_handle_stream_close(struct connectdata *conn, /* Reset to FALSE to prevent infinite loop in readwrite_data function. */ stream->closed = FALSE; - if(stream->error_code != NGHTTP2_NO_ERROR) { + if(httpc->error_code != NGHTTP2_NO_ERROR) { failf(data, "HTTP/2 stream %u was not closed cleanly: %s (err %d)", - stream->stream_id, Curl_http2_strerror(stream->error_code), - stream->error_code); + stream->stream_id, Curl_http2_strerror(httpc->error_code), + httpc->error_code); *err = CURLE_HTTP2_STREAM; return -1; } @@ -1370,7 +1388,7 @@ static ssize_t http2_handle_stream_close(struct connectdata *conn, stream->close_handled = TRUE; - DEBUGF(infof(data, "http2_recv returns 0, http2_handle_stream_close\n")); + H2BUGF(infof(data, "http2_recv returns 0, http2_handle_stream_close\n")); return 0; } @@ -1411,7 +1429,7 @@ static int h2_session_send(struct Curl_easy *data, h2_pri_spec(data, &pri_spec); - DEBUGF(infof(data, "Queuing PRIORITY on stream %u (easy %p)\n", + H2BUGF(infof(data, "Queuing PRIORITY on stream %u (easy %p)\n", stream->stream_id, data)); rv = nghttp2_submit_priority(h2, NGHTTP2_FLAG_NONE, stream->stream_id, &pri_spec); @@ -1435,7 +1453,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, (void)sockindex; /* we always do HTTP2 on sockindex 0 */ if(should_close_session(httpc)) { - DEBUGF(infof(data, + H2BUGF(infof(data, "http2_recv: nothing to do in this session\n")); *err = CURLE_HTTP2; return -1; @@ -1461,16 +1479,16 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, ncopy); stream->nread_header_recvbuf += ncopy; - DEBUGF(infof(data, "http2_recv: Got %d bytes from header_recvbuf\n", + H2BUGF(infof(data, "http2_recv: Got %d bytes from header_recvbuf\n", (int)ncopy)); return ncopy; } - DEBUGF(infof(data, "http2_recv: easy %p (stream %u)\n", + H2BUGF(infof(data, "http2_recv: easy %p (stream %u)\n", data, stream->stream_id)); if((data->state.drain) && stream->memlen) { - DEBUGF(infof(data, "http2_recv: DRAIN %zu bytes stream %u!! (%p => %p)\n", + H2BUGF(infof(data, "http2_recv: DRAIN %zu bytes stream %u!! (%p => %p)\n", stream->memlen, stream->stream_id, stream->mem, mem)); if(mem != stream->mem) { @@ -1484,7 +1502,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, /* We have paused nghttp2, but we have no pause data (see on_data_chunk_recv). */ httpc->pause_stream_id = 0; - if(h2_process_pending_input(data, httpc, &result) != 0) { + if(h2_process_pending_input(conn, httpc, &result) != 0) { *err = result; return -1; } @@ -1500,7 +1518,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, infof(data, "%zu data bytes written\n", nread); if(stream->pauselen == 0) { - DEBUGF(infof(data, "Unpaused by stream %u\n", stream->stream_id)); + H2BUGF(infof(data, "Unpaused by stream %u\n", stream->stream_id)); DEBUGASSERT(httpc->pause_stream_id == stream->stream_id); httpc->pause_stream_id = 0; @@ -1514,12 +1532,12 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, frames, then we have to call it again with 0-length data. Without this, on_stream_close callback will not be called, and stream could be hanged. */ - if(h2_process_pending_input(data, httpc, &result) != 0) { + if(h2_process_pending_input(conn, httpc, &result) != 0) { *err = result; return -1; } } - DEBUGF(infof(data, "http2_recv: returns unpaused %zd bytes on stream %u\n", + H2BUGF(infof(data, "http2_recv: returns unpaused %zd bytes on stream %u\n", nread, stream->stream_id)); return nread; } @@ -1532,7 +1550,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, socket is not read. But it seems that usually streams are notified with its drain property, and socket is read again quickly. */ - DEBUGF(infof(data, "stream %x is paused, pause id: %x\n", + H2BUGF(infof(data, "stream %x is paused, pause id: %x\n", stream->stream_id, httpc->pause_stream_id)); *err = CURLE_AGAIN; return -1; @@ -1566,7 +1584,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, return -1; } - DEBUGF(infof(data, "nread=%zd\n", nread)); + H2BUGF(infof(data, "nread=%zd\n", nread)); httpc->inbuflen = nread; inbuf = httpc->inbuf; @@ -1575,7 +1593,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, nread = httpc->inbuflen - httpc->nread_inbuf; inbuf = httpc->inbuf + httpc->nread_inbuf; - DEBUGF(infof(data, "Use data left in connection buffer, nread=%zd\n", + H2BUGF(infof(data, "Use data left in connection buffer, nread=%zd\n", nread)); } rv = nghttp2_session_mem_recv(httpc->h2, (const uint8_t *)inbuf, nread); @@ -1586,15 +1604,15 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, *err = CURLE_RECV_ERROR; return -1; } - DEBUGF(infof(data, "nghttp2_session_mem_recv() returns %zd\n", rv)); + H2BUGF(infof(data, "nghttp2_session_mem_recv() returns %zd\n", rv)); if(nread == rv) { - DEBUGF(infof(data, "All data in connection buffer processed\n")); + H2BUGF(infof(data, "All data in connection buffer processed\n")); httpc->inbuflen = 0; httpc->nread_inbuf = 0; } else { httpc->nread_inbuf += rv; - DEBUGF(infof(data, "%zu bytes left in connection buffer\n", + H2BUGF(infof(data, "%zu bytes left in connection buffer\n", httpc->inbuflen - httpc->nread_inbuf)); } /* Always send pending frames in nghttp2 session, because @@ -1606,21 +1624,21 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, } if(should_close_session(httpc)) { - DEBUGF(infof(data, "http2_recv: nothing to do in this session\n")); + H2BUGF(infof(data, "http2_recv: nothing to do in this session\n")); *err = CURLE_HTTP2; return -1; } } if(stream->memlen) { ssize_t retlen = stream->memlen; - DEBUGF(infof(data, "http2_recv: returns %zd for stream %u\n", + H2BUGF(infof(data, "http2_recv: returns %zd for stream %u\n", retlen, stream->stream_id)); stream->memlen = 0; if(httpc->pause_stream_id == stream->stream_id) { /* data for this stream is returned now, but this stream caused a pause already so we need it called again asap */ - DEBUGF(infof(data, "Data returned for PAUSED stream %u\n", + H2BUGF(infof(data, "Data returned for PAUSED stream %u\n", stream->stream_id)); } else if(!stream->closed) { @@ -1637,7 +1655,7 @@ static ssize_t http2_recv(struct connectdata *conn, int sockindex, return http2_handle_stream_close(conn, data, stream, err); } *err = CURLE_AGAIN; - DEBUGF(infof(data, "http2_recv returns AGAIN for stream %u\n", + H2BUGF(infof(data, "http2_recv returns AGAIN for stream %u\n", stream->stream_id)); return -1; } @@ -1739,7 +1757,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, (void)sockindex; - DEBUGF(infof(conn->data, "http2_send len=%zu\n", len)); + H2BUGF(infof(conn->data, "http2_send len=%zu\n", len)); if(stream->stream_id != -1) { if(stream->close_handled) { @@ -1768,7 +1786,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, stream->upload_len = 0; if(should_close_session(httpc)) { - DEBUGF(infof(conn->data, "http2_send: nothing to do in this session\n")); + H2BUGF(infof(conn->data, "http2_send: nothing to do in this session\n")); *err = CURLE_HTTP2; return -1; } @@ -1781,7 +1799,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, nghttp2_session_resume_data(h2, stream->stream_id); } - DEBUGF(infof(conn->data, "http2_send returns %zu for stream %u\n", len, + H2BUGF(infof(conn->data, "http2_send returns %zu for stream %u\n", len, stream->stream_id)); return len; } @@ -1937,7 +1955,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, for(i = 0; i < nheader; ++i) { acc += nva[i].namelen + nva[i].valuelen; - DEBUGF(infof(conn->data, "h2 header: %.*s:%.*s\n", + H2BUGF(infof(conn->data, "h2 header: %.*s:%.*s\n", nva[i].namelen, nva[i].name, nva[i].valuelen, nva[i].value)); } @@ -1975,7 +1993,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, Curl_safefree(nva); if(stream_id < 0) { - DEBUGF(infof(conn->data, "http2_send() send error\n")); + H2BUGF(infof(conn->data, "http2_send() send error\n")); *err = CURLE_SEND_ERROR; return -1; } @@ -1994,7 +2012,7 @@ static ssize_t http2_send(struct connectdata *conn, int sockindex, } if(should_close_session(httpc)) { - DEBUGF(infof(conn->data, "http2_send: nothing to do in this session\n")); + H2BUGF(infof(conn->data, "http2_send: nothing to do in this session\n")); *err = CURLE_HTTP2; return -1; } @@ -2152,7 +2170,7 @@ CURLcode Curl_http2_switched(struct connectdata *conn, return CURLE_HTTP2; } - DEBUGF(infof(data, "nghttp2_session_mem_recv() returns %zd\n", nproc)); + H2BUGF(infof(data, "nghttp2_session_mem_recv() returns %zd\n", nproc)); if((ssize_t)nread == nproc) { httpc->inbuflen = 0; @@ -2172,7 +2190,7 @@ CURLcode Curl_http2_switched(struct connectdata *conn, } if(should_close_session(httpc)) { - DEBUGF(infof(data, + H2BUGF(infof(data, "nghttp2_session_send(): nothing to do in this session\n")); return CURLE_HTTP2; } diff --git a/lib/http_chunks.c b/lib/http_chunks.c index 16164296..8368eeca 100644 --- a/lib/http_chunks.c +++ b/lib/http_chunks.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -74,15 +74,6 @@ */ -/* Check for an ASCII hex digit. - We avoid the use of isxdigit to accommodate non-ASCII hosts. */ -static bool Curl_isxdigit(char digit) -{ - return ( (digit >= 0x30 && digit <= 0x39) /* 0-9 */ - || (digit >= 0x41 && digit <= 0x46) /* A-F */ - || (digit >= 0x61 && digit <= 0x66) /* a-f */) ? TRUE : FALSE; -} - void Curl_httpchunk_init(struct connectdata *conn) { struct Curl_chunker *chunk = &conn->chunk; @@ -187,15 +178,15 @@ CHUNKcode Curl_httpchunk_read(struct connectdata *conn, piece = curlx_sotouz((ch->datasize >= length)?length:ch->datasize); /* Write the data portion available */ - if(conn->data->set.http_ce_skip || !k->writer_stack) { - if(!k->ignorebody) + if(!conn->data->set.http_te_skip && !k->ignorebody) { + if(!conn->data->set.http_ce_skip && k->writer_stack) + result = Curl_unencode_write(conn, k->writer_stack, datap, piece); + else result = Curl_client_write(conn, CLIENTWRITE_BODY, datap, piece); - } - else - result = Curl_unencode_write(conn, k->writer_stack, datap, piece); - if(result) - return CHUNKE_WRITE_ERROR; + if(result) + return CHUNKE_WRITE_ERROR; + } *wrote += piece; ch->datasize -= piece; /* decrease amount left to expect */ diff --git a/lib/http_proxy.c b/lib/http_proxy.c index 7f504054..c1eb177d 100644 --- a/lib/http_proxy.c +++ b/lib/http_proxy.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -252,7 +252,7 @@ static CURLcode CONNECT(struct connectdata *conn, return CURLE_OUT_OF_MEMORY; } - if(!Curl_checkProxyheaders(conn, "Host:")) { + if(!Curl_checkProxyheaders(conn, "Host")) { host = aprintf("Host: %s\r\n", hostheader); if(!host) { free(hostheader); @@ -260,10 +260,10 @@ static CURLcode CONNECT(struct connectdata *conn, return CURLE_OUT_OF_MEMORY; } } - if(!Curl_checkProxyheaders(conn, "Proxy-Connection:")) + if(!Curl_checkProxyheaders(conn, "Proxy-Connection")) proxyconn = "Proxy-Connection: Keep-Alive\r\n"; - if(!Curl_checkProxyheaders(conn, "User-Agent:") && + if(!Curl_checkProxyheaders(conn, "User-Agent") && data->set.str[STRING_USERAGENT]) useragent = conn->allocptr.uagent; @@ -85,7 +85,7 @@ krb5_decode(void *app_data, void *buf, int len, enc.value = buf; enc.length = len; - maj = gss_unseal(&min, *context, &enc, &dec, NULL, NULL); + maj = gss_unwrap(&min, *context, &enc, &dec, NULL, NULL); if(maj != GSS_S_COMPLETE) { if(len >= 4) strcpy(buf, "599 "); @@ -119,11 +119,11 @@ krb5_encode(void *app_data, const void *from, int length, int level, void **to) int len; /* NOTE that the cast is safe, neither of the krb5, gnu gss and heimdal - * libraries modify the input buffer in gss_seal() + * libraries modify the input buffer in gss_wrap() */ dec.value = (void *)from; dec.length = length; - maj = gss_seal(&min, *context, + maj = gss_wrap(&min, *context, level == PROT_PRIVATE, GSS_C_QOP_DEFAULT, &dec, &state, &enc); diff --git a/lib/libcurl.plist b/lib/libcurl.plist index 5b02d28b..da7037d5 100644 --- a/lib/libcurl.plist +++ b/lib/libcurl.plist @@ -15,7 +15,7 @@ <string>se.haxx.curl.libcurl</string> <key>CFBundleVersion</key> - <string>7.58.0</string> + <string>7.59.0</string> <key>CFBundleName</key> <string>libcurl</string> @@ -27,9 +27,9 @@ <string>????</string> <key>CFBundleShortVersionString</key> - <string>libcurl 7.58.0</string> + <string>libcurl 7.59.0</string> <key>CFBundleGetInfoString</key> - <string>libcurl.plist 7.58.0</string> + <string>libcurl.plist 7.59.0</string> </dict> </plist> @@ -51,10 +51,6 @@ #endif -#define FILE_CONTENTTYPE_DEFAULT "application/octet-stream" -#define MULTIPART_CONTENTTYPE_DEFAULT "multipart/mixed" -#define DISPOSITION_DEFAULT "attachment" - #define READ_ERROR ((size_t) -1) /* Encoders. */ @@ -1642,8 +1638,7 @@ static CURLcode add_content_type(struct curl_slist **slp, boundary? boundary: ""); } - -static const char *ContentTypeForFilename(const char *filename) +const char *Curl_mime_contenttype(const char *filename) { unsigned int i; @@ -1715,14 +1710,14 @@ CURLcode Curl_mime_prepare_headers(curl_mimepart *part, contenttype = MULTIPART_CONTENTTYPE_DEFAULT; break; case MIMEKIND_FILE: - contenttype = ContentTypeForFilename(part->filename); + contenttype = Curl_mime_contenttype(part->filename); if(!contenttype) - contenttype = ContentTypeForFilename(part->data); + contenttype = Curl_mime_contenttype(part->data); if(!contenttype && part->filename) contenttype = FILE_CONTENTTYPE_DEFAULT; break; default: - contenttype = ContentTypeForFilename(part->filename); + contenttype = Curl_mime_contenttype(part->filename); break; } } @@ -30,6 +30,10 @@ #define MIME_USERHEADERS_OWNER (1 << 0) #define MIME_BODY_ONLY (1 << 1) +#define FILE_CONTENTTYPE_DEFAULT "application/octet-stream" +#define MULTIPART_CONTENTTYPE_DEFAULT "multipart/mixed" +#define DISPOSITION_DEFAULT "attachment" + /* Part source kinds. */ enum mimekind { MIMEKIND_NONE = 0, /* Part not set. */ @@ -134,5 +138,6 @@ size_t Curl_mime_read(char *buffer, size_t size, size_t nitems, void *instream); CURLcode Curl_mime_rewind(curl_mimepart *part); CURLcode Curl_mime_add_header(struct curl_slist **slp, const char *fmt, ...); +const char *Curl_mime_contenttype(const char *filename); #endif /* HEADER_CURL_MIME_H */ diff --git a/lib/mk-ca-bundle.vbs b/lib/mk-ca-bundle.vbs index a9b983e9..8da27926 100755 --- a/lib/mk-ca-bundle.vbs +++ b/lib/mk-ca-bundle.vbs @@ -314,7 +314,7 @@ Function RegExprFirst(SearchPattern, TheString) Set objRegExp = New RegExp ' create a regular expression.
objRegExp.Pattern = SearchPattern ' sets the search pattern.
objRegExp.IgnoreCase = TRUE ' set to ignores case.
- objRegExp.Global = TRUE ' set to gloabal search.
+ objRegExp.Global = TRUE ' set to global search.
Set Matches = objRegExp.Execute(TheString) ' do the search.
If (Matches.Count) Then
RegExprFirst = Matches(0).SubMatches(0) ' return first match.
diff --git a/lib/multi.c b/lib/multi.c index 43823cc9..98e5fca2 100644 --- a/lib/multi.c +++ b/lib/multi.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -366,6 +366,9 @@ CURLMcode curl_multi_add_handle(struct Curl_multi *multi, if(data->multi) return CURLM_ADDED_ALREADY; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + /* Initialize timeout list for this handle */ Curl_llist_init(&data->state.timeoutlist, NULL); @@ -535,11 +538,8 @@ static CURLcode multi_done(struct connectdata **connp, result = CURLE_ABORTED_BY_CALLBACK; } - if(conn->send_pipe.size + conn->recv_pipe.size != 0 && - !data->set.reuse_forbid && - !conn->bits.close) { - /* Stop if pipeline is not empty and we do not have to close - connection. */ + if(conn->send_pipe.size || conn->recv_pipe.size) { + /* Stop if pipeline is not empty . */ data->easy_conn = NULL; DEBUGF(infof(data, "Connection still in use, no more multi_done now!\n")); return CURLE_OK; @@ -640,6 +640,9 @@ CURLMcode curl_multi_remove_handle(struct Curl_multi *multi, if(!data->multi) return CURLM_OK; /* it is already removed so let's say it is fine! */ + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + premature = (data->mstate < CURLM_STATE_COMPLETED) ? TRUE : FALSE; easy_owns_conn = (data->easy_conn && (data->easy_conn->data == easy)) ? TRUE : FALSE; @@ -903,6 +906,9 @@ CURLMcode curl_multi_fdset(struct Curl_multi *multi, if(!GOOD_MULTI_HANDLE(multi)) return CURLM_BAD_HANDLE; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + data = multi->easyp; while(data) { bitmap = multi_getsock(data, sockbunch, MAX_SOCKSPEREASYHANDLE); @@ -956,6 +962,9 @@ CURLMcode curl_multi_wait(struct Curl_multi *multi, if(!GOOD_MULTI_HANDLE(multi)) return CURLM_BAD_HANDLE; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + /* If the internally desired timeout is actually shorter than requested from the outside, then use the shorter time! But only if the internal timer is actually larger than -1! */ @@ -1121,6 +1130,9 @@ CURLMcode Curl_multi_add_perform(struct Curl_multi *multi, { CURLMcode rc; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + rc = curl_multi_add_handle(multi, data); if(!rc) { struct SingleRequest *k = &data->req; @@ -2127,6 +2139,9 @@ CURLMcode curl_multi_perform(struct Curl_multi *multi, int *running_handles) if(!GOOD_MULTI_HANDLE(multi)) return CURLM_BAD_HANDLE; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + data = multi->easyp; while(data) { CURLMcode result; @@ -2174,6 +2189,9 @@ CURLMcode curl_multi_cleanup(struct Curl_multi *multi) struct Curl_easy *nextdata; if(GOOD_MULTI_HANDLE(multi)) { + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + multi->type = 0; /* not good anymore */ /* Firsrt remove all remaining easy handles */ @@ -2234,7 +2252,9 @@ CURLMsg *curl_multi_info_read(struct Curl_multi *multi, int *msgs_in_queue) *msgs_in_queue = 0; /* default to none */ - if(GOOD_MULTI_HANDLE(multi) && Curl_llist_count(&multi->msglist)) { + if(GOOD_MULTI_HANDLE(multi) && + !multi->in_callback && + Curl_llist_count(&multi->msglist)) { /* there is one or more messages in the list */ struct curl_llist_element *e; @@ -2624,6 +2644,9 @@ CURLMcode curl_multi_setopt(struct Curl_multi *multi, if(!GOOD_MULTI_HANDLE(multi)) return CURLM_BAD_HANDLE; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + va_start(param, option); switch(option) { @@ -2688,7 +2711,10 @@ CURLMcode curl_multi_setopt(struct Curl_multi *multi, CURLMcode curl_multi_socket(struct Curl_multi *multi, curl_socket_t s, int *running_handles) { - CURLMcode result = multi_socket(multi, FALSE, s, 0, running_handles); + CURLMcode result; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + result = multi_socket(multi, FALSE, s, 0, running_handles); if(CURLM_OK >= result) update_timer(multi); return result; @@ -2697,8 +2723,10 @@ CURLMcode curl_multi_socket(struct Curl_multi *multi, curl_socket_t s, CURLMcode curl_multi_socket_action(struct Curl_multi *multi, curl_socket_t s, int ev_bitmask, int *running_handles) { - CURLMcode result = multi_socket(multi, FALSE, s, - ev_bitmask, running_handles); + CURLMcode result; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + result = multi_socket(multi, FALSE, s, ev_bitmask, running_handles); if(CURLM_OK >= result) update_timer(multi); return result; @@ -2707,8 +2735,10 @@ CURLMcode curl_multi_socket_action(struct Curl_multi *multi, curl_socket_t s, CURLMcode curl_multi_socket_all(struct Curl_multi *multi, int *running_handles) { - CURLMcode result = multi_socket(multi, TRUE, CURL_SOCKET_BAD, 0, - running_handles); + CURLMcode result; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + result = multi_socket(multi, TRUE, CURL_SOCKET_BAD, 0, running_handles); if(CURLM_OK >= result) update_timer(multi); return result; @@ -2760,6 +2790,9 @@ CURLMcode curl_multi_timeout(struct Curl_multi *multi, if(!GOOD_MULTI_HANDLE(multi)) return CURLM_BAD_HANDLE; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + return multi_timeout(multi, timeout_ms); } @@ -2992,6 +3025,9 @@ CURLMcode curl_multi_assign(struct Curl_multi *multi, curl_socket_t s, { struct Curl_sh_entry *there = NULL; + if(multi->in_callback) + return CURLM_RECURSIVE_API_CALL; + there = sh_getentry(&multi->sockhash, s); if(!there) @@ -3054,6 +3090,20 @@ void Curl_multi_process_pending_handles(struct Curl_multi *multi) } } +void Curl_set_in_callback(struct Curl_easy *easy, bool value) +{ + if(easy->multi_easy) + easy->multi_easy->in_callback = value; + else if(easy->multi) + easy->multi->in_callback = value; +} + +bool Curl_is_in_callback(struct Curl_easy *easy) +{ + return ((easy->multi && easy->multi->in_callback) || + (easy->multi_easy && easy->multi_easy->in_callback)); +} + #ifdef DEBUGBUILD void Curl_multi_dump(struct Curl_multi *multi) { diff --git a/lib/multihandle.h b/lib/multihandle.h index de9a7cf5..1a5017f4 100644 --- a/lib/multihandle.h +++ b/lib/multihandle.h @@ -146,6 +146,7 @@ struct Curl_multi { void *timer_userp; struct curltime timer_lastcall; /* the fixed time for the timeout for the previous callback */ + bool in_callback; /* true while executing a callback */ }; #endif /* HEADER_CURL_MULTIHANDLE_H */ diff --git a/lib/multiif.h b/lib/multiif.h index a877571a..a988bfd4 100644 --- a/lib/multiif.h +++ b/lib/multiif.h @@ -31,6 +31,8 @@ void Curl_expire_clear(struct Curl_easy *data); void Curl_expire_done(struct Curl_easy *data, expire_id id); bool Curl_pipeline_wanted(const struct Curl_multi* multi, int bits); void Curl_multi_handlePipeBreak(struct Curl_easy *data); +void Curl_set_in_callback(struct Curl_easy *data, bool value); +bool Curl_is_in_callback(struct Curl_easy *easy); /* Internal version of curl_multi_init() accepts size parameters for the socket and connection hashes */ diff --git a/lib/non-ascii.c b/lib/non-ascii.c index 92b2f8d7..14143248 100644 --- a/lib/non-ascii.c +++ b/lib/non-ascii.c @@ -30,6 +30,7 @@ #include "formdata.h" #include "sendf.h" #include "urldata.h" +#include "multiif.h" #include "curl_memory.h" /* The last #include file should be: */ @@ -84,7 +85,10 @@ CURLcode Curl_convert_to_network(struct Curl_easy *data, { if(data && data->set.convtonetwork) { /* use translation callback */ - CURLcode result = data->set.convtonetwork(buffer, length); + CURLcode result; + Curl_set_in_callback(data, true); + result = data->set.convtonetwork(buffer, length); + Curl_set_in_callback(data, false); if(result) { failf(data, "CURLOPT_CONV_TO_NETWORK_FUNCTION callback returned %d: %s", @@ -147,7 +151,10 @@ CURLcode Curl_convert_from_network(struct Curl_easy *data, { if(data && data->set.convfromnetwork) { /* use translation callback */ - CURLcode result = data->set.convfromnetwork(buffer, length); + CURLcode result; + Curl_set_in_callback(data, true); + result = data->set.convfromnetwork(buffer, length); + Curl_set_in_callback(data, false); if(result) { failf(data, "CURLOPT_CONV_FROM_NETWORK_FUNCTION callback returned %d: %s", @@ -210,7 +217,10 @@ CURLcode Curl_convert_from_utf8(struct Curl_easy *data, { if(data && data->set.convfromutf8) { /* use translation callback */ - CURLcode result = data->set.convfromutf8(buffer, length); + CURLcode result; + Curl_set_in_callback(data, true); + result = data->set.convfromutf8(buffer, length); + Curl_set_in_callback(data, false); if(result) { failf(data, "CURLOPT_CONV_FROM_UTF8_FUNCTION callback returned %d: %s", diff --git a/lib/objnames.inc b/lib/objnames.inc index 6a5b2a83..e362f6e8 100644 --- a/lib/objnames.inc +++ b/lib/objnames.inc @@ -86,7 +86,7 @@ curl_10char_object_name() { # curl_8char_object_name # # Same as curl_10char_object_name() description and details above, except -# that object name is limited to 8 charcters maximum. +# that object name is limited to 8 characters maximum. # curl_8char_object_name() { diff --git a/lib/openldap.c b/lib/openldap.c index f2ffdfe6..187c8999 100644 --- a/lib/openldap.c +++ b/lib/openldap.c @@ -5,8 +5,8 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 2010, 2017, Howard Chu, <hyc@openldap.org> - * Copyright (C) 2011 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 2010, Howard Chu, <hyc@openldap.org> + * Copyright (C) 2011 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -221,7 +221,7 @@ static CURLcode ldap_connect(struct connectdata *conn, bool *done) if(conn->handler->flags & PROTOPT_SSL) *ptr++ = 's'; snprintf(ptr, sizeof(hosturl)-(ptr-hosturl), "://%s:%d", - conn->host.name, conn->remote_port); + conn->host.name, conn->remote_port); #ifdef CURL_OPENLDAP_DEBUG static int do_trace = 0; @@ -286,7 +286,7 @@ static CURLcode ldap_connecting(struct connectdata *conn, bool *done) tvp = &tv; -retry: + retry: if(!li->didbind) { char *binddn; struct berval passwd; @@ -472,8 +472,8 @@ static ssize_t ldap_recv(struct connectdata *conn, int sockindex, char *buf, return ret; for(ent = ldap_first_message(li->ld, msg); ent; - ent = ldap_next_message(li->ld, ent)) { - struct berval bv, *bvals, **bvp = &bvals; + ent = ldap_next_message(li->ld, ent)) { + struct berval bv, *bvals; int binary = 0, msgtype; CURLcode writeerr; @@ -535,12 +535,13 @@ static ssize_t ldap_recv(struct connectdata *conn, int sockindex, char *buf, } data->req.bytecount += bv.bv_len + 5; - for(rc = ldap_get_attribute_ber(li->ld, ent, ber, &bv, bvp); - rc == LDAP_SUCCESS; - rc = ldap_get_attribute_ber(li->ld, ent, ber, &bv, bvp)) { + for(rc = ldap_get_attribute_ber(li->ld, ent, ber, &bv, &bvals); + (rc == LDAP_SUCCESS) && bvals; + rc = ldap_get_attribute_ber(li->ld, ent, ber, &bv, &bvals)) { int i; - if(bv.bv_val == NULL) break; + if(bv.bv_val == NULL) + break; if(bv.bv_len > 7 && !strncmp(bv.bv_val + bv.bv_len - 7, ";binary", 7)) binary = 1; @@ -555,24 +556,24 @@ static ssize_t ldap_recv(struct connectdata *conn, int sockindex, char *buf, return -1; } - writeerr = Curl_client_write(conn, CLIENTWRITE_BODY, (char *)bv.bv_val, - bv.bv_len); - if(writeerr) { - *err = writeerr; - return -1; - } + writeerr = Curl_client_write(conn, CLIENTWRITE_BODY, (char *)bv.bv_val, + bv.bv_len); + if(writeerr) { + *err = writeerr; + return -1; + } writeerr = Curl_client_write(conn, CLIENTWRITE_BODY, (char *)":", 1); - if(writeerr) { - *err = writeerr; - return -1; - } + if(writeerr) { + *err = writeerr; + return -1; + } data->req.bytecount += bv.bv_len + 2; if(!binary) { /* check for leading or trailing whitespace */ if(ISSPACE(bvals[i].bv_val[0]) || - ISSPACE(bvals[i].bv_val[bvals[i].bv_len-1])) + ISSPACE(bvals[i].bv_val[bvals[i].bv_len-1])) binval = 1; else { /* check for unprintable characters */ @@ -610,7 +611,7 @@ static ssize_t ldap_recv(struct connectdata *conn, int sockindex, char *buf, data->req.bytecount += 2; if(val_b64_sz > 0) { writeerr = Curl_client_write(conn, CLIENTWRITE_BODY, val_b64, - val_b64_sz); + val_b64_sz); if(writeerr) { *err = writeerr; return -1; diff --git a/lib/parsedate.c b/lib/parsedate.c index 0fabbd26..aa27303f 100644 --- a/lib/parsedate.c +++ b/lib/parsedate.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -276,26 +276,23 @@ struct my_tm { int tm_hour; int tm_mday; int tm_mon; - int tm_year; + int tm_year; /* full year */ }; /* struct tm to time since epoch in GMT time zone. * This is similar to the standard mktime function but for GMT only, and * doesn't suffer from the various bugs and portability problems that * some systems' implementations have. + * + * Returns 0 on success, otherwise non-zero. */ -static time_t my_timegm(struct my_tm *tm) +static void my_timegm(struct my_tm *tm, time_t *t) { static const int month_days_cumulative [12] = { 0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334 }; int month, year, leap_days; - if(tm->tm_year < 70) - /* we don't support years before 1970 as they will cause this function - to return a negative value */ - return -1; - - year = tm->tm_year + 1900; + year = tm->tm_year; month = tm->tm_mon; if(month < 0) { year += (11 - month) / 12; @@ -310,9 +307,9 @@ static time_t my_timegm(struct my_tm *tm) leap_days = ((leap_days / 4) - (leap_days / 100) + (leap_days / 400) - (1969 / 4) + (1969 / 100) - (1969 / 400)); - return ((((time_t) (year - 1970) * 365 - + leap_days + month_days_cumulative [month] + tm->tm_mday - 1) * 24 - + tm->tm_hour) * 60 + tm->tm_min) * 60 + tm->tm_sec; + *t = ((((time_t) (year - 1970) * 365 + + leap_days + month_days_cumulative[month] + tm->tm_mday - 1) * 24 + + tm->tm_hour) * 60 + tm->tm_min) * 60 + tm->tm_sec; } /* @@ -436,7 +433,7 @@ static int parsedate(const char *date, time_t *output) tzoff = (val/100 * 60 + val%100)*60; /* the + and - prefix indicates the local time compared to GMT, - this we need ther reversed math to get what we want */ + this we need their reversed math to get what we want */ tzoff = date[-1]=='+'?-tzoff:tzoff; } @@ -462,7 +459,7 @@ static int parsedate(const char *date, time_t *output) if(!found && (dignext == DATE_YEAR) && (yearnum == -1)) { yearnum = val; found = TRUE; - if(yearnum < 1900) { + if(yearnum < 100) { if(yearnum > 70) yearnum += 1900; else @@ -491,18 +488,39 @@ static int parsedate(const char *date, time_t *output) /* lacks vital info, fail */ return PARSEDATE_FAIL; -#if SIZEOF_TIME_T < 5 - /* 32 bit time_t can only hold dates to the beginning of 2038 */ - if(yearnum > 2037) { - *output = 0x7fffffff; - return PARSEDATE_LATER; +#ifdef HAVE_TIME_T_UNSIGNED + if(yearnum < 1970) { + /* only positive numbers cannot return earlier */ + *output = TIME_T_MIN; + return PARSEDATE_SOONER; } #endif - if(yearnum < 1970) { - *output = 0; +#if (SIZEOF_TIME_T < 5) + +#ifdef HAVE_TIME_T_UNSIGNED + /* an unsigned 32 bit time_t can only hold dates to 2106 */ + if(yearnum > 2105) { + *output = TIME_T_MAX; + return PARSEDATE_LATER; + } +#else + /* a signed 32 bit time_t can only hold dates to the beginning of 2038 */ + if(yearnum > 2037) { + *output = TIME_T_MAX; + return PARSEDATE_LATER; + } + if(yearnum < 1903) { + *output = TIME_T_MIN; return PARSEDATE_SOONER; } +#endif + +#else + /* The Gregorian calendar was introduced 1582 */ + if(yearnum < 1583) + return PARSEDATE_FAIL; +#endif if((mdaynum > 31) || (monnum > 11) || (hournum > 23) || (minnum > 59) || (secnum > 60)) @@ -513,31 +531,25 @@ static int parsedate(const char *date, time_t *output) tm.tm_hour = hournum; tm.tm_mday = mdaynum; tm.tm_mon = monnum; - tm.tm_year = yearnum - 1900; - - /* my_timegm() returns a time_t. time_t is often 32 bits, even on many - architectures that feature 64 bit 'long'. + tm.tm_year = yearnum; - Some systems have 64 bit time_t and deal with years beyond 2038. However, - even on some of the systems with 64 bit time_t mktime() returns -1 for - dates beyond 03:14:07 UTC, January 19, 2038. (Such as AIX 5100-06) + /* my_timegm() returns a time_t. time_t is often 32 bits, sometimes even on + architectures that feature 64 bit 'long' but ultimately time_t is the + correct data type to use. */ - t = my_timegm(&tm); - - /* time zone adjust (cast t to int to compare to negative one) */ - if(-1 != (int)t) { + my_timegm(&tm, &t); - /* Add the time zone diff between local time zone and GMT. */ - long delta = (long)(tzoff!=-1?tzoff:0); + /* Add the time zone diff between local time zone and GMT. */ + if(tzoff == -1) + tzoff = 0; - if((delta>0) && (t > LONG_MAX - delta)) { - *output = 0x7fffffff; - return PARSEDATE_LATER; /* time_t overflow */ - } - - t += delta; + if((tzoff > 0) && (t > TIME_T_MAX - tzoff)) { + *output = TIME_T_MAX; + return PARSEDATE_LATER; /* time_t overflow */ } + t += tzoff; + *output = t; return PARSEDATE_OK; @@ -549,10 +561,10 @@ time_t curl_getdate(const char *p, const time_t *now) int rc = parsedate(p, &parsed); (void)now; /* legacy argument from the past that we ignore */ - switch(rc) { - case PARSEDATE_OK: - case PARSEDATE_LATER: - case PARSEDATE_SOONER: + if(rc == PARSEDATE_OK) { + if(parsed == -1) + /* avoid returning -1 for a working scenario */ + parsed++; return parsed; } /* everything else is fail */ diff --git a/lib/progress.c b/lib/progress.c index cc5e8be7..ce8be7ff 100644 --- a/lib/progress.c +++ b/lib/progress.c @@ -24,6 +24,7 @@ #include "urldata.h" #include "sendf.h" +#include "multiif.h" #include "progress.h" #include "curl_printf.h" @@ -237,8 +238,8 @@ void Curl_pgrsStartNow(struct Curl_easy *data) } /* - * This is used to handle speed limits, calculating how much milliseconds we - * need to wait until we're back under the speed limit, if needed. + * This is used to handle speed limits, calculating how many milliseconds to + * wait until we're back under the speed limit, if needed. * * The way it works is by having a "starting point" (time & amount of data * transferred by then) used in the speed computation, to be used instead of @@ -250,16 +251,15 @@ void Curl_pgrsStartNow(struct Curl_easy *data) * the starting point, the limit (in bytes/s), the time of the starting point * and the current time. * - * Returns -1 if no waiting is needed (not enough data transferred since - * starting point yet), 0 when no waiting is needed but the starting point - * should be reset (to current), or the number of milliseconds to wait to get - * back under the speed limit. + * Returns 0 if no waiting is needed or when no waiting is needed but the + * starting point should be reset (to current); or the number of milliseconds + * to wait to get back under the speed limit. */ -long Curl_pgrsLimitWaitTime(curl_off_t cursize, - curl_off_t startsize, - curl_off_t limit, - struct curltime start, - struct curltime now) +timediff_t Curl_pgrsLimitWaitTime(curl_off_t cursize, + curl_off_t startsize, + curl_off_t limit, + struct curltime start, + struct curltime now) { curl_off_t size = cursize - startsize; time_t minimum; @@ -269,16 +269,23 @@ long Curl_pgrsLimitWaitTime(curl_off_t cursize, if(start.tv_sec == 0 && start.tv_usec == 0) return 0; - /* not enough data yet */ - if(size < limit) - return -1; + if(!limit) + return 0; + + if(size < CURL_OFF_T_MAX/1000) + minimum = (time_t) (CURL_OFF_T_C(1000) * size / limit); + else { + minimum = (time_t) (size / limit); + if(minimum < TIME_T_MAX/1000) + minimum *= 1000; + else + minimum = TIME_T_MAX; + } - minimum = (time_t) (CURL_OFF_T_C(1000) * size / limit); actual = Curl_timediff(now, start); if(actual < minimum) - /* this is a conversion on some systems (64bit time_t => 32bit long) */ - return (long)(minimum - actual); + return (minimum - actual); return 0; } @@ -461,22 +468,26 @@ int Curl_pgrsUpdate(struct connectdata *conn) if(data->set.fxferinfo) { /* There's a callback set, call that */ + Curl_set_in_callback(data, true); result = data->set.fxferinfo(data->set.progress_client, data->progress.size_dl, data->progress.downloaded, data->progress.size_ul, data->progress.uploaded); + Curl_set_in_callback(data, false); if(result) failf(data, "Callback aborted"); return result; } if(data->set.fprogress) { /* The older deprecated callback is set, call that */ + Curl_set_in_callback(data, true); result = data->set.fprogress(data->set.progress_client, (double)data->progress.size_dl, (double)data->progress.downloaded, (double)data->progress.size_ul, (double)data->progress.uploaded); + Curl_set_in_callback(data, false); if(result) failf(data, "Callback aborted"); return result; diff --git a/lib/progress.h b/lib/progress.h index 9333ab25..3c2231cb 100644 --- a/lib/progress.h +++ b/lib/progress.h @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -49,11 +49,11 @@ void Curl_pgrsSetUploadCounter(struct Curl_easy *data, curl_off_t size); int Curl_pgrsUpdate(struct connectdata *); void Curl_pgrsResetTransferSizes(struct Curl_easy *data); void Curl_pgrsTime(struct Curl_easy *data, timerid timer); -long Curl_pgrsLimitWaitTime(curl_off_t cursize, - curl_off_t startsize, - curl_off_t limit, - struct curltime start, - struct curltime now); +timediff_t Curl_pgrsLimitWaitTime(curl_off_t cursize, + curl_off_t startsize, + curl_off_t limit, + struct curltime start, + struct curltime now); /* Don't show progress for sizes smaller than: */ #define LEAST_SIZE_PROGRESS BUFSIZE @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -47,7 +47,7 @@ * -incoming server requests * -server CSeq counter * -digest authentication - * -connect thru proxy + * -connect through proxy * -pipelining? */ @@ -357,7 +357,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) } /* Transport Header for SETUP requests */ - p_transport = Curl_checkheaders(conn, "Transport:"); + p_transport = Curl_checkheaders(conn, "Transport"); if(rtspreq == RTSPREQ_SETUP && !p_transport) { /* New Transport: setting? */ if(data->set.str[STRING_RTSP_TRANSPORT]) { @@ -381,11 +381,11 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) /* Accept Headers for DESCRIBE requests */ if(rtspreq == RTSPREQ_DESCRIBE) { /* Accept Header */ - p_accept = Curl_checkheaders(conn, "Accept:")? + p_accept = Curl_checkheaders(conn, "Accept")? NULL:"Accept: application/sdp\r\n"; /* Accept-Encoding header */ - if(!Curl_checkheaders(conn, "Accept-Encoding:") && + if(!Curl_checkheaders(conn, "Accept-Encoding") && data->set.str[STRING_ENCODING]) { Curl_safefree(conn->allocptr.accept_encoding); conn->allocptr.accept_encoding = @@ -402,11 +402,11 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) it might have been used in the proxy connect, but if we have got a header with the user-agent string specified, we erase the previously made string here. */ - if(Curl_checkheaders(conn, "User-Agent:") && conn->allocptr.uagent) { + if(Curl_checkheaders(conn, "User-Agent") && conn->allocptr.uagent) { Curl_safefree(conn->allocptr.uagent); conn->allocptr.uagent = NULL; } - else if(!Curl_checkheaders(conn, "User-Agent:") && + else if(!Curl_checkheaders(conn, "User-Agent") && data->set.str[STRING_USERAGENT]) { p_uagent = conn->allocptr.uagent; } @@ -421,7 +421,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) /* Referrer */ Curl_safefree(conn->allocptr.ref); - if(data->change.referer && !Curl_checkheaders(conn, "Referer:")) + if(data->change.referer && !Curl_checkheaders(conn, "Referer")) conn->allocptr.ref = aprintf("Referer: %s\r\n", data->change.referer); else conn->allocptr.ref = NULL; @@ -438,7 +438,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) (rtspreq & (RTSPREQ_PLAY | RTSPREQ_PAUSE | RTSPREQ_RECORD))) { /* Check to see if there is a range set in the custom headers */ - if(!Curl_checkheaders(conn, "Range:") && data->state.range) { + if(!Curl_checkheaders(conn, "Range") && data->state.range) { Curl_safefree(conn->allocptr.rangeline); conn->allocptr.rangeline = aprintf("Range: %s\r\n", data->state.range); p_range = conn->allocptr.rangeline; @@ -448,11 +448,11 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) /* * Sanity check the custom headers */ - if(Curl_checkheaders(conn, "CSeq:")) { + if(Curl_checkheaders(conn, "CSeq")) { failf(data, "CSeq cannot be set as a custom header."); return CURLE_RTSP_CSEQ_ERROR; } - if(Curl_checkheaders(conn, "Session:")) { + if(Curl_checkheaders(conn, "Session")) { failf(data, "Session ID cannot be set as a custom header."); return CURLE_BAD_FUNCTION_ARGUMENT; } @@ -542,7 +542,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) if(putsize > 0 || postsize > 0) { /* As stated in the http comments, it is probably not wise to * actually set a custom Content-Length in the headers */ - if(!Curl_checkheaders(conn, "Content-Length:")) { + if(!Curl_checkheaders(conn, "Content-Length")) { result = Curl_add_bufferf(req_buffer, "Content-Length: %" CURL_FORMAT_CURL_OFF_T"\r\n", (data->set.upload ? putsize : postsize)); @@ -552,7 +552,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) if(rtspreq == RTSPREQ_SET_PARAMETER || rtspreq == RTSPREQ_GET_PARAMETER) { - if(!Curl_checkheaders(conn, "Content-Type:")) { + if(!Curl_checkheaders(conn, "Content-Type")) { result = Curl_add_bufferf(req_buffer, "Content-Type: text/parameters\r\n"); if(result) @@ -561,7 +561,7 @@ static CURLcode rtsp_do(struct connectdata *conn, bool *done) } if(rtspreq == RTSPREQ_ANNOUNCE) { - if(!Curl_checkheaders(conn, "Content-Type:")) { + if(!Curl_checkheaders(conn, "Content-Type")) { result = Curl_add_bufferf(req_buffer, "Content-Type: application/sdp\r\n"); if(result) @@ -770,7 +770,9 @@ CURLcode rtp_client_write(struct connectdata *conn, char *ptr, size_t len) user_ptr = data->set.out; } + Curl_set_in_callback(data, true); wrote = writeit(ptr, 1, len, user_ptr); + Curl_set_in_callback(data, false); if(CURL_WRITEFUNC_PAUSE == wrote) { failf(data, "Cannot pause RTP"); diff --git a/lib/sendf.c b/lib/sendf.c index 027f97c4..27c0ccc7 100644 --- a/lib/sendf.c +++ b/lib/sendf.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -37,6 +37,7 @@ #include "connect.h" #include "vtls/vtls.h" #include "ssh.h" +#include "easyif.h" #include "multiif.h" #include "non-ascii.h" #include "strerror.h" @@ -388,7 +389,7 @@ ssize_t Curl_send_plain(struct connectdata *conn, int num, (WSAEWOULDBLOCK == err) #else /* errno may be EWOULDBLOCK or on some systems EAGAIN when it returned - due to its inability to send off data without blocking. We therefor + due to its inability to send off data without blocking. We therefore treat both error codes the same here */ (EWOULDBLOCK == err) || (EAGAIN == err) || (EINTR == err) || (EINPROGRESS == err) @@ -455,7 +456,7 @@ ssize_t Curl_recv_plain(struct connectdata *conn, int num, char *buf, (WSAEWOULDBLOCK == err) #else /* errno may be EWOULDBLOCK or on some systems EAGAIN when it returned - due to its inability to send off data without blocking. We therefor + due to its inability to send off data without blocking. We therefore treat both error codes the same here */ (EWOULDBLOCK == err) || (EAGAIN == err) || (EINTR == err) #endif @@ -540,18 +541,20 @@ static CURLcode pausewrite(struct Curl_easy *data, } -/* Curl_client_chop_write() writes chunks of data not larger than - * CURL_MAX_WRITE_SIZE via client write callback(s) and - * takes care of pause requests from the callbacks. +/* chop_write() writes chunks of data not larger than CURL_MAX_WRITE_SIZE via + * client write callback(s) and takes care of pause requests from the + * callbacks. */ -CURLcode Curl_client_chop_write(struct connectdata *conn, - int type, - char *ptr, - size_t len) +static CURLcode chop_write(struct connectdata *conn, + int type, + char *optr, + size_t olen) { struct Curl_easy *data = conn->data; curl_write_callback writeheader = NULL; curl_write_callback writebody = NULL; + char *ptr = optr; + size_t len = olen; if(!len) return CURLE_OK; @@ -597,25 +600,30 @@ CURLcode Curl_client_chop_write(struct connectdata *conn, } } - if(writeheader) { - size_t wrote = writeheader(ptr, 1, chunklen, data->set.writeheader); - - if(CURL_WRITEFUNC_PAUSE == wrote) - /* here we pass in the HEADER bit only since if this was body as well - then it was passed already and clearly that didn't trigger the - pause, so this is saved for later with the HEADER bit only */ - return pausewrite(data, CLIENTWRITE_HEADER, ptr, len); - - if(wrote != chunklen) { - failf(data, "Failed writing header"); - return CURLE_WRITE_ERROR; - } - } - ptr += chunklen; len -= chunklen; } + if(writeheader) { + size_t wrote; + ptr = optr; + len = olen; + Curl_set_in_callback(data, true); + wrote = writeheader(ptr, 1, len, data->set.writeheader); + Curl_set_in_callback(data, false); + + if(CURL_WRITEFUNC_PAUSE == wrote) + /* here we pass in the HEADER bit only since if this was body as well + then it was passed already and clearly that didn't trigger the + pause, so this is saved for later with the HEADER bit only */ + return pausewrite(data, CLIENTWRITE_HEADER, ptr, len); + + if(wrote != len) { + failf(data, "Failed writing header"); + return CURLE_WRITE_ERROR; + } + } + return CURLE_OK; } @@ -657,7 +665,7 @@ CURLcode Curl_client_write(struct connectdata *conn, #endif /* CURL_DO_LINEEND_CONV */ } - return Curl_client_chop_write(conn, type, ptr, len); + return chop_write(conn, type, ptr, len); } CURLcode Curl_read_plain(curl_socket_t sockfd, @@ -798,8 +806,11 @@ static int showit(struct Curl_easy *data, curl_infotype type, } #endif /* CURL_DOES_CONVERSIONS */ - if(data->set.fdebug) + if(data->set.fdebug) { + Curl_set_in_callback(data, true); rc = (*data->set.fdebug)(data, type, ptr, size, data->set.debugdata); + Curl_set_in_callback(data, false); + } else { switch(type) { case CURLINFO_TEXT: diff --git a/lib/sendf.h b/lib/sendf.h index fbe4f99c..7c9134de 100644 --- a/lib/sendf.h +++ b/lib/sendf.h @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2014, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -51,8 +51,6 @@ void Curl_failf(struct Curl_easy *, const char *fmt, ...); #define CLIENTWRITE_HEADER (1<<1) #define CLIENTWRITE_BOTH (CLIENTWRITE_BODY|CLIENTWRITE_HEADER) -CURLcode Curl_client_chop_write(struct connectdata *conn, int type, char *ptr, - size_t len) WARN_UNUSED_RESULT; CURLcode Curl_client_write(struct connectdata *conn, int type, char *ptr, size_t len) WARN_UNUSED_RESULT; diff --git a/lib/setopt.c b/lib/setopt.c index a5ef75c7..9c96eb35 100644 --- a/lib/setopt.c +++ b/lib/setopt.c @@ -43,6 +43,7 @@ #include "sendf.h" #include "http2.h" #include "setopt.h" +#include "multiif.h" /* The last 3 #include files should be in this order */ #include "curl_printf.h" @@ -361,6 +362,14 @@ CURLcode Curl_vsetopt(struct Curl_easy *data, CURLoption option, data->set.timevalue = (time_t)va_arg(param, long); break; + case CURLOPT_TIMEVALUE_LARGE: + /* + * This is the value to compare with the remote document with the + * method set with CURLOPT_TIMECONDITION + */ + data->set.timevalue = (time_t)va_arg(param, curl_off_t); + break; + case CURLOPT_SSLVERSION: case CURLOPT_PROXY_SSLVERSION: /* @@ -2101,6 +2110,21 @@ CURLcode Curl_vsetopt(struct Curl_easy *data, CURLoption option, data->set.fclosesocket = va_arg(param, curl_closesocket_callback); break; + case CURLOPT_RESOLVER_START_FUNCTION: + /* + * resolver start callback function: called before a new resolver request + * is started + */ + data->set.resolver_start = va_arg(param, curl_resolver_start_callback); + break; + + case CURLOPT_RESOLVER_START_DATA: + /* + * resolver start callback data pointer. Might be NULL. + */ + data->set.resolver_start_client = va_arg(param, void *); + break; + case CURLOPT_CLOSESOCKETDATA: /* * socket callback data pointer. Might be NULL. @@ -2524,6 +2548,12 @@ CURLcode Curl_vsetopt(struct Curl_easy *data, CURLoption option, case CURLOPT_SSH_COMPRESSION: data->set.ssh_compression = (0 != va_arg(param, long))?TRUE:FALSE; break; + case CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS: + arg = va_arg(param, long); + if(arg < 0) + return CURLE_BAD_FUNCTION_ARGUMENT; + data->set.happy_eyeballs_timeout = arg; + break; default: /* unknown tag and its companion, just ignore: */ result = CURLE_UNKNOWN_OPTION; @@ -2536,6 +2566,9 @@ CURLcode Curl_vsetopt(struct Curl_easy *data, CURLoption option, /* * curl_easy_setopt() is the external interface for setting options on an * easy handle. + * + * NOTE: This is one of few API functions that are allowed to be called from + * within a callback. */ #undef curl_easy_setopt diff --git a/lib/sha256.c b/lib/sha256.c index cd81c025..55716c63 100644 --- a/lib/sha256.c +++ b/lib/sha256.c @@ -29,9 +29,17 @@ #if defined(USE_OPENSSL) +#include <openssl/opensslv.h> + +#if (OPENSSL_VERSION_NUMBER >= 0x0090800fL) +#define USE_OPENSSL_SHA256 +#endif + +#endif + +#ifdef USE_OPENSSL_SHA256 /* When OpenSSL is available we use the SHA256-function from OpenSSL */ #include <openssl/sha.h> - #else /* When no other crypto library is available we use this code segment */ @@ -234,7 +242,7 @@ static int SHA256_Final(unsigned char *out, sha256_compress(md, md->buf); md->curlen = 0; } - /* pad upto 56 bytes of zeroes */ + /* pad up to 56 bytes of zeroes */ while(md->curlen < 56) { md->buf[md->curlen++] = (unsigned char)0; } @@ -709,14 +709,21 @@ static CURLcode smb_connection_state(struct connectdata *conn, bool *done) } /* - * Convert a timestamp from the Windows world (100 nsec units from - * 1 Jan 1601) to Posix time. + * Convert a timestamp from the Windows world (100 nsec units from 1 Jan 1601) + * to Posix time. Cap the output to fit within a time_t. */ -static void get_posix_time(long *out, curl_off_t timestamp) +static void get_posix_time(time_t *out, curl_off_t timestamp) { timestamp -= 116444736000000000; timestamp /= 10000000; - *out = (long) timestamp; +#if SIZEOF_TIME_T < SIZEOF_CURL_OFF_T + if(timestamp > TIME_T_MAX) + *out = TIME_T_MAX; + else if(timestamp < TIME_T_MIN) + *out = TIME_T_MIN; + else +#endif + *out = (time_t) timestamp; } static CURLcode smb_request_state(struct connectdata *conn, bool *done) @@ -1289,6 +1289,11 @@ static CURLcode smtp_perform(struct connectdata *conn, bool *connected, /* Store the first recipient (or NULL if not specified) */ smtp->rcpt = data->set.mail_rcpt; + /* Initial data character is the first character in line: it is implicitly + preceded by a virtual CRLF. */ + smtp->trailing_crlf = TRUE; + smtp->eob = 2; + /* Start the first command in the DO phase */ if((data->set.upload || data->set.mimepost.kind) && data->set.mail_rcpt) /* MAIL transfer */ diff --git a/lib/ssh-libssh.c b/lib/ssh-libssh.c index 56775d70..9e666729 100644 --- a/lib/ssh-libssh.c +++ b/lib/ssh-libssh.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 2017 Red Hat, Inc. + * Copyright (C) 2017 - 2018 Red Hat, Inc. * * Authors: Nikos Mavrogiannopoulos, Tomas Mraz, Stanislav Zidek, * Robert Kolcun, Andreas Schneider @@ -383,8 +383,10 @@ static int myssh_is_known(struct connectdata *conn) } /* we don't have anything equivalent to knownkey. Always NULL */ + Curl_set_in_callback(data, true); rc = func(data, NULL, &foundkey, /* from the remote host */ keymatch, data->set.ssh_keyfunc_userp); + Curl_set_in_callback(data, false); switch(rc) { case CURLKHSTAT_FINE_ADD_TO_FILE: @@ -1046,7 +1048,7 @@ static CURLcode myssh_statemach_act(struct connectdata *conn, bool *block) attrs = sftp_stat(sshc->sftp_session, protop->path); if(attrs != 0) { - data->info.filetime = (long)attrs->mtime; + data->info.filetime = attrs->mtime; sftp_attributes_free(attrs); } @@ -1128,8 +1130,10 @@ static CURLcode myssh_statemach_act(struct connectdata *conn, bool *block) if(data->state.resume_from > 0) { /* Let's read off the proper amount of bytes from the input. */ if(conn->seek_func) { + Curl_set_in_callback(data, true); seekerr = conn->seek_func(conn->seek_client, data->state.resume_from, SEEK_SET); + Curl_set_in_callback(data, false); } if(seekerr != CURL_SEEKFUNC_OK) { @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -342,6 +342,7 @@ static void state(struct connectdata *conn, sshstate nowstate) "SSH_AUTH_HOST", "SSH_AUTH_KEY_INIT", "SSH_AUTH_KEY", + "SSH_AUTH_GSSAPI", "SSH_AUTH_DONE", "SSH_SFTP_INIT", "SSH_SFTP_REALPATH", @@ -376,6 +377,7 @@ static void state(struct connectdata *conn, sshstate nowstate) "SSH_SCP_TRANS_INIT", "SSH_SCP_UPLOAD_INIT", "SSH_SCP_DOWNLOAD_INIT", + "SSH_SCP_DOWNLOAD", "SSH_SCP_DONE", "SSH_SCP_SEND_EOF", "SSH_SCP_WAIT_EOF", @@ -386,6 +388,9 @@ static void state(struct connectdata *conn, sshstate nowstate) "QUIT" }; + /* a precaution to make sure the lists are in sync */ + DEBUGASSERT(sizeof(names)/sizeof(names[0]) == SSH_LAST); + if(sshc->state != nowstate) { infof(conn->data, "SFTP %p state change from %s to %s\n", (void *)sshc, names[sshc->state], names[nowstate]); @@ -523,9 +528,11 @@ static CURLcode ssh_knownhost(struct connectdata *conn) keymatch = (enum curl_khmatch)keycheck; /* Ask the callback how to behave */ + Curl_set_in_callback(data, true); rc = func(data, knownkeyp, /* from the knownhosts file */ &foundkey, /* from the remote host */ keymatch, data->set.ssh_keyfunc_userp); + Curl_set_in_callback(data, false); } else /* no remotekey means failure! */ @@ -1627,7 +1634,7 @@ static CURLcode ssh_statemach_act(struct connectdata *conn, bool *block) break; } if(rc == 0) { - data->info.filetime = (long)attrs.mtime; + data->info.filetime = attrs.mtime; } state(conn, SSH_SFTP_TRANS_INIT); @@ -1747,8 +1754,10 @@ static CURLcode ssh_statemach_act(struct connectdata *conn, bool *block) if(data->state.resume_from > 0) { /* Let's read off the proper amount of bytes from the input. */ if(conn->seek_func) { + Curl_set_in_callback(data, true); seekerr = conn->seek_func(conn->seek_client, data->state.resume_from, SEEK_SET); + Curl_set_in_callback(data, false); } if(seekerr != CURL_SEEKFUNC_OK) { @@ -1765,9 +1774,12 @@ static CURLcode ssh_statemach_act(struct connectdata *conn, bool *block) (size_t)data->set.buffer_size : curlx_sotouz(data->state.resume_from - passed); - size_t actuallyread = - data->state.fread_func(data->state.buffer, 1, - readthisamountnow, data->state.in); + size_t actuallyread; + Curl_set_in_callback(data, true); + actuallyread = data->state.fread_func(data->state.buffer, 1, + readthisamountnow, + data->state.in); + Curl_set_in_callback(data, false); passed += actuallyread; if((actuallyread == 0) || (actuallyread > readthisamountnow)) { diff --git a/lib/strerror.c b/lib/strerror.c index 83a96dda..0295d6c2 100644 --- a/lib/strerror.c +++ b/lib/strerror.c @@ -312,6 +312,9 @@ curl_easy_strerror(CURLcode error) case CURLE_HTTP2_STREAM: return "Stream error in the HTTP/2 framing layer"; + case CURLE_RECURSIVE_API_CALL: + return "API function called from within callback"; + /* error codes not used by current libcurl */ case CURLE_OBSOLETE20: case CURLE_OBSOLETE24: @@ -380,6 +383,9 @@ curl_multi_strerror(CURLMcode error) case CURLM_ADDED_ALREADY: return "The easy handle is already added to a multi handle"; + case CURLM_RECURSIVE_API_CALL: + return "API function called from within callback"; + case CURLM_LAST: break; } diff --git a/lib/telnet.c b/lib/telnet.c index 48b134ee..78d3a853 100644 --- a/lib/telnet.c +++ b/lib/telnet.c @@ -1460,7 +1460,8 @@ static CURLcode telnet_do(struct connectdata *conn, bool *done) if(n == 0) /* no bytes */ break; - readfile_read = (DWORD)n; /* fall thru with number of bytes read */ + /* fall through with number of bytes read */ + readfile_read = (DWORD)n; } else { /* read from stdin */ diff --git a/lib/transfer.c b/lib/transfer.c index 8f15b1a1..fd9af315 100644 --- a/lib/transfer.c +++ b/lib/transfer.c @@ -5,7 +5,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -85,7 +85,7 @@ !defined(CURL_DISABLE_IMAP) /* * checkheaders() checks the linked list of custom headers for a - * particular header (prefix). + * particular header (prefix). Provide the prefix without colon! * * Returns a pointer to the first matching header or NULL if none matched. */ @@ -97,7 +97,8 @@ char *Curl_checkheaders(const struct connectdata *conn, struct Curl_easy *data = conn->data; for(head = data->set.headers; head; head = head->next) { - if(strncasecompare(head->data, thisheader, thislen)) + if(strncasecompare(head->data, thisheader, thislen) && + Curl_headersep(head->data[thislen]) ) return head->data; } @@ -135,8 +136,10 @@ CURLcode Curl_fillreadbuffer(struct connectdata *conn, int bytes, int *nreadp) /* this function returns a size_t, so we typecast to int to prevent warnings with picky compilers */ + Curl_set_in_callback(data, true); nread = (int)data->state.fread_func(data->req.upload_fromhere, 1, buffersize, data->state.in); + Curl_set_in_callback(data, false); if(nread == CURL_READFUNC_ABORT) { failf(data, "operation aborted by callback"); @@ -302,7 +305,9 @@ CURLcode Curl_readrewind(struct connectdata *conn) if(data->set.seek_func) { int err; + Curl_set_in_callback(data, true); err = (data->set.seek_func)(data->set.seek_client, 0, SEEK_SET); + Curl_set_in_callback(data, false); if(err) { failf(data, "seek callback returned error %d", (int)err); return CURLE_SEND_FAIL_REWIND; @@ -311,8 +316,10 @@ CURLcode Curl_readrewind(struct connectdata *conn) else if(data->set.ioctl_func) { curlioerr err; + Curl_set_in_callback(data, true); err = (data->set.ioctl_func)(data, CURLIOCMD_RESTARTREAD, data->set.ioctl_client); + Curl_set_in_callback(data, false); infof(data, "the ioctl callback returned %d\n", (int)err); if(err) { @@ -801,10 +808,15 @@ static CURLcode readwrite_data(struct Curl_easy *data, } /* if(!header and data to read) */ - if(conn->handler->readwrite && - (excess > 0 && !conn->bits.stream_was_rewound)) { + if(conn->handler->readwrite && excess && !conn->bits.stream_was_rewound) { /* Parse the excess data */ k->str += nread; + + if(&k->str[excess] > &k->buf[data->set.buffer_size]) { + /* the excess amount was too excessive(!), make sure + it doesn't read out of buffer */ + excess = &k->buf[data->set.buffer_size] - k->str; + } nread = (ssize_t)excess; result = conn->handler->readwrite(data, conn, &nread, &readmore); diff --git a/lib/transfer.h b/lib/transfer.h index 72526a83..9ba398d2 100644 --- a/lib/transfer.h +++ b/lib/transfer.h @@ -7,7 +7,7 @@ * | (__| |_| | _ <| |___ * \___|\___/|_| \_\_____| * - * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -22,6 +22,7 @@ * ***************************************************************************/ +#define Curl_headersep(x) ((((x)==':') || ((x)==';'))) char *Curl_checkheaders(const struct connectdata *conn, const char *thisheader); @@ -527,6 +527,7 @@ CURLcode Curl_init_userdefined(struct Curl_easy *data) set->expect_100_timeout = 1000L; /* Wait for a second by default. */ set->sep_headers = TRUE; /* separated header lists by default */ set->buffer_size = READBUFFER_SIZE; + set->happy_eyeballs_timeout = CURL_HET_DEFAULT; Curl_http2_init_userset(set); return result; @@ -2573,7 +2574,15 @@ static bool check_noproxy(const char *name, const char *no_proxy) /* NO_PROXY was specified and it wasn't just an asterisk */ no_proxy_len = strlen(no_proxy); - endptr = strchr(name, ':'); + if(name[0] == '[') { + /* IPv6 numerical address */ + endptr = strchr(name, ']'); + if(!endptr) + return FALSE; + name++; + } + else + endptr = strchr(name, ':'); if(endptr) namelen = endptr - name; else @@ -4116,7 +4125,7 @@ static CURLcode create_conn(struct Curl_easy *data, *************************************************************/ if(prot_missing) { /* We're guessing prefixes here and if we're told to use a proxy or if - we're gonna follow a Location: later or... then we need the protocol + we're going to follow a Location: later or... then we need the protocol part added so that we have a valid URL. */ char *reurl; char *ch_lower; diff --git a/lib/urldata.h b/lib/urldata.h index 5c04ad17..3d7b9e5a 100644 --- a/lib/urldata.h +++ b/lib/urldata.h @@ -1024,10 +1024,8 @@ struct PureInfo { int httpcode; /* Recent HTTP, FTP, RTSP or SMTP response code */ int httpproxycode; /* response code from proxy when received separate */ int httpversion; /* the http version number X.Y = X*10+Y */ - long filetime; /* If requested, this is might get set. Set to -1 if the time - was unretrievable. We cannot have this of type time_t, - since time_t is unsigned on several platforms such as - OpenVMS. */ + time_t filetime; /* If requested, this is might get set. Set to -1 if the + time was unretrievable. */ bool timecond; /* set to TRUE if the time condition didn't match, which thus made the document NOT get fetched */ long header_size; /* size of read header(s) in bytes */ @@ -1168,7 +1166,7 @@ struct Curl_http2_dep { }; /* - * This struct is for holding data that was attemped to get sent to the user's + * This struct is for holding data that was attempted to get sent to the user's * callback but is held due to pausing. One instance per type (BOTH, HEADER, * BODY). */ @@ -1522,6 +1520,7 @@ struct UserDefined { long timeout; /* in milliseconds, 0 means no timeout */ long connecttimeout; /* in milliseconds, 0 means no timeout */ long accepttimeout; /* in milliseconds, 0 means no timeout */ + long happy_eyeballs_timeout; /* in milliseconds, 0 is a valid value */ long server_response_timeout; /* in milliseconds, 0 means no timeout */ long tftp_blksize; /* in bytes, 0 means use default */ bool tftp_no_options; /* do not send TFTP options requests */ @@ -1682,6 +1681,10 @@ struct UserDefined { struct Curl_http2_dep *stream_dependents; bool abstract_unix_socket; + + curl_resolver_start_callback resolver_start; /* optional callback called + before resolver start */ + void *resolver_start_client; /* pointer to pass to resolver start callback */ }; struct Names { diff --git a/lib/vtls/cyassl.c b/lib/vtls/cyassl.c index 46b71bfd..1bd42d2c 100644 --- a/lib/vtls/cyassl.c +++ b/lib/vtls/cyassl.c @@ -199,8 +199,14 @@ cyassl_connect_step1(struct connectdata *conn, use_sni(TRUE); break; case CURL_SSLVERSION_TLSv1_3: +#ifdef WOLFSSL_TLS13 + req_method = wolfTLSv1_3_client_method(); + use_sni(TRUE); + break; +#else failf(data, "CyaSSL: TLS 1.3 is not yet supported"); return CURLE_SSL_CONNECT_ERROR; +#endif case CURL_SSLVERSION_SSLv3: #ifdef WOLFSSL_ALLOW_SSLV3 req_method = SSLv3_client_method(); @@ -245,7 +251,11 @@ cyassl_connect_step1(struct connectdata *conn, */ if((wolfSSL_CTX_SetMinVersion(BACKEND->ctx, WOLFSSL_TLSV1) != 1) && (wolfSSL_CTX_SetMinVersion(BACKEND->ctx, WOLFSSL_TLSV1_1) != 1) && - (wolfSSL_CTX_SetMinVersion(BACKEND->ctx, WOLFSSL_TLSV1_2) != 1)) { + (wolfSSL_CTX_SetMinVersion(BACKEND->ctx, WOLFSSL_TLSV1_2) != 1) +#ifdef WOLFSSL_TLS13 + && (wolfSSL_CTX_SetMinVersion(BACKEND->ctx, WOLFSSL_TLSV1_3) != 1) +#endif + ) { failf(data, "SSL: couldn't set the minimum protocol version"); return CURLE_SSL_CONNECT_ERROR; } diff --git a/lib/vtls/darwinssl.c b/lib/vtls/darwinssl.c index 53a7ec37..694ac572 100644 --- a/lib/vtls/darwinssl.c +++ b/lib/vtls/darwinssl.c @@ -1135,28 +1135,77 @@ static OSStatus CopyIdentityFromPKCS12File(const char *cPath, raise linker errors when used on that cat for some reason. */ #if CURL_BUILD_MAC_10_7 || CURL_BUILD_IOS if(CFURLCreateDataAndPropertiesFromResource(NULL, pkcs_url, &pkcs_data, - NULL, NULL, &status)) { + NULL, NULL, &status)) { + CFArrayRef items = NULL; + + /* On iOS SecPKCS12Import will never add the client certificate to the + * Keychain. + * + * It gives us back a SecIdentityRef that we can use directly. */ +#if CURL_BUILD_IOS const void *cKeys[] = {kSecImportExportPassphrase}; const void *cValues[] = {password}; CFDictionaryRef options = CFDictionaryCreate(NULL, cKeys, cValues, password ? 1L : 0L, NULL, NULL); - CFArrayRef items = NULL; - /* Here we go: */ - status = SecPKCS12Import(pkcs_data, options, &items); - if(status == errSecSuccess && items && CFArrayGetCount(items)) { - CFDictionaryRef identity_and_trust = CFArrayGetValueAtIndex(items, 0L); - const void *temp_identity = CFDictionaryGetValue(identity_and_trust, - kSecImportItemIdentity); + if(options != NULL) { + status = SecPKCS12Import(pkcs_data, options, &items); + CFRelease(options); + } + - /* Retain the identity; we don't care about any other data... */ - CFRetain(temp_identity); - *out_cert_and_key = (SecIdentityRef)temp_identity; + /* On macOS SecPKCS12Import will always add the client certificate to + * the Keychain. + * + * As this doesn't match iOS, and apps may not want to see their client + * certificate saved in the the user's keychain, we use SecItemImport + * with a NULL keychain to avoid importing it. + * + * This returns a SecCertificateRef from which we can construct a + * SecIdentityRef. + */ +#elif CURL_BUILD_MAC_10_7 + SecItemImportExportKeyParameters keyParams; + SecExternalFormat inputFormat = kSecFormatPKCS12; + SecExternalItemType inputType = kSecItemTypeCertificate; + + memset(&keyParams, 0x00, sizeof(keyParams)); + keyParams.version = SEC_KEY_IMPORT_EXPORT_PARAMS_VERSION; + keyParams.passphrase = password; + + status = SecItemImport(pkcs_data, NULL, &inputFormat, &inputType, + 0, &keyParams, NULL, &items); +#endif + + + /* Extract the SecIdentityRef */ + if(status == errSecSuccess && items && CFArrayGetCount(items)) { + CFIndex i, count; + count = CFArrayGetCount(items); + + for(i = 0; i < count; i++) { + CFTypeRef item = (CFTypeRef) CFArrayGetValueAtIndex(items, i); + CFTypeID itemID = CFGetTypeID(item); + + if(itemID == CFDictionaryGetTypeID()) { + CFTypeRef identity = (CFTypeRef) CFDictionaryGetValue( + (CFDictionaryRef) item, + kSecImportItemIdentity); + CFRetain(identity); + *out_cert_and_key = (SecIdentityRef) identity; + break; + } + else if(itemID == SecCertificateGetTypeID()) { + status = SecIdentityCreateWithCertificate(NULL, + (SecCertificateRef) item, + out_cert_and_key); + break; + } + } } if(items) CFRelease(items); - CFRelease(options); CFRelease(pkcs_data); } #endif /* CURL_BUILD_MAC_10_7 || CURL_BUILD_IOS */ diff --git a/lib/vtls/gskit.c b/lib/vtls/gskit.c index 8f0cc0bb..afc90a85 100644 --- a/lib/vtls/gskit.c +++ b/lib/vtls/gskit.c @@ -1355,7 +1355,7 @@ const struct Curl_ssl Curl_ssl_gskit = { 0, /* have_ca_path */ 1, /* have_certinfo */ - 0, /* have_pinnedpubkey */ + 1, /* have_pinnedpubkey */ 0, /* have_ssl_ctx */ /* TODO: convert to 1 and fix test #1014 (if need) */ 0, /* support_https_proxy */ diff --git a/lib/vtls/gtls.c b/lib/vtls/gtls.c index 30b255b8..07887410 100644 --- a/lib/vtls/gtls.c +++ b/lib/vtls/gtls.c @@ -60,15 +60,6 @@ /* The last #include file should be: */ #include "memdebug.h" -#ifndef GNUTLS_POINTER_TO_SOCKET_CAST -#define GNUTLS_POINTER_TO_SOCKET_CAST(p) \ - ((curl_socket_t) ((char *)(p) - (char *)NULL)) -#endif -#ifndef GNUTLS_SOCKET_TO_POINTER_CAST -#define GNUTLS_SOCKET_TO_POINTER_CAST(s) \ - ((void *) ((char *)NULL + (s))) -#endif - /* Enable GnuTLS debugging by defining GTLSDEBUG */ /*#define GTLSDEBUG */ @@ -161,7 +152,7 @@ static int gtls_mapped_sockerrno(void) static ssize_t Curl_gtls_push(void *s, const void *buf, size_t len) { - ssize_t ret = swrite(GNUTLS_POINTER_TO_SOCKET_CAST(s), buf, len); + ssize_t ret = swrite(CURLX_POINTER_TO_INTEGER_CAST(s), buf, len); #if defined(USE_WINSOCK) && !defined(GNUTLS_MAPS_WINSOCK_ERRORS) if(ret < 0) gnutls_transport_set_global_errno(gtls_mapped_sockerrno()); @@ -171,7 +162,7 @@ static ssize_t Curl_gtls_push(void *s, const void *buf, size_t len) static ssize_t Curl_gtls_pull(void *s, void *buf, size_t len) { - ssize_t ret = sread(GNUTLS_POINTER_TO_SOCKET_CAST(s), buf, len); + ssize_t ret = sread(CURLX_POINTER_TO_INTEGER_CAST(s), buf, len); #if defined(USE_WINSOCK) && !defined(GNUTLS_MAPS_WINSOCK_ERRORS) if(ret < 0) gnutls_transport_set_global_errno(gtls_mapped_sockerrno()); @@ -857,7 +848,7 @@ gtls_connect_step1(struct connectdata *conn, } else { /* file descriptor for the socket */ - transport_ptr = GNUTLS_SOCKET_TO_POINTER_CAST(conn->sock[sockindex]); + transport_ptr = CURLX_INTEGER_TO_POINTER_CAST(conn->sock[sockindex]); gnutls_transport_push = Curl_gtls_push; gnutls_transport_pull = Curl_gtls_pull; } diff --git a/lib/vtls/nss.c b/lib/vtls/nss.c index a3ef37a1..458f9d81 100644 --- a/lib/vtls/nss.c +++ b/lib/vtls/nss.c @@ -440,7 +440,17 @@ static CURLcode nss_create_object(struct ssl_connect_data *connssl, PK11_SETATTRS(attrs, attr_cnt, CKA_TRUST, pval, sizeof(*pval)); } - obj = PK11_CreateGenericObject(slot, attrs, attr_cnt, PR_FALSE); + /* PK11_CreateManagedGenericObject() was introduced in NSS 3.34 because + * PK11_DestroyGenericObject() does not release resources allocated by + * PK11_CreateGenericObject() early enough. */ + obj = +#ifdef HAVE_PK11_CREATEMANAGEDGENERICOBJECT + PK11_CreateManagedGenericObject +#else + PK11_CreateGenericObject +#endif + (slot, attrs, attr_cnt, PR_FALSE); + PK11_FreeSlot(slot); if(!obj) return result; diff --git a/lib/vtls/openssl.c b/lib/vtls/openssl.c index 93faa6fa..2a6b3cfa 100644 --- a/lib/vtls/openssl.c +++ b/lib/vtls/openssl.c @@ -2338,10 +2338,11 @@ static CURLcode ossl_connect_step1(struct connectdata *conn, int sockindex) #endif if(ssl_cafile || ssl_capath) { - /* tell SSL where to find CA certificates that are used to verify - the servers certificate. */ - if(!SSL_CTX_load_verify_locations(BACKEND->ctx, ssl_cafile, ssl_capath)) { - if(verifypeer) { + if(verifypeer) { + /* tell SSL where to find CA certificates that are used to verify + the servers certificate. */ + if(!SSL_CTX_load_verify_locations(BACKEND->ctx, + ssl_cafile, ssl_capath)) { /* Fail if we insist on successfully verifying the server. */ failf(data, "error setting certificate verify locations:\n" " CAfile: %s\n CApath: %s", @@ -2349,20 +2350,18 @@ static CURLcode ossl_connect_step1(struct connectdata *conn, int sockindex) ssl_capath ? ssl_capath : "none"); return CURLE_SSL_CACERT_BADFILE; } - /* Just continue with a warning if no strict certificate verification - is required. */ - infof(data, "error setting certificate verify locations," - " continuing anyway:\n"); + else { + /* Everything is fine. */ + infof(data, "successfully set certificate verify locations:\n" + " CAfile: %s\n CApath: %s\n", + ssl_cafile ? ssl_cafile : "none", + ssl_capath ? ssl_capath : "none"); + } } else { - /* Everything is fine. */ - infof(data, "successfully set certificate verify locations:\n"); + infof(data, "ignoring certificate verify locations due to " + "disabled peer verification\n"); } - infof(data, - " CAfile: %s\n" - " CApath: %s\n", - ssl_cafile ? ssl_cafile : "none", - ssl_capath ? ssl_capath : "none"); } #ifdef CURL_CA_FALLBACK else if(verifypeer) { @@ -3580,11 +3579,15 @@ static CURLcode Curl_ossl_md5sum(unsigned char *tmp, /* input */ unsigned char *md5sum /* output */, size_t unused) { - MD5_CTX MD5pw; - (void)unused; - MD5_Init(&MD5pw); - MD5_Update(&MD5pw, tmp, tmplen); - MD5_Final(md5sum, &MD5pw); + EVP_MD_CTX *mdctx; + unsigned int len = 0; + (void) unused; + + mdctx = EVP_MD_CTX_create(); + EVP_DigestInit_ex(mdctx, EVP_md5(), NULL); + EVP_DigestUpdate(mdctx, tmp, tmplen); + EVP_DigestFinal_ex(mdctx, md5sum, &len); + EVP_MD_CTX_destroy(mdctx); return CURLE_OK; } @@ -3594,11 +3597,15 @@ static void Curl_ossl_sha256sum(const unsigned char *tmp, /* input */ unsigned char *sha256sum /* output */, size_t unused) { - SHA256_CTX SHA256pw; - (void)unused; - SHA256_Init(&SHA256pw); - SHA256_Update(&SHA256pw, tmp, tmplen); - SHA256_Final(sha256sum, &SHA256pw); + EVP_MD_CTX *mdctx; + unsigned int len = 0; + (void) unused; + + mdctx = EVP_MD_CTX_create(); + EVP_DigestInit_ex(mdctx, EVP_sha256(), NULL); + EVP_DigestUpdate(mdctx, tmp, tmplen); + EVP_DigestFinal_ex(mdctx, sha256sum, &len); + EVP_MD_CTX_destroy(mdctx); } #endif diff --git a/lib/vtls/schannel.c b/lib/vtls/schannel.c index 85c64cf4..b8afe46f 100644 --- a/lib/vtls/schannel.c +++ b/lib/vtls/schannel.c @@ -7,7 +7,7 @@ * * Copyright (C) 2012 - 2016, Marc Hoersken, <info@marc-hoersken.de> * Copyright (C) 2012, Mark Salisbury, <mark.salisbury@hp.com> - * Copyright (C) 2012 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al. + * Copyright (C) 2012 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al. * * This software is licensed as described in the file COPYING, which * you should have received as part of this distribution. The terms @@ -129,6 +129,10 @@ * #define failf(x, y, ...) printf(y, __VA_ARGS__) */ +#ifndef CALG_SHA_256 +# define CALG_SHA_256 0x0000800c +#endif + /* Structs to store Schannel handles */ struct curl_schannel_cred { CredHandle cred_handle; @@ -165,6 +169,9 @@ struct ssl_backend_data { static Curl_recv schannel_recv; static Curl_send schannel_send; +static CURLcode pkp_pin_peer_pubkey(struct connectdata *conn, int sockindex, + const char *pinnedpubkey); + #ifdef _WIN32_WCE static CURLcode verify_certificate(struct connectdata *conn, int sockindex); #endif @@ -542,6 +549,7 @@ schannel_connect_step2(struct connectdata *conn, int sockindex) bool doread; char * const hostname = SSL_IS_PROXY() ? conn->http_proxy.host.name : conn->host.name; + const char *pubkey_ptr; doread = (connssl->connecting_state != ssl_connect_2_writing) ? TRUE : FALSE; @@ -761,6 +769,17 @@ schannel_connect_step2(struct connectdata *conn, int sockindex) infof(data, "schannel: SSL/TLS handshake complete\n"); } + pubkey_ptr = SSL_IS_PROXY() ? + data->set.str[STRING_SSL_PINNEDPUBLICKEY_PROXY] : + data->set.str[STRING_SSL_PINNEDPUBLICKEY_ORIG]; + if(pubkey_ptr) { + result = pkp_pin_peer_pubkey(conn, sockindex, pubkey_ptr); + if(result) { + failf(data, "SSL: public key does not match pinned public key!"); + return result; + } + } + #ifdef _WIN32_WCE /* Windows CE doesn't do any server certificate validation. We have to do it manually. */ @@ -1669,6 +1688,68 @@ static CURLcode Curl_schannel_random(struct Curl_easy *data UNUSED_PARAM, return CURLE_OK; } +static CURLcode pkp_pin_peer_pubkey(struct connectdata *conn, int sockindex, + const char *pinnedpubkey) +{ + SECURITY_STATUS status; + struct Curl_easy *data = conn->data; + struct ssl_connect_data *connssl = &conn->ssl[sockindex]; + CERT_CONTEXT *pCertContextServer = NULL; + const char *x509_der; + DWORD x509_der_len; + curl_X509certificate x509_parsed; + curl_asn1Element *pubkey; + + /* Result is returned to caller */ + CURLcode result = CURLE_SSL_PINNEDPUBKEYNOTMATCH; + + /* if a path wasn't specified, don't pin */ + if(!pinnedpubkey) + return CURLE_OK; + + do { + status = s_pSecFn->QueryContextAttributes(&BACKEND->ctxt->ctxt_handle, + SECPKG_ATTR_REMOTE_CERT_CONTEXT, + &pCertContextServer); + + if((status != SEC_E_OK) || (pCertContextServer == NULL)) { + failf(data, "schannel: Failed to read remote certificate context: %s", + Curl_sspi_strerror(conn, status)); + break; /* failed */ + } + + + if(!(((pCertContextServer->dwCertEncodingType & X509_ASN_ENCODING) != 0) && + (pCertContextServer->cbCertEncoded > 0))) + break; + + x509_der = (const char *)pCertContextServer->pbCertEncoded; + x509_der_len = pCertContextServer->cbCertEncoded; + memset(&x509_parsed, 0, sizeof x509_parsed); + if(Curl_parseX509(&x509_parsed, x509_der, x509_der + x509_der_len)) + break; + + pubkey = &x509_parsed.subjectPublicKeyInfo; + if(!pubkey->header || pubkey->end <= pubkey->header) { + failf(data, "SSL: failed retrieving public key from server certificate"); + break; + } + + result = Curl_pin_peer_pubkey(data, + pinnedpubkey, + (const unsigned char *)pubkey->header, + (size_t)(pubkey->end - pubkey->header)); + if(result) { + failf(data, "SSL: public key does not match pinned public key!"); + } + } while(0); + + if(pCertContextServer) + CertFreeCertificateContext(pCertContextServer); + + return result; +} + #ifdef _WIN32_WCE static CURLcode verify_certificate(struct connectdata *conn, int sockindex) { @@ -1809,6 +1890,74 @@ static CURLcode verify_certificate(struct connectdata *conn, int sockindex) } #endif /* _WIN32_WCE */ +static void Curl_schannel_checksum(const unsigned char *input, + size_t inputlen, + unsigned char *checksum, + size_t checksumlen, + DWORD provType, + const unsigned int algId) +{ + HCRYPTPROV hProv = 0; + HCRYPTHASH hHash = 0; + DWORD cbHashSize = 0; + DWORD dwHashSizeLen = (DWORD)sizeof(cbHashSize); + DWORD dwChecksumLen = (DWORD)checksumlen; + + /* since this can fail in multiple ways, zero memory first so we never + * return old data + */ + memset(checksum, 0, checksumlen); + + if(!CryptAcquireContext(&hProv, NULL, NULL, provType, + CRYPT_VERIFYCONTEXT)) + return; /* failed */ + + do { + if(!CryptCreateHash(hProv, algId, 0, 0, &hHash)) + break; /* failed */ + + if(!CryptHashData(hHash, (const BYTE*)input, (DWORD)inputlen, 0)) + break; /* failed */ + + /* get hash size */ + if(!CryptGetHashParam(hHash, HP_HASHSIZE, (BYTE *)&cbHashSize, + &dwHashSizeLen, 0)) + break; /* failed */ + + /* check hash size */ + if(checksumlen < cbHashSize) + break; /* failed */ + + if(CryptGetHashParam(hHash, HP_HASHVAL, checksum, &dwChecksumLen, 0)) + break; /* failed */ + } while(0); + + if(hHash) + CryptDestroyHash(hHash); + + if(hProv) + CryptReleaseContext(hProv, 0); +} + +static CURLcode Curl_schannel_md5sum(unsigned char *input, + size_t inputlen, + unsigned char *md5sum, + size_t md5len) +{ + Curl_schannel_checksum(input, inputlen, md5sum, md5len, + PROV_RSA_FULL, CALG_MD5); + return CURLE_OK; +} + +static void Curl_schannel_sha256sum(const unsigned char *input, + size_t inputlen, + unsigned char *sha256sum, + size_t sha256len) +{ + Curl_schannel_checksum(input, inputlen, sha256sum, sha256len, + PROV_RSA_AES, CALG_SHA_256); +} + static void *Curl_schannel_get_internals(struct ssl_connect_data *connssl, CURLINFO info UNUSED_PARAM) { @@ -1821,7 +1970,7 @@ const struct Curl_ssl Curl_ssl_schannel = { 0, /* have_ca_path */ 1, /* have_certinfo */ - 0, /* have_pinnedpubkey */ + 1, /* have_pinnedpubkey */ 0, /* have_ssl_ctx */ 0, /* support_https_proxy */ @@ -1845,8 +1994,8 @@ const struct Curl_ssl Curl_ssl_schannel = { Curl_none_set_engine_default, /* set_engine_default */ Curl_none_engines_list, /* engines_list */ Curl_none_false_start, /* false_start */ - Curl_none_md5sum, /* md5sum */ - NULL /* sha256sum */ + Curl_schannel_md5sum, /* md5sum */ + Curl_schannel_sha256sum /* sha256sum */ }; #endif /* USE_SCHANNEL */ diff --git a/lib/warnless.h b/lib/warnless.h index ab6d2999..efd552a6 100644 --- a/lib/warnless.h +++ b/lib/warnless.h @@ -26,6 +26,11 @@ #include <curl/curl.h> /* for curl_socket_t */ #endif +#define CURLX_POINTER_TO_INTEGER_CAST(p) \ + ((char *)(p) - (char *)NULL) +#define CURLX_INTEGER_TO_POINTER_CAST(i) \ + ((void *)((char *)NULL + (i))) + unsigned short curlx_ultous(unsigned long ulnum); unsigned char curlx_ultouc(unsigned long ulnum); |