Curl Ssl Sample Cover Letter

SSL/TLS Client is sample code for a basic web client that fetches a page. The code shown below omits error checking for brevity, but the sample available for download performs the error checking.

The sample code will set up to fet a page from . The code uses TLS (not SSL) and utilizes the Server Name Indication (SNI) extension from RFC 3546, Transport Layer Security (TLS) Extensions.

If you need features beyond the example below, then you should examine in the directory of the OpenSSL distribution. OpenSSL's implements nearly every client side feature available from the library.

The code below does not perform hostname verification. OpenSSL prior to 1.1.0 does not perform the check, and you must perform the check yourself. The OpenSSL Change Log for OpenSSL 1.1.0 states you can use option, and offers . But does not respond to either switch, so its unclear how hostname checking will be implemented or invoked for a client. Note (N.B.): hostname verification is marked as experimental, so switches, options, and implementations could change.

Finally, if you are looking for guidance on which protocols and ciphers you should be using, then see Adam Langley's blog The POODLE bites again. The short version: use only TLS 1.2, use only ephemeral key exchanges, and use only AEAD ciphers (like AES/GCM, Camellia/GCM, ChaCha/Poly1305).

Implementation[edit]

The code below demonstrates a basic client that uses BIOs and TLS to connect to , and fetches 32 bytes of random data through an HTTP request. The sample code is available for download below.

#define HOST_NAME "www.random.org" #define HOST_PORT "443" #define HOST_RESOURCE "/cgi-bin/randbyte?nbytes=32&format=h" long res = 1; SSL_CTX* ctx = NULL; BIO *web = NULL, *out = NULL; SSL *ssl = NULL; init_openssl_library(); const SSL_METHOD* method = SSLv23_method(); if(!(NULL != method)) handleFailure(); ctx = SSL_CTX_new(method); if(!(ctx != NULL)) handleFailure(); /* Cannot fail ??? */ SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback); /* Cannot fail ??? */ SSL_CTX_set_verify_depth(ctx, 4); /* Cannot fail ??? */ const long flags = SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3 | SSL_OP_NO_COMPRESSION; SSL_CTX_set_options(ctx, flags); res = SSL_CTX_load_verify_locations(ctx, "random-org-chain.pem", NULL); if(!(1 == res)) handleFailure(); web = BIO_new_ssl_connect(ctx); if(!(web != NULL)) handleFailure(); res = BIO_set_conn_hostname(web, HOST_NAME ":" HOST_PORT); if(!(1 == res)) handleFailure(); BIO_get_ssl(web, &ssl); if(!(ssl != NULL)) handleFailure(); const char* const PREFERRED_CIPHERS = "HIGH:!aNULL:!kRSA:!PSK:!SRP:!MD5:!RC4"; res = SSL_set_cipher_list(ssl, PREFERRED_CIPHERS); if(!(1 == res)) handleFailure(); res = SSL_set_tlsext_host_name(ssl, HOST_NAME); if(!(1 == res)) handleFailure(); out = BIO_new_fp(stdout, BIO_NOCLOSE); if(!(NULL != out)) handleFailure(); res = BIO_do_connect(web); if(!(1 == res)) handleFailure(); res = BIO_do_handshake(web); if(!(1 == res)) handleFailure(); /* Step 1: verify a server certificate was presented during the negotiation */ X509* cert = SSL_get_peer_certificate(ssl); if(cert) { X509_free(cert); } /* Free immediately */ if(NULL == cert) handleFailure(); /* Step 2: verify the result of chain verification */ /* Verification performed according to RFC 4158 */ res = SSL_get_verify_result(ssl); if(!(X509_V_OK == res)) handleFailure(); /* Step 3: hostname verification */ /* An exercise left to the reader */ BIO_puts(web, "GET " HOST_RESOURCE " HTTP/1.1\r\n" "Host: " HOST_NAME "\r\n" "Connection: close\r\n\r\n"); BIO_puts(out, "\n"); int len = 0; do { char buff[1536] = {}; len = BIO_read(web, buff, sizeof(buff)); if(len > 0) BIO_write(out, buff, len); } while (len > 0 || BIO_should_retry(web)); if(out) BIO_free(out); if(web != NULL) BIO_free_all(web); if(NULL != ctx) SSL_CTX_free(ctx);

Initialization[edit]

The sample program initializes the OpenSSL library with . calls three OpenSSL functions.

#if (SSLEAY_VERSION_NUMBER >= 0x0907000L) # include <openssl/conf.h> #endif ... void init_openssl_library(void) { (void)SSL_library_init(); SSL_load_error_strings(); /* ERR_load_crypto_strings(); */ OPENSSL_config(NULL); /* Include <openssl/opensslconf.h> to get this define */ #if defined (OPENSSL_THREADS) fprintf(stdout, "Warning: thread locking is not implemented\n"); #endif }

performs initialization of and , and loads required algorithms. The documents state always returns , so its a useless return value.

loads error strings from both and . There's no need to call .

is a for , so the call is omitted.

may (or may not) be needed. Internally, is called based on a configuration options via . If you are dynamically loading an engine specified in , then you might need it so you should call it. That is, don't depend upon the OpenSSL library to call it for you.

If you are building a multi-threaded client, you should set the locking callbacks. See threads(3) for details.

A detailed treatment of initialization can be found at Library Initialization.

Context Setup[edit]

The sample program uses to create a context. specifies that version negotiation will be used. Do not be confused by the name (it does NOT mean that only SSLv2 or SSLv3 will be used). The name is like that for historical reasons, and the function has been renamed to in the forthcoming OpenSSL version 1.1.0. Using this method will negotiate the highest protocol version supported by both the server and the client. SSL/TLS versions currently supported by OpenSSL 1.0.2 are SSLv2, SSLv3, TLS1.0, TLS1.1 and TLS1.2.

The actual SSL and TLS protocols are further tuned through options. By using (and removing the unwanted protocol versions with and ), then you will effectively use TLS v1.0 and above, including TLS v1.2. You can also use and if you want to use the TLS 1.2 protocol only.

uses the method to create a new SSL/TLS context object. If you use, for example , then you will only use TLS v1.0, and if you use then you will only use TLS v1.1. Typically you should always use in preference to the version specific methods.

OpenSSL 1.1.0 improves protocol selection by providing and . You no longer need to subtract unwanted options with and . Also see the and man pages.

Options (1)[edit]

After creating a context with and , the context object is tuned with the following functions:

    sets the flag and the verify callback. This ensures the chain is verified according to RFC 4158 and Issuer and Subject information can be printed. If you don't want to perform custom processing (such as printing or checking), then don't set the callback. OpenSSL's default checking should be sufficient, so pass to .

    There is also a flag, but it is used for servers and has no effect on clients. If you accidentally use , then you chain will always verify when call because the flag is ignored for clients (essentially, 0 is passed for the flag which performs no verification).

    sets the chain depth to 4. Chain depth is fairly useless in practice.

    set the , , , options. In essence, it takes all the bug fixes and work arounds for the various servers, removes the SSL protocols (leaving only TLS protocols), and removes compression. The remaining TLS protocols are TLS 1.0, TLS 1.1, and TLS 1.2.

    loads the certificate chain for the site. The site's CA is Comodo, and the chain includes AddTrust External CA Root, COMODO Certification Authority, and COMODO Extended Validation Secure Server CA. Though the chain is provided, only the single trust anchor is needed for validation. The additional intermediate certs are provided to show how to concatenate and load them.

    The PEM format means the file is a concatenation of Base64 encoded certificates with the prologue (and associated epilogue). If the server sends all certificates required to verify the chain (which it should), then only the AddTrust External CA Root certificate is needed.

    The options set on the can be overridden on a per-connection basis by modifying the using , and (and friends).

    SSL BIO[edit]

    The sample program uses BIOs for input and output. One BIO is used to connect to , and a second BIO is used to print output to .

    creates a new BIO chain consisting of an SSL BIO (using ctx) followed by a connect BIO.

    is used to set the hostname and port that will be used by the connection.

    Options (2)[edit]

    is used to fetch the SSL connection object created by . The connection object inherits from the context object, and can override the settings on the context. The connection object is tuned with the following functions:

      sets the cipher list. The list prefers elliptic curves, ephemeral [Diffie-Hellman], AES and SHA. It also removes NULL authentication methods and ciphers; and removes medium-security, low-security and export-grade security ciphers, such as 40-bit RC2. If desired, you could set the options on the context with .

      uses the TLS SNI extension to set the hostname. If you are connecting to a Server Name Indication-aware server (such as Apache with name-based virtual hosts or IIS 8.0), then you will receive the proper certificate during the handshake.

      Cipher Suites[edit]

      According to , there are just over 110 cipher suites available. Each cipher suite takes 2 bytes in the , so advertising every cipher suite available at the client is going to cause a big (or bigger then needed to get the job done). When using or with the string , you'll cut the number of cipher suites down to about 45. If you know the server does not support DSA, then you can add and reduce the list further by about 7. And removing RSA key transport () removes another 9 more (this is a good practice because it uses ephemeral key exchanges which provide forward secrecy). Advertising 35 or so ciphers saves about 160 bytes in the .

      Better, pick 16 or 20 ciphers you want to support and advertise them. Order them so the GCM mode ciphers from TLS 1.2 are listed first, and the AES-SHA ciphers from TLS 1.0 are listed last. Though TLS 1.0 should be avoided, its probably needed for interop because only about half the servers on the internet support TLS 1.2. If you control the server, then it should be offering TLS 1.2 and clients only need to advertise AEAD ciphers like AES/GCM or Camellia/GCM.

      Keeping the small is important for older F5 and IronPort devices. Apparently, the devices used fixed sized buffers and choke on large 's. In fact, a "large hello" was the cause of the TLS padding bug on IronPort devices. See TLS padding breaks ironport on the TLS mailing list for details.

      Connection[edit]

      After setting the connection object options, the sample connects to the site and negotiates a secure channel.

        performs the name lookup for the host and standard TCP/IP three way handshake.

        performs the SSL/TLS handshake. If you set a callback with or , then you callback will be invoked for each certificate in the chain used during the execution of the protocol.

        The Wireshark packet capture to the right shows the TLS handshake with the SNI extension encountered during the execution of . OpenSSL 1.0.1e advertises TLSv1.2 as the highest protocol level in its .

        Callback[edit]

        OpenSSL provides the ability for an application to interact with the chain validation by way of a callback. Normally, most application don't need to use it since the default OpenSSL behavior is usually adequate. In the callback, you can pass the result back to the library (leaving library behavior unchanged), or you can modify the result to account for a specific issue that your software should address (override default behavior). If you don't need to interact with chain validation, then don't set the callback.

        The example program returned the result to the library and just printed information about the certificate in the chain. It did so by using with and the .

        int verify_callback(int preverify, X509_STORE_CTX* x509_ctx) { int depth = X509_STORE_CTX_get_error_depth(x509_ctx); int err = X509_STORE_CTX_get_error(x509_ctx); X509* cert = X509_STORE_CTX_get_current_cert(x509_ctx); X509_NAME* iname = cert ? X509_get_issuer_name(cert) : NULL; X509_NAME* sname = cert ? X509_get_subject_name(cert) : NULL; print_cn_name("Issuer (cn)", iname); print_cn_name("Subject (cn)", sname); if(depth == 0) { /* If depth is 0, its the server's certificate. Print the SANs too */ print_san_name("Subject (san)", cert); } return preverify; }

        The OpenSSL library will pass in the value of its preliminary checking of the certificate through . If you always return regardless of the value of or the actual result of your processing, then will always return . That's probably a bad idea for production software.

        If you don't need to perform special processing on the chain, then you should forgo the altogether by supplying to :

        SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);

        Verification[edit]

        You use one of two verification procedures, depending on the version of OpenSSL you are using. The change occurs at OpenSSL 1.1.0 because 1.1.0 (and above) implements hostname verification that 1.0.2 (and below) lacked. Painting with a broad brush, minimal checking includes: (1) confirm the server has a certificate, (2) confirm the certificate chain verifies back to a trusted root, and (3) confirm the name of the host matches a hostname listed in the server's certificate.

        In the end, its probably better to ignore PKI and just use Public Key Pinning (or Certificate Pinning) when a pre-exisiting relationship exists; or use a Perspectives-like system or a Trust-On-First-Use (TOFU) system when there's no a priori relationship (similar to SSH's option). See Peter Gutmann's Engineering Security for details of a security diversification strategy (Chapter 4, starting on page 292).

        You usually don't perform revocation in real time because it essentially creates a denial of service on your application. That is, your app will hang while downloading a multi-megabyte CRL or contacts a missing OCSP responder. For a detailed treatment of problems with PKI and Revocation, see Peter Gutmann's Engineering Security (Chapters 1 and 8).

        OpenSSL 1.0.2[edit]

        OpenSSL 1.0.2 and below requires at least three checks. These versions of OpenSSL do not perform hostname validation and the API user must perform it.

        Server Certificate[edit]

        You must confirm the server provided a certificate. This is because a server might be misconfigured, or the client and server used Anonymous Diffie-Hellman. You do so as follows:

        X509* cert = SSL_get_peer_certificate(ssl); if(cert) { X509_free(cert); } if(NULL == cert) handleFailure();

        If the server has a certificate, then will return a non-NULL value. You don't really need the certificate, so its 'd immediately.

        Certificate Chain[edit]

        You must confirm the server's certificate chains back to a trusted root, and all the certificates in the chain are valid. You do so as follows:

        long res = SSL_get_verify_result(ssl); if(!(X509_V_OK == res)) handleFailure();

        returns the result of verifying the chain. See the earlier warning on doing the wrong thing in the verification callback.

        Certificate Names[edit]

        You must confirm a match between the hostname you contacted and the hostnames listed in the certificate. OpenSSL prior to 1.1.0 does not perform hostname verification, so you will have to perform the checking yourself. The sample code does not offer code at the moment, so you will need to borrow it or implement it.

        If you want to borrow the code, take a look at libcurl and the verification procedure in source file . Another source is the C/C++ Secure Coding Guide and Section 10.8, Adding Hostname Checking to Certificate Verification. If you implement the code for checking, the sample code shows you how to extract the Common Name (CN) and Subject Alternate Names (SAN) from the certificate in and .

        Note: matching between the hostname (used in ) and names in the certificate (from ) must also be validated. For example, a certificate cannot claim to be wildcarded for , , or other Top Level Domains (TLDs). In addition to the TLDs, you also have to country level or ccTLDs, so it can't match , , , or similar levels either. Mozilla maintains a list of ccTLDs that are off limits at the Public Suffix List, and there are currently 6136 entries on the list.

        Program Output[edit]

        After all this musing, here's the lousy output you get when running the program:

           

        Session Reuse[edit]

        According to Viktor Dukhovni at Possible to control session reuse from the client:

        > For performance testing purposes, I would like to turn off session > reuse in the (homegrown) client I use for testing. Is there a function > in the openssl library to do it? > > I tried googling for "openssl client don't send session id" but I didn't > find anything useful. Just do nothing. Client sessions are not reused unless you explicitly arrange for reuse of a session by calling SSL_set_session() before SSL_connect(). If you're trying to avoid wasting memory on storing client-side sessions that you'll never reuse then this may help: SSL_CTX_set_session_cache_mode(client_ctx, SSL_SESS_CACHE_OFF); but note this is also the default state, so is also not needed unless some other code has explicitly enabled client-side caching of sessions. Only the server-side cache is enabled by default.

        Session Tickets[edit]

        Session tickets are specified in RFC 5077. You can disable session tickets with :

        const long flags = SSL_OP_NO_SSLv3 | ... | SSL_OP_NO_TICKET; SSL_CTX_set_options(ctx, flags);

        0-RTT[edit]

        0-RTT is specified in XXX (TODO). 0-RTT allows an application to immediately resume a previous session at the expense of consuming unauthenticated data. You should avoid 0-RTT if possible. In fact, an organization's data security policy may not allow it for some higher data sensitivity levels.

        Care should be taken if enabling 0-RTT at the client because a number of protections must be enabled at the server. Additionally, some of the protections are required higher up in the stack, outside of the secure socket layer. Below is a list of potential problems from 0-RTT and Anti-Replay and Closing on 0-RTT on the IETF TLS working group mailing list.

        • 0-RTT without stateful anti-replay allows for very high number of replays, breaking rate limiting systems, even high-performance ones, resulting in an opening for DDoS attacks.
        • 0-RTT without stateful anti-replay allows for very high number of replays, allowing exploiting timing side channels for information leakage. Very few if any applications are engineered to mitigate or eliminate such side channels.
        • 0-RTT without global anti-replay allows leaking information from the 0-RTT data via cache timing attacks. HTTP GET URLs sent to CDNs are especially vulnerable.
        • 0-RTT without global anti-replay allows non-idempotent actions contained in 0-RTT data to be repeated potentially lots of times. Abuse of HTTP GET for non-idempotent actions is fairly common.
        • 0-RTT allows easily reordering request with re-transmission from the client. This can lead to various unexpected application behavior if possibility of such reordering is not taken into account. "Eventually consistent" datastores are especially vulnerable.
        • 0-RTT exporters are not safe for authentication unless the server does global anti-replay on 0-RTT.

        Downloads[edit]

        openssl-bio-fetch.tar.gz - The program and Makefile used for this wiki page.

        Wireshark and ClientHello
        Wireshark and TLS versions

        Curl use case for webdav access using SSL

        Here is curl version:

        $ curl -V
        curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b
        zlib/1.2.3 libidn/0.6.5
        Protocols: tftp ftp telnet dict ldap http file https ftps
        Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

        curl --ciphers ALL:NULL --show-error --connect-timeout 300 --max-time 3600 --capath $X509_CERT_DIR --cert $X509_USER_PROXY --key $X509_USER_PROXY \
         -L https://$StorageElement:2880/$Namespace/$SmallFile -o /tmp/$SmallFile  -3

        But I got trouble when I was trying new curl version

        curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0
        zlib/1.2.3 libidn/1.18 libssh2/1.4.2
        Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp
        Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

        I got Error message like below
        * NSS error -5961
        * Closing connection #0
        * SSL connect error

        curl: (35) SSL connect error

        More detail, there has been some changes since 7.19.7, which started to use NSS.
        http://curl.haxx.se/docs/sslcerts.html
        But, I did not figure how properly use NSS for SSL connection

        However, here is a new way.

        curl --silent --show-error --cacert /tmp/curltest/<x509 proxy> --connect-timeout 300 --max-time 3600 --capath $X509_CERT_DIR --cert $X509_USER_PROXY --key $X509_USER_PROXY -L https://webdavpath -o /tmp/ddddd-https

        Problem is solved, but question is still there, in document, I see

         --cacert (HTTPS) Tells curl to use the specified certificate file to verify the peer.
        The file may contain multiple CA certificates. The certificate(s) must be in PEM format.
        If this option is used several times, the last one will be used.
         --capath (HTTPS) Tells curl to use the specified certificate directory to verify the peer.
        The certificates must be in PEM format, and the directory must have been processed using
        the c_rehash utility supplied with openssl. Certificate directories are not supported
        under Windows (because c_rehash uses symbolink links to create them). Using --capath
        can allow curl to make https connections much more efficiently than using --cacert if
        the --cacert file contains many CA certificates. If this option is used several times,
        the last one will be used.

        So, --capath option is used to specify a directory containing the CA certs to verify the certs of remote servers that curl connects to, and --cacert, to use a single file which the CA certs are stored in. They should be doing the same function.

        I tried to convert CA certs into CA bundle file, or add CA certs into NSS, but non of them worked.

        So the use case is that capath is define to CA bundle and cacert is defined to user proxy to verify host cert

        Details
        Category: Scripting
        Hits: 4552

        One thought on “Curl Ssl Sample Cover Letter

        Leave a Reply

        Your email address will not be published. Required fields are marked *