lynx-dev Updated openssl docs

[Ilya Zakharevich]

These docs contain some OS/2-specific part (please ignore it), but the
rest is not OS/2-specific.  I spent a lot of time making this readable
and informative.  I marked by ??? the places which I do not understand
or do not qualify to answer.

Any comment is going to be appreciated; especially those clarifying
??? parts.  Feel free to use this as a base of Lynx docs.

Thanks,
Ilya

==================================================================

This is openssl-0.9.7c binary for OS/2.


I compiled it on Warp4 with emx 0.9d after applying a patch from 
Ilya Zakharevich. See end of this file for the descritopn of the patch.
The DLLs and EXE are compressed with LXLite.

Grab the full source at http://www.openssl.org/


Q. How to use the openssl DLLs with third-parties applications?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Put the DLLs to a directory in your LIBPATH.  If you have applications
using the dlls with old names ssl.dll and crypto.dll install the wrapper
dlls from the backward subdirectory too. 

To use certificates or a cert bundle within a SSL enabled application
like lynx you have to place your certificate files into the diretory
/usr/local/sll.  Alternatively, set the enviroenment variables to a
propper value (e.g., in CONFIG.SYS file)

 set SSL_CERT_DIR=x:/usr/local/ssl/certs
 set SSL_CERT_FILE=x:/usr/local/ssl/cert.pem

(See "What are root certificates" below.)

For more details about using the open SLL libraries from an application
read the README files of the application.

Q.  Why would I want to install openssl.exe?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

openssl.exe is used to manage certificates.  (See "What are root certificates"
below.)

Q.  How to install openssl.exe?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Put openssl.exe in a directory in your PATH and the DLLs to a directory 
in your LIBPATH.

Copy conf\openssl.cnf.demoCA to a directory of your 
choice, rename it to openssl.conf and set the environement variable 
OPENSSL_CONF by putting

SET OPENSSL_CONF=<your-direcotry>\openssl.cnf

into CONFIG.SYS.

...  Should not one edit dir= line in this file???


Q. Why is this document so paranoid?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want to use OpenSSL, then probably your Internet transaction have
*real* monetary value embedded in them.  And as usual, the security is as good
as the weakiest link.  This document unravels only the tip of the iceberg
of what can go wrong with unproperly established "secure" connections.  And
given the monetary value involved, "bad guys" have a high incentive to exploit
the weakiest links.  As experience shows, do not underestimate intelligence
of bad guys...

Really, with security, a little knowledge is a dangerous thing; one can
suspect that many people, if they really understood the trust structures
associated with SSL, would be rather careful about checking the details
of certificates.

Q. What are root certificates?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Making a secure connection is like sending your valuables (for storage or
consumption) to somebody which agreed to be at a prearranged place.  To
guard the valuables on the way there, you can ask for a police escort; this is
what https:// connections are about.  However, it does not make any sense to
have an escort if the goods are transfered to a random person who happens to be
at this place; one needs to certify the identity of the receiver as well.

The certification process is a chain; when a site A wants to certify that it is
actually what it claims, it actually says "Check this certificate with site B";
to proceed, one needs to certify that the site B is what it claims, so B may
redirect to site C etc.  For this process to stop, some sites claim that
"You must know my certificate, check it yourselves".  These certificates are
"root certificates"; one cannot verify a site unless one has the certificate
for the "end of its certification chain".  If you don't have the
relevant root certificate in the your local certificates file, it means that
you don't trust anyone to vouch for the authenticity of the site.

So one should have a collection of known certificates from several well-known
sites known as "Root Certification Authorities".  Most sites for large-scale
businesses have certificates which will eventually resolve to these places.
Such certicates represent people like Verisign that are in the business of
confirming the identity of servers, etc.

Additionally, since having yourselves certified through another site costs,
some sites would avoid this cost via presenting "end-of-chain certificates".
One should have a way to obtain these certificates via other mean than
unsecure Internet connection (e.g., one can walk into the office and copy
the certificate file to a floppy).  These are so-called "Self-signed
certificates"; they are "root certificates" as well.  The locally-installed
securely obtained copies of such certificates are refered to as
"local certificates".  (See 'What is "Snake Oil Ltd."' below.)

If you are presented with locally-unresolvable root certificate, and you
*believe* that you are really talking to the site, and not someone
in between (who is either completely simulating the site or relaying
your requests onto the real site - called a "man in the middle" attack),
you will still have an encrypted connection.  Otherwise, you should act
as though the site was an impostor, unless and until you manage to get
a root certificate from a trustworthy source, and that root certificate
represents someone that you would trust to have vetted the site you
want to connect to.

Local certificates are stored in SSL_CERT_FILE ("cert bundle", usually named
cert.pem, usually contains several signatures for "Root Certification
Authorities") and SSL_CERT_DIR (which has a signature per file, and usually
contain local copies of self-signed certificates).

There are three crucial considerations to be added to this picture:

  a) While there are ways to ensure that the receivers are who they claims,
     there is absolutely no technological way to verify how *trustworthy*
     the receiving party is.  It does not make sends to secure-send your
     valuables to a certified receiver if this receiver is a crook (or will
     just keep them later in a publicly accessible place).

  b) "VeriSign Syndrom".  For the above scheme of "a chain of trust" to work,
     the "Root Certification Authorities" should be *very* trustworthy
     high-integrity entities.  Unfortunately, there are certain doubts that
     this is so.  E.g., fall 2003, VeriSign started an attack on DNS scheme
     which could disrupt the whole architecture of Internet (hijacking *all*
     unclaimed Internet addresses and redirecting them to a promotional site;
     google for VeriSign DNS hijack).

     One major company even issued a Microsoft certificate to a company
     other than Microsoft, and there had to be a Windows critical update
     to block that certificate.

  c) Keep in mind that the "big 2 browsers" are adding an increasing
     number of root certificates, and most users fail to realise that they
     are putting a trust in the supply chain for the browser to give them
     the certificates of reliable organisations (the browser suppliers could
     make bad choices, or the browser could have been hacked before you got
     it).

     Incidentally, standard browsers come with certificates representing
     very different levels of identity verification, but most people accept
     all of those supplied with the big 2 as equally valid.

Q. How to obtain root certificates?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Certificate files, such as cert.pem, are security critical; you have to
trust whoever supplies it to you; all your certification process is no more
trustful than the cite you downloaded cert.pem from.  So you shouldn't just
accept any offer.

One way is to copy them from a machine which already obtained them in a secure
way.  Another one is to extract them from a web browser which was itself
obtained in a secure way (see "How to extract certificates from Internet
Explorer" below).  If anything else fails, obtaining some privately-generated
bundle from third-parties, such as

  http://www.kfu.com/~nsayer/encryption/ca-bundle.crt.text

is *not* much better than no certificates at all, but may avoid some warnings
from applications.  One of the places which has a bundle is mod_ssl site.

Q. Should you trust this distribution?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is very hard to imagine a situation when the answer is different from
"Absolutely not!".  

Indeed, obtaining the certificates are only one half of the problem.
The certificates are going to be checked by the SSL library.  Can you trust
these executables (DLLs)?  Did you obtain the library via a secure connection?
Are you sure that the place you obtained it from has reasonable security
practice, so that the archive could not be tampered with?  The latter place
most probably did not build the DLLs themselves; the chances are they just
store what a fourth-party supplied them.  Was *that* file transfer done via
secure channels?  Can you trust this fourth-party so that it did not insert
Trojans?

Chances are that all of these questions are answered "No".  There are still
major problems with bootstrapping security via Internet...

What about the application which uses these DLLs?  Do you have any reason to
trust it?  What about the OS itself?  Did it come from a trustful source via
trustful channels?  Are you sure it was not tampered with?

Q.  How to compile and link with OpenSSL libraries?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Put the files from include and lib to your emx directory,
or directories on C_INCLUDE_PATH and LIBRARY_PATH. 
Note that openssl should become a subdirectory of your include directory.
If you need .lib files you can create them using emxomf.

The supplied library files link against the new renamed dlls open_ssl and
cryptsll.

See the doc directory for some information and visit
http://www.columbia.edu/~ariel/ssleay/ for more infos.


Q. Why do you need your own keys and certificates?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are several situations: having a server which accepts secure connections;
authenticating yourself to a server by means other than login/password,
sending S-Mime crypto-mail, ????.  In each of these situations one needs the
following types of keys: ????

The following sites may be useful????:

   http://www.pseudonym.org/ssl/ssl_cook.html#environment

Q. How to generate your own keys and certificates?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are many ways.  A good solution is to use sslRexx. It provides everything
you need.

Below is a short descriptions how I made my own Certification Authority, 
a Server Key for Apache and a client Key/Certificate for me, signed by my
own CA.


Q. Howto: Root CA (needed to self-sign all certificates)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Generate a CA-Key and store it in sub-directory private: 

  openssl genrsa -des3  -out private/MyOwnCA.pem 2048

Make a selfsigned certificate based on above key.

???? Why is the file name different?

  openssl req -new -x509 -days 730 -key private/CAkey.pem -out CAcert.pem

This certificate will expire in 2 years.  Then one should do ????.  The -x509
switch is ????.

Optional: generate text output of this certificate:

  openssl x509 -in ./CAcert.pem -text > CAcert.txt

Now you have a key and certificate for your own CA which can be used
to sign user and server keys. The CAcert is also needed to configure 
Apache and Netscape. You can/should give away the CA certificate but
never give the CA key to anybody.


Q. Howto: a Key and Certificate for the Apache Server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Generate a key

  openssl genrsa -des3 -out server-key.pem 2048

???  What makes openssl prompt in this case, but not in the previous one
(same command as for root key - except the output file)?

With this variant, you will be prompted for a protecting password.  If
you don't want your key to be protected by a password, remove the flag
'-des3' from the command line above.

NOTE: if you intend to use the key together with a server
certificate, you may want to avoid protecting it with a password,
since that would mean someone would have to type in the password every
time the server needs to access the key. But then you should protect
the server key in another way.

??? "Someone" is who?  Who connects to the server of what?

Create a signing request
------------------------

  openssl req -new -key server-key.pem -out server-req.pem

This request has no -x509 and no expiration date since ????.

Now send this request to your CA for signing. Since you are your own CA
sign it:
--------

  openssl ca -in server-req.pem -out server-cert.pem -outdir MyOwnCA/newcerts

Which CA key is used by this, if any????  What is the significance of
-outdir????  Should -outdir exist????

Verify the certificate

  openssl verify -CAfile CAcert.pem server-cert.pem

Now you have a key and a certificate for your Apache Webserver.  (This assumes
that the CAcert.pem generated on the previous step is still in the current
directory.)  Where should one place these guys????


Q. Howto: send request to your CA for signing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

http://www.modssl.org/related/cert.html????
(Unless you are your own CA; then see above.)

Q. Howto: Your Client Certificate/Key
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Generate a private key 
----------------------

  openssl genrsa -des3 -out hrom-key.pem 2048

(still the same command with yet different file name).  What with passwords
here????

Create a signing request (same command again)
------------------------

  openssl req -new -key hrom-key.pem -out hrom-req.pem

Let the CA sign it (same command again)
------------------

  openssl ca -in hrom-req.pem -out hrom-cert.pem -outdir MyOwnCA/newcerts

After you get back the certificate from the CA, combine it with 
your private key and store the result as p12 file. This file can
be imported into your browser.  The browser will use this file for ????.

  openssl pkcs12 -export -name Hromadka -in hrom-cert.pem -inkey hrom-key.pem 
-out hrom.p12


Security Notes: Never give your private key to a CA, they only need the
signing request. Never give away your p12 file. Always secure your private
keys with a passphrase.


Q. How to use c_rehash?
~~~~~~~~~~~~~~~~~~~~~~

????

One needs a working port of Perl and cp.exe to run this.  Set OPENSSL to the
full name of openssl executable.  One may also need to change some ':' to
$Config{path_sep}.  (Not checked...)

Q. How to extract certificates from Internet Explorer?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To make your own file of certificates, go to the
"Tools/Internet Options/Content/Certificates/Trusted Root Certificates"
section of IE. Select all the certificates, then "export" to a file.
It will be saved as a PKCS#7 file, with suffix ".p7b". You can call
it "ca_bundle.p7b". Then use openssl to convert it with the command:
"openssl pkcs7 -inform DER -in ca_bundle.p7b -print_certs -text -out cert.pem".
Ask your system administrator to put the file "cert.pem" in the openssl
directory. Then lynx can check the certificates against the set of
certificates that you (or Microsoft) trusts, and you won't get the
warning message any more.

Q. How to install a self-signed certificate?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When you would like to trust a self-signed (non-commercial) certificate you will
need to get hold of the actual file. If it's a cert local to your network you
can ask the sysadmin to make it available for download as a link on a webpage.

If such file is not human-readable it's probably DER formatted and will need to 
be converted to PEM format to allow openssl to use it.

To convert DER formatted certificates into something openssl can deal with:

Save the cert as site_name.crt in a directory. In that directory, type:

  openssl x509 -inform DER -in site_name.crt -outform PEM -out site_name.pem

You can now copy this individual cert into the directory for that, usually
/usr/local/ssl/certs.  The alternative is to concatenate the individual certs 
to the cert.pem bundle in /usr/local/ssl. (Please see INSTALLING OR UPDATING 
THE CA BUNDLE below). 

The cert file will now be in an acceptable format to openssl, PEM encoded. 
However, openssl, (and by extension applications, such as lynx), will not know
about it until that cert is present in a file named after the hash value of
that cert, in the cert directory (default /usr/local/ssl/certs).

So the next thing to do is to hash the cert by running c_rehash.


Q.  What is "Snake Oil Ltd."?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The "Snake Oil Ltd." certificate is a testing certificate.
Specifically, it is the self-signed certificate generated by the
installation procedure of Apache-SSL.  The presence of this certificate does
not make your SSL connections less secure: they will still be encrypted and
therefore difficult to intercept or corrupt.

What the web server at "Linux Web Toast" is saying is "Our name is company
XYZ, just take our word for it."  Your software (the browser) is bringing
this to your attention because it is not configured to just take anybody's
word for anything.  A normal secure web server would say something like "Our
name is company XYZ according to VeriSign, Inc, and you can take their word
for it."  Your web browser is probably configured to automatically trust
VeriSign, Inc.



Q. How this build differs from older builds (Ilya's patch)?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This patch

a) Introduces a new file os2/backwardify.pl.

b) Introduces a new mk1mf.pl variable $preamble.  As you can see, it may
   be used also to move some OS-specific code to VC-CE too;

c) The DESCRIPTION specifier of the .def file is made more informative:
   now it contains the version number too.  On OS/2 it is made conformant
   to OS/2 conventions; in particular, when one runs the standard command
        BLDLEVEL this.DLL
   one can see:

   Vendor:      www.openssl.org/
   Revision:    0.9.7c
   Description: OpenSSL: implementation of Secure Socket Layer; DLL for library 
crypto.  Build for EMX -Zmtd

   [I did not make Win32 descriptions as informative as this - I'm afraid to
    break something.  Be welcome to fix this.]

d) On OS/2 the generated DLL was hardly usable (it had a shared initialized
   data segment).

e) On OS/2 the generated DLLs had names like ssl.dll.  However, DLL names on
   OS/2 are "global data".  It is hard to have several DLLs with the same
   name on the system.  Thus this precluded coexistence of OpenSSL with DLLs
   for other SLL implementations - or other name clashes.  I transparently
   changed the names of the DLLs to open_ssl.dll and cryptssl.dll.

f) The file added in (a) is used to create "forwarder" DLLs, so the
   applications expecting the "old" DLL names may use the new DLLs
   transparently.  (A presence of these DLLs on the system nullifies (e),
   but makes old applications work.  This is a stopgap measure until the
   old applications are relinked.  Systems with no old applications do not
   need these DLLs, so may enjoy all the benefits of (e).)

   The new DLLs are placed in os2/ and os2/noname subdirectories.



-- 
Johannes Hromadka
address@hidden
2003-11-08

; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden