[Classpath-inetlib] Re: IMAP and SMTP Issues

[Chris Burdess]

Phillip Smith wrote:
> I'm not sure if you're the right person for this information - but here 
> goes.

Absolutely; I've taken the liberty of crossposting to classpath-inetlib
as well.

> I run Courier-IMAP and Postfix; both which only allow DIGEST-MD5 and 
> CRAM-MD5 authentication.  After some code changes, I got it working so I 
> wanted to share my experience so others would not have to go through 
> what I did.
> 
> **** IMAP ****
> 
> 1. The capability() method on gnu.inet.imap.IMAPConnection had to be 
> modified to also parse the asyncResponses:
> 
> // The capability "list" is actually contained in the response
> // text.
> StringBuffer caps = new StringBuffer(response.getText());
> Iterator it = asyncResponses.iterator();
> 
> it.next();
> 
> while (it.hasNext())
> {
>   caps.append(" ").append(((IMAPResponse)it.next()).getText());
> }
> 
> String text = caps.toString();

Ouch - that's not what the specification describes. I'm happy to put
this in, if we can show that it won't result in erroneous data with
other servers.

Jus out of interest, what's the deal with the first next() - a spurious
entry?

> 2. The authenticate() method on gnu.inet.imap.IMAPConnection had to 
> modified to fail-over th custom authenticate_CRAM_MD5() when sasl was 
> null.  I could not figure out how to plug in the GNU sasl provider, I 
> tried putting the JAR Services file in a META-INF directory and 
> explicitly setting the System property 
> -Djavax.security.sasl.client.pkgs=gnu.crypto.sasl.ClientFactory but to 
> no avail.  It always can back null.
> 
> SaslClient sasl = Sasl.createSaslClient(m, null, "smtp",
>                                       socket.getInetAddress().
>                                       getHostName(), p, ch);
> if (sasl == null) {
>   if (CRAM_MD5.equalsIgnoreCase(mechanism)) {
>     if (debug) {
>       Logger logger = Logger.getInstance();
>       logger.log("imap", "sasl not configured");
>     }
> 
>     return authenticate_CRAM_MD5(username, password);
>   }
> 
>   return false;
> }
> 
> 3. I uncommented the authenticate_CRAM_MD5() method hoping that would 
> work 'out of the box'.  Unfortunately it didn't.  Using the bundled 
> BASE64 was not working so I replaced with the JakarataCommons Codec 
> Base64 class and it worked.  I also think it was missing converting the 
> digest to hex - addressed later.
> 
> else if (response.isContinuation())
> {
>   try
>   {
>     /*  GNU Code... replace below
>     byte[] s = secret.getBytes(US_ASCII); // TODO encoding?
>     byte[] c0 = response.getText().getBytes(US_ASCII);
>     byte[] c1 = BASE64.decode(c0); // challenge
>     byte[] digest = hmac_md5(s, c1);
>     byte[] r0 = username.getBytes(US_ASCII); // username
>     byte[] r1 = new byte[r0.length+digest.length+1]; // response
>     System.arraycopy(r0, 0, r1, 0, r0.length); // add username
>     r1[r0.length] = 0x20; // SPACE
>     System.arraycopy(digest, 0, r1, r0.length+1, digest.length);
>     byte[] r2 = BASE64.encode(r1);
>     out.write(r2);
>     out.writeln();
>     out.flush();
>     */
> 
>     String challenge = response.getText();
>     byte decoded[] = Base64.decodeBase64(challenge.getBytes(US_ASCII));
>     String digest = hmac_md5(secret.getBytes(US_ASCII), decoded);
>     String cram = username + " " + digest;
>     byte[] crammed = Base64.encodeBase64(cram.getBytes(US_ASCII));
> 
>     out.write(crammed);
>     out.writeln();
>     out.flush();
>   }
> 
> 4. I also had to uncomment the hmac_md5() method.  I commented out the 
> call to digest.reset() and added hexing the digest, which caused the 
> return type to change from byte[] to String.
> 
> private static String hmac_md5(byte[] key, byte[] text)
>   throws NoSuchAlgorithmException
> {
>   byte[] k_ipad = new byte[64];
>   byte[] k_opad = new byte[64];
>   byte[] digest;
>   MessageDigest md5 = MessageDigest.getInstance("MD5");
>   // if key is longer than 64 bytes reset it to key=MD5(key)
>   if (key.length>64)
>   {
>     md5.update(key);
>     key = md5.digest();
>   }
>   // start out by storing key in pads
>   System.arraycopy(key, 0, k_ipad, 0, key.length);
>   System.arraycopy(key, 0, k_opad, 0, key.length);
>   // XOR key with ipad and opad values
>   for (int i=0; i<64; i++)
>   {
>     k_ipad[i] ^= 0x36;
>     k_opad[i] ^= 0x5c;
>   }
>   // perform inner MD5
>   md5.reset();
>   md5.update(k_ipad);
>   md5.update(text);
>   digest = md5.digest();
>   // perform outer MD5
>   //md5.reset();
>   md5.update(k_opad);
>   md5.update(digest);
>   digest = md5.digest();
>   //return digest;
> 
>   StringBuffer stringbuffer = new StringBuffer();
> 
>   for (int k = 0; k < digest.length; k++) {
>     if ((digest[k] & 0xff) < 16) {
>       stringbuffer.append("0" + Integer.toHexString(digest[k] & 0xff));
>     }
>     else {
>       stringbuffer.append(Integer.toHexString(digest[k] & 0xff));
>     }
>   }
> 
>   return stringbuffer.toString();
> }

These methods are obsolete, since current GNU-Crypto provides a better,
more modular system, and I don't want to regress.

You need a very up-to-date version of GNU-Crypto, like current CVS HEAD.
It shouldn't require any configuration as such, you simply ask for a
SASL provider by name (the javamail providers ask for the first
authentication mechanism in the capabilities list).

I notice GNU-Crypto has made another release in the meantime. If it's
failing now we should try to correct it there.

> **** SMTP ****
> 
> 5. I made similar changes to gnu.inet.smtp.SMTPConnection to get 
> CRAM-MD5 to work with Postfix.

See above.

> 6. I believe there is a bug in gnu.mail.providers.smtp.SMTPTransport. 
> I registered a javax.mail.event.TransportListener on the SMTPTransport 
> and when the debug output showed a 250 (success) response from sending 
> an email the messageNotDelivered() callback method was invoked.  I 
> traced the error to the sendMessage(Message message, Address[] 
> addresses) method.
> 
> 
>         // DATA
>         OutputStream dataStream = connection.data();
>         if (dataStream==null)
>           throw new MessagingException(connection.getLastResponse());
>         mimeMessage.writeTo(dataStream);
>         dataStream.flush();
>         if (connection.finishData())  <- no bang
>         {
> 
> should be
> 
>         // DATA
>         OutputStream dataStream = connection.data();
>         if (dataStream==null)
>           throw new MessagingException(connection.getLastResponse());
>         mimeMessage.writeTo(dataStream);
>         dataStream.flush();
>         if (!connection.finishData()) <- negated
>         {
> 
> I negated the if and received the appropriate callback.  I'm not sure if 
> this addresses the problem or the symptom.

No, you're right - thanks for spotting this. It's now fixed in javamail
HEAD.
-- 
Chris Burdess