[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[cp-patches] RFC: [Patch] rewrite of classpath/gnu/java/net/protocol/htt
|
From: |
David Daney |
|
Subject: |
[cp-patches] RFC: [Patch] rewrite of classpath/gnu/java/net/protocol/http/* |
|
Date: |
Mon, 12 Sep 2005 14:05:37 -0700 |
|
User-agent: |
Mozilla Thunderbird 1.0.6-1.1.fc3 (X11/20050720) |
As I threatened last week, I made some major changes to the
gnu.java.net.protocol.http package. The patch is probably not in its
final form, but I thought I would post it to see what the reaction to it
is. At a minimum I would update the copyright dates.
The current version reads the entire body of a HTTP response in to a
memory buffer (ByteArrayResponseBodyReader) and if user code needs the
content it gets a ByteArrayInputStream wrapped around the buffer.
There are several problems with this approach The main one is that if
the response is larger than one half of the heap limit (a copy of the
data is made so you need space for two copies) You get an
OutOfMemoryError. And related is that if the response is larger than
Integer.MAX_VALUE you run into the java language limit on array sizes.
A secondary problem that I noticed (but did not fully analyze) is that
the connection pool and keep-alive logic is not thread safe and had some
other logic bugs in it.
Here is an outline of what I did:
Gratuitously remove RequestListener support. Perhaps it is used by a
third party package, I did not bother to check or investigate. If
needed, I suppose it could be restored.
Removed ResponseBodyReader and ByteArrayResponseBodyReader. Since we
are not going to buffer anything anymore these are no longer needed.
The core to the rewrite is the new LimitedLengthInputStream class. It
serves a couple of purposes. First it controls/protects access to the
underlying socket's InputStream so that it does not get closed too soon
or have too much data read from it. The second thing it does is to help
manage the connection pool as described below.
When a connection is made, a LimitedLengthInputStream is attached to the
socket's InputStream ahead of any other encoding filters that might be
needed. This prevents actions by the users code from harming the socket
in any way that would make it unusable for re-use in a keep-alive situation.
When all the data has been been read from the body of the response or if
the InputStream is closed, then the connection is returned to the
connection pool.
With this patch I can successfully transfer (via HTTP) content that is
much larger than my heap.
Some testing done with GCJ (HEAD) on i686-pc-linux-gnu as well as
mipsel-linux. More testing is probably needed.
2005-09-12 David Daney <address@hidden>
* classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java:
Removed.
* classpath/gnu/java/net/protocol/http/ResponseBodyReader.java:
Removed.
* classpath/gnu/java/net/protocol/http/HTTPConnection.java
(requestListeners): Removed.
(pool): New field.
(Constructor): Don't initialize requestListeners.
(getPoolKey): New method.
(setPool): New method.
(release): New method.
(newRequest): Don't call fireRequestEvent.
(addRequestListener): Removed.
(fireRequestEvent): Removed.
* classpath/gnu/java/net/protocol/http/HTTPURLConnection.java
(java.util.ByteArrayInputStream): Don't import.
(connectionPool): Changed type to LinkedHashMap.
(maxConnections): Made static.
(responseSink): Changed type to InputStream.
(errorSink): Likewise.
(connect): Eliminate reader and get responseSink from response.
(getConnection): Rewrote.
* classpath/gnu/java/net/protocol/http/Headers.java (getLongValue): New
method.
* classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java:
New class.
* classpath/gnu/java/net/protocol/http/Request.java
(responseBodyReader): Removed.
(setResponseBodyReader): Removed.
(dispatch): Don't call fireRequestEvent. Handle unsolicited 100
Continue response.
(readResponse): Rewrote.
(readResponseBody): Rewrote.
* classpath/gnu/java/net/protocol/http/Response.java
(java.io.InputStream): Import it.
(codeClass): Removed.
(Constructor): Don't initialize codeClass. Initialize body.
(body): New field.
(getCodeClass): Rewrote.
(getLongHeader): New method.
(getBody): New nethod.
Comments?
David Daney
Index: classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java
===================================================================
RCS file: classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java
diff -N classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java
*** classpath/gnu/java/net/protocol/http/ByteArrayResponseBodyReader.java
16 Jul 2005 00:33:57 -0000 1.1.1.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
***************
*** 1,123 ****
- /* Authenticator.java --ByteArrayResponseBodyReader.java --
- Copyright (C) 2004 Free Software Foundation, Inc.
-
- This file is part of GNU Classpath.
-
- GNU Classpath is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2, or (at your option)
- any later version.
-
- GNU Classpath is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with GNU Classpath; see the file COPYING. If not, write to the
- Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- 02110-1301 USA.
-
- Linking this library statically or dynamically with other modules is
- making a combined work based on this library. Thus, the terms and
- conditions of the GNU General Public License cover the whole
- combination.
-
- As a special exception, the copyright holders of this library give you
- permission to link this library with independent modules to produce an
- executable, regardless of the license terms of these independent
- modules, and to copy and distribute the resulting executable under
- terms of your choice, provided that you also meet, for each linked
- independent module, the terms and conditions of the license of that
- module. An independent module is a module which is not derived from
- or based on this library. If you modify this library, you may extend
- this exception to your version of the library, but you are not
- obligated to do so. If you do not wish to do so, delete this
- exception statement from your version. */
-
-
- package gnu.java.net.protocol.http;
-
- /**
- * Simple response body reader that stores content in a byte array.
- *
- * @author Chris Burdess (address@hidden)
- */
- public class ByteArrayResponseBodyReader
- implements ResponseBodyReader
- {
-
- /**
- * The content.
- */
- protected byte[] content;
-
- /**
- * The position in the content at which the next write will occur.
- */
- protected int pos;
-
- /**
- * The length of the buffer.
- */
- protected int len;
-
- /**
- * Constructs a new byte array response body reader.
- */
- public ByteArrayResponseBodyReader()
- {
- this(4096);
- }
-
- /**
- * Constructs a new byte array response body reader with the specified
- * initial buffer size.
- * @param size the initial buffer size
- */
- public ByteArrayResponseBodyReader(int size)
- {
- content = new byte[size];
- pos = len = 0;
- }
-
- /**
- * This reader accepts all responses.
- */
- public boolean accept(Request request, Response response)
- {
- return true;
- }
-
- public void read(byte[] buffer, int offset, int length)
- {
- int l = length - offset;
- if (pos + l > content.length)
- {
- byte[] tmp = new byte[content.length * 2];
- System.arraycopy(content, 0, tmp, 0, pos);
- content = tmp;
- }
- System.arraycopy(buffer, offset, content, pos, l);
- pos += l;
- len = pos;
- }
-
- public void close()
- {
- pos = 0;
- }
-
- /**
- * Retrieves the content of this reader as a byte array.
- * The size of the returned array is the number of bytes read.
- */
- public byte[] toByteArray()
- {
- byte[] ret = new byte[len];
- System.arraycopy(content, 0, ret, 0, len);
- return ret;
- }
-
- }
-
--- 0 ----
Index: classpath/gnu/java/net/protocol/http/HTTPConnection.java
===================================================================
RCS file:
/cvs/gcc/gcc/libjava/classpath/gnu/java/net/protocol/http/HTTPConnection.java,v
retrieving revision 1.1.1.1
diff -c -p -r1.1.1.1 HTTPConnection.java
*** classpath/gnu/java/net/protocol/http/HTTPConnection.java 16 Jul 2005
00:33:57 -0000 1.1.1.1
--- classpath/gnu/java/net/protocol/http/HTTPConnection.java 12 Sep 2005
19:43:58 -0000
*************** import java.security.GeneralSecurityExce
*** 57,62 ****
--- 57,63 ----
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
+ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
*************** public class HTTPConnection
*** 132,138 ****
protected int minorVersion;
private final List connectionListeners;
- private final List requestListeners;
private final List handshakeCompletedListeners;
/**
--- 133,138 ----
*************** public class HTTPConnection
*** 165,170 ****
--- 165,176 ----
*/
protected CookieManager cookieManager;
+
+ /**
+ * The pool that this connection is a member of (if any).
+ */
+ private LinkedHashMap pool;
+
/**
* Creates a new HTTP connection.
* @param hostname the name of the host to connect to
*************** public class HTTPConnection
*** 237,243 ****
this.timeout = timeout;
majorVersion = minorVersion = 1;
connectionListeners = new ArrayList(4);
- requestListeners = new ArrayList(4);
handshakeCompletedListeners = new ArrayList(2);
}
--- 243,248 ----
*************** public class HTTPConnection
*** 331,336 ****
--- 336,381 ----
return cookieManager;
}
+ static Object getPoolKey(String h, int p, boolean sec)
+ {
+ StringBuffer buf = new StringBuffer(sec ? "https://"; : "http://";);
+ buf.append(h);
+ buf.append(':');
+ buf.append(p);
+ return buf.toString();
+ }
+
+ void setPool(LinkedHashMap p)
+ {
+ pool = p;
+ }
+
+ void release()
+ {
+ if (pool != null)
+ {
+ synchronized (pool)
+ {
+ Object key = HTTPConnection.getPoolKey(hostname, port, secure);
+ pool.put(key, this);
+ while (pool.size() >= HTTPURLConnection.maxConnections)
+ {
+ // maxConnections must always be >= 1
+ Object lru = pool.keySet().iterator().next();
+ HTTPConnection c = (HTTPConnection)pool.remove(lru);
+ try
+ {
+ c.closeConnection();
+ }
+ catch (IOException ioe)
+ {
+ // Ignore it. We are just cleaning up.
+ }
+ }
+ }
+ }
+ }
+
/**
* Creates a new request using this connection.
* @param method the HTTP method to invoke
*************** public class HTTPConnection
*** 378,384 ****
ret.setHeader("Cookie", buf.toString());
}
}
- fireRequestEvent(RequestEvent.REQUEST_CREATED, ret);
return ret;
}
--- 423,428 ----
*************** public class HTTPConnection
*** 620,667 ****
}
}
- public void addRequestListener(RequestListener l)
- {
- synchronized (requestListeners)
- {
- requestListeners.add(l);
- }
- }
-
- public void removeRequestListener(RequestListener l)
- {
- synchronized (requestListeners)
- {
- requestListeners.remove(l);
- }
- }
-
- protected void fireRequestEvent(int type, Request request)
- {
- RequestEvent event = new RequestEvent(this, type, request);
- RequestListener[] l = null;
- synchronized (requestListeners)
- {
- l = new RequestListener[requestListeners.size()];
- requestListeners.toArray(l);
- }
- for (int i = 0; i < l.length; i++)
- {
- switch (type)
- {
- case RequestEvent.REQUEST_CREATED:
- l[i].requestCreated(event);
- break;
- case RequestEvent.REQUEST_SENDING:
- l[i].requestSent(event);
- break;
- case RequestEvent.REQUEST_SENT:
- l[i].requestSent(event);
- break;
- }
- }
- }
-
void addHandshakeCompletedListener(HandshakeCompletedListener l)
{
synchronized (handshakeCompletedListeners)
--- 664,669 ----
Index: classpath/gnu/java/net/protocol/http/HTTPURLConnection.java
===================================================================
RCS file:
/cvs/gcc/gcc/libjava/classpath/gnu/java/net/protocol/http/HTTPURLConnection.java,v
retrieving revision 1.1.1.1
diff -c -p -r1.1.1.1 HTTPURLConnection.java
*** classpath/gnu/java/net/protocol/http/HTTPURLConnection.java 16 Jul 2005
00:33:57 -0000 1.1.1.1
--- classpath/gnu/java/net/protocol/http/HTTPURLConnection.java 12 Sep 2005
19:43:58 -0000
*************** exception statement from your version. *
*** 37,44 ****
package gnu.java.net.protocol.http;
-
- import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
--- 37,42 ----
*************** public class HTTPURLConnection
*** 75,81 ****
/**
* Pool of reusable connections, used if keepAlive is true.
*/
! private static final Map connectionPool = new LinkedHashMap();
/*
* The underlying connection.
--- 73,80 ----
/**
* Pool of reusable connections, used if keepAlive is true.
*/
! private static final LinkedHashMap connectionPool = new LinkedHashMap();
! static int maxConnections;
/*
* The underlying connection.
*************** public class HTTPURLConnection
*** 87,93 ****
int proxyPort;
String agent;
boolean keepAlive;
- int maxConnections;
private Request request;
private Headers requestHeaders;
--- 86,91 ----
*************** public class HTTPURLConnection
*** 95,102 ****
private boolean requestMethodSetExplicitly;
private Response response;
! private ByteArrayInputStream responseSink;
! private ByteArrayInputStream errorSink;
private HandshakeCompletedEvent handshakeEvent;
--- 93,100 ----
private boolean requestMethodSetExplicitly;
private Response response;
! private InputStream responseSink;
! private InputStream errorSink;
private HandshakeCompletedEvent handshakeEvent;
*************** public class HTTPURLConnection
*** 133,143 ****
--- 131,144 ----
}
}
agent = System.getProperty("http.agent");
+
String ka = System.getProperty("http.keepAlive");
keepAlive = !(ka != null && "false".equals(ka));
+
String mc = System.getProperty("http.maxConnections");
maxConnections = (mc != null && mc.length() > 0) ?
Math.max(Integer.parseInt(mc), 1) : 5;
+
return null;
}
*************** public class HTTPURLConnection
*** 218,225 ****
RequestBodyWriter writer = new
ByteArrayRequestBodyWriter(content);
request.setRequestBodyWriter(writer);
}
- ByteArrayResponseBodyReader reader = new
ByteArrayResponseBodyReader();
- request.setResponseBodyReader(reader);
if (creds != null)
{
request.setAuthenticator(new Authenticator() {
--- 219,224 ----
*************** public class HTTPURLConnection
*** 307,313 ****
}
else
{
! responseSink = new ByteArrayInputStream(reader.toByteArray ());
if (response.getCode() == 404)
{
errorSink = responseSink;
--- 306,312 ----
}
else
{
! responseSink = response.getBody();
if (response.getCode() == 404)
{
errorSink = responseSink;
*************** public class HTTPURLConnection
*** 328,354 ****
HTTPConnection connection;
if (keepAlive)
{
! StringBuffer buf = new StringBuffer(secure ? "https://"; : "http://";);
! buf.append(Thread.currentThread().hashCode());
! buf.append('@');
! buf.append(host);
! buf.append(':');
! buf.append(port);
! String key = buf.toString();
synchronized (connectionPool)
{
! connection = (HTTPConnection) connectionPool.get(key);
if (connection == null)
{
connection = new HTTPConnection(host, port, secure);
! // Good housekeeping
! if (connectionPool.size() == maxConnections)
! {
! // maxConnections must always be >= 1
! Object lru = connectionPool.keySet().iterator().next();
! connectionPool.remove(lru);
! }
! connectionPool.put(key, connection);
}
}
}
--- 327,340 ----
HTTPConnection connection;
if (keepAlive)
{
! Object key = HTTPConnection.getPoolKey(host, port, secure);
synchronized (connectionPool)
{
! connection = (HTTPConnection) connectionPool.remove(key);
if (connection == null)
{
connection = new HTTPConnection(host, port, secure);
! connection.setPool(connectionPool);
}
}
}
Index: classpath/gnu/java/net/protocol/http/Headers.java
===================================================================
RCS file:
/cvs/gcc/gcc/libjava/classpath/gnu/java/net/protocol/http/Headers.java,v
retrieving revision 1.1.1.1
diff -c -p -r1.1.1.1 Headers.java
*** classpath/gnu/java/net/protocol/http/Headers.java 16 Jul 2005 00:33:57
-0000 1.1.1.1
--- classpath/gnu/java/net/protocol/http/Headers.java 12 Sep 2005 19:43:58
-0000
*************** public class Headers
*** 205,210 ****
--- 205,231 ----
}
/**
+ * Returns the value of the specified header as an integer,
+ * or -1 if the header is not present or not an integer.
+ */
+ public long getLongValue(String header)
+ {
+ String val = getValue(header);
+ if (val == null)
+ {
+ return -1;
+ }
+ try
+ {
+ return Long.parseLong(val);
+ }
+ catch (NumberFormatException e)
+ {
+ }
+ return -1;
+ }
+
+ /**
* Returns the value of the specified header as a date,
* or <code>null</code> if the header is not present or not a date.
*/
Index: classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java
===================================================================
RCS file: classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java
diff -N classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java
*** /dev/null 1 Jan 1970 00:00:00 -0000
--- classpath/gnu/java/net/protocol/http/LimitedLengthInputStream.java 12 Sep
2005 19:43:58 -0000
***************
*** 0 ****
--- 1,227 ----
+ /* LimitedLengthInputStream.java --
+ Copyright (C) 2005 Free Software Foundation, Inc.
+
+ This file is part of GNU Classpath.
+
+ GNU Classpath is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2, or (at your option)
+ any later version.
+
+ GNU Classpath is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GNU Classpath; see the file COPYING. If not, write to the
+ Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ 02110-1301 USA.
+
+ Linking this library statically or dynamically with other modules is
+ making a combined work based on this library. Thus, the terms and
+ conditions of the GNU General Public License cover the whole
+ combination.
+
+ As a special exception, the copyright holders of this library give you
+ permission to link this library with independent modules to produce an
+ executable, regardless of the license terms of these independent
+ modules, and to copy and distribute the resulting executable under
+ terms of your choice, provided that you also meet, for each linked
+ independent module, the terms and conditions of the license of that
+ module. An independent module is a module which is not derived from
+ or based on this library. If you modify this library, you may extend
+ this exception to your version of the library, but you are not
+ obligated to do so. If you do not wish to do so, delete this
+ exception statement from your version. */
+
+
+ package gnu.java.net.protocol.http;
+
+ import java.io.IOException;
+ import java.io.InputStream;
+
+ /**
+ * InputStream that limits the total number of bytes that can be read
+ * from an underlying stream. In addition to limiting the number of
+ * bytes read, close() is not propagated to the underlying stream.
+ *
+ * @author David Daney (address@hidden)
+ */
+ class LimitedLengthInputStream
+ extends InputStream
+ {
+ private long remainingLen;
+ private boolean restrictLen;
+ private HTTPConnection connection;
+ private boolean eof;
+ private InputStream in;
+ private boolean doClose;
+
+
+ private void handleClose()
+ throws IOException
+ {
+ eof = true;
+ if (doClose)
+ {
+ in.close();
+ }
+ else
+ {
+ connection.release();
+ }
+ in = null;
+ connection = null;
+ }
+
+ /**
+ * Constructor.
+ * @param in the underlying stream
+ * @param maxLen the maximum number of bytes to read
+ * @param con the HTTPConnection associated with this stream
+ * @param doClose if true con will be closed when finished reading,
+ * else it will be placed back in the connection pool.
+ */
+ LimitedLengthInputStream(InputStream in,
+ long maxLen,
+ boolean restrictLen,
+ HTTPConnection con,
+ boolean doClose)
+ throws IOException
+
+ {
+ this.in = in;
+ this.remainingLen = maxLen;
+ this.restrictLen = restrictLen;
+ this.connection = con;
+ this.doClose = doClose;
+
+ if (restrictLen)
+ {
+ if (maxLen < 0)
+ throw new IllegalArgumentException();
+ else if (maxLen == 0)
+ handleClose(); // Nothing to do, release the connection.
+ }
+ }
+
+ public synchronized int read()
+ throws IOException
+ {
+ if (eof)
+ return -1; // EOF
+
+ int r;
+
+ if (restrictLen)
+ {
+ r = in.read();
+ if (-1 != r)
+ remainingLen--;
+
+ if (0 == remainingLen)
+ handleClose();
+ }
+ else
+ {
+ r = in.read();
+ }
+
+ return r;
+ }
+
+ public int read(byte[] buffer)
+ throws IOException
+ {
+ return read(buffer, 0, buffer.length);
+ }
+
+ public synchronized int read(byte[] buffer, int offset, int length)
+ throws IOException
+ {
+ if (eof)
+ return -1; // EOF
+
+ if (restrictLen && length > remainingLen)
+ length = (int) remainingLen;
+
+ int r = in.read(buffer, offset, length);
+
+ if (-1 == r)
+ handleClose();
+
+ if (restrictLen && r > 0)
+ {
+ remainingLen -= r;
+ if (0 == remainingLen)
+ handleClose();
+ }
+ return r;
+ }
+
+ public synchronized long skip(long n)
+ throws IOException
+ {
+
+ if (eof)
+ return 0;
+
+ if (restrictLen && n > remainingLen)
+ n = remainingLen;
+
+ long r = in.skip(n);
+
+ if (restrictLen)
+ {
+ remainingLen -= r;
+ if (0 == remainingLen)
+ handleClose();
+ }
+ return r;
+ }
+
+ public synchronized int available()
+ throws IOException
+ {
+ if (eof)
+ return 0;
+
+ int a = in.available();
+ if (restrictLen && a > remainingLen)
+ a = (int)remainingLen;
+ return a;
+ }
+
+ public synchronized void close()
+ throws IOException
+ {
+ if (eof)
+ return;
+
+ try
+ {
+ while (restrictLen && remainingLen > 0)
+ {
+ long r = in.skip(remainingLen);
+ remainingLen -= r;
+ if (r == 0)
+ doClose = true; // Something is broken, close the stream.
+ }
+ }
+ catch (IOException ioe)
+ {
+ // Something is broken, close the stream.
+ doClose = true;
+ }
+
+ handleClose();
+ }
+
+ protected void finalize()
+ throws Throwable
+ {
+ close();
+ }
+
+ }
Index: classpath/gnu/java/net/protocol/http/Request.java
===================================================================
RCS file:
/cvs/gcc/gcc/libjava/classpath/gnu/java/net/protocol/http/Request.java,v
retrieving revision 1.1.1.1
diff -c -p -r1.1.1.1 Request.java
*** classpath/gnu/java/net/protocol/http/Request.java 16 Jul 2005 00:33:57
-0000 1.1.1.1
--- classpath/gnu/java/net/protocol/http/Request.java 12 Sep 2005 19:43:58
-0000
*************** public class Request
*** 100,110 ****
protected int requestBodyNegotiationThreshold;
/**
- * The response body reader.
- */
- protected ResponseBodyReader responseBodyReader;
-
- /**
* Map of response header handlers.
*/
protected Map responseHeaderHandlers;
--- 100,105 ----
*************** public class Request
*** 236,251 ****
}
/**
- * Sets the response body reader.
- * @param responseBodyReader the handler to receive notifications of
- * response body content
- */
- public void setResponseBodyReader(ResponseBodyReader responseBodyReader)
- {
- this.responseBodyReader = responseBodyReader;
- }
-
- /**
* Sets a callback handler to be invoked for the specified header name.
* @param name the header name
* @param handler the handler to receive the value for the header
--- 231,236 ----
*************** public class Request
*** 324,331 ****
do
{
retry = false;
- // Send request
- connection.fireRequestEvent(RequestEvent.REQUEST_SENDING, this);
// Get socket output and input streams
OutputStream out = connection.getOutputStream();
--- 309,314 ----
*************** public class Request
*** 372,396 ****
out.write(CRLF.getBytes(US_ASCII));
}
out.flush();
- // Sent event
- connection.fireRequestEvent(RequestEvent.REQUEST_SENT, this);
// Get response
! response = readResponse(in);
! int sc = response.getCode();
! if (sc == 401 && authenticator != null)
! {
! if (authenticate(response, attempts++))
! {
! retry = true;
! }
! }
! else if (sc == 100 && expectingContinue)
! {
! requestHeaders.remove("Expect");
! setHeader("Content-Length", Integer.toString(contentLength));
! expectingContinue = false;
! retry = true;
! }
}
while (retry);
}
--- 355,394 ----
out.write(CRLF.getBytes(US_ASCII));
}
out.flush();
// Get response
! while(true)
! {
! response = readResponse(in);
! int sc = response.getCode();
! if (sc == 401 && authenticator != null)
! {
! if (authenticate(response, attempts++))
! {
! retry = true;
! }
! }
! else if (sc == 100)
! {
! if (expectingContinue)
! {
! requestHeaders.remove("Expect");
! setHeader("Content-Length",
! Integer.toString(contentLength));
! expectingContinue = false;
! retry = true;
! }
! else
! {
! // A conforming server can send an unsoliceted
! // Continue response but should not (RFC 2616
! // sec 8.2.3). Ignore the bogus Continue
! // response and get the real response that
! // should follow
! continue;
! }
! }
! break;
! }
}
while (retry);
}
*************** public class Request
*** 440,467 ****
Headers responseHeaders = new Headers();
responseHeaders.parse(in);
notifyHeaderHandlers(responseHeaders);
! // Construct response
! int codeClass = code / 100;
! Response ret = new Response(majorVersion, minorVersion, code,
! codeClass, message, responseHeaders);
switch (code)
{
case 204:
case 205:
case 304:
break;
default:
! // Does response body reader want body?
! boolean notify = (responseBodyReader != null);
! if (notify)
! {
! if (!responseBodyReader.accept(this, ret))
! {
! notify = false;
! }
! }
! readResponseBody(ret, in, notify);
}
return ret;
}
--- 438,460 ----
Headers responseHeaders = new Headers();
responseHeaders.parse(in);
notifyHeaderHandlers(responseHeaders);
! InputStream body = null;
!
switch (code)
{
+ case 100:
case 204:
case 205:
case 304:
break;
default:
! body = readResponseBody(responseHeaders, majorVersion,
! minorVersion, in);
}
+
+ // Construct response
+ Response ret = new Response(majorVersion, minorVersion, code,
+ message, responseHeaders, body);
return ret;
}
*************** public class Request
*** 487,511 ****
}
}
! void readResponseBody(Response response, InputStream in,
! boolean notify)
throws IOException
{
! byte[] buffer = new byte[4096];
! int contentLength = -1;
Headers trailer = null;
! String transferCoding = response.getHeader("Transfer-Encoding");
if ("chunked".equalsIgnoreCase(transferCoding))
{
! trailer = new Headers();
! in = new ChunkedInputStream(in, trailer);
}
else
{
! contentLength = response.getIntHeader("Content-Length");
}
! String contentCoding = response.getHeader("Content-Encoding");
if (contentCoding != null && !"identity".equals(contentCoding))
{
if ("gzip".equals(contentCoding))
--- 480,514 ----
}
}
! private InputStream readResponseBody(Headers responseHeaders,
! int majorVersion,
! int minorVersion,
! InputStream in)
throws IOException
{
! long contentLength = -1;
Headers trailer = null;
! // Persistent connections are the default in HTTP/1.1
! boolean doClose = "close".equalsIgnoreCase(getHeader("Connection")) ||
! "close".equalsIgnoreCase(responseHeaders.getValue("Connection")) ||
! (connection.majorVersion == 1 && connection.minorVersion == 0) ||
! (majorVersion == 1 && minorVersion == 0);
!
! String transferCoding = responseHeaders.getValue("Transfer-Encoding");
if ("chunked".equalsIgnoreCase(transferCoding))
{
! in = new LimitedLengthInputStream(in, -1, false, connection, doClose);
!
! in = new ChunkedInputStream(in, responseHeaders);
}
else
{
! contentLength = responseHeaders.getLongValue("Content-Length");
! in = new LimitedLengthInputStream(in, contentLength, true,
! connection, doClose);
}
! String contentCoding = responseHeaders.getValue("Content-Encoding");
if (contentCoding != null && !"identity".equals(contentCoding))
{
if ("gzip".equals(contentCoding))
*************** public class Request
*** 522,572 ****
contentCoding);
}
}
!
! // Persistent connections are the default in HTTP/1.1
! boolean doClose = "close".equalsIgnoreCase(getHeader("Connection")) ||
! "close".equalsIgnoreCase(response.getHeader("Connection")) ||
! (connection.majorVersion == 1 && connection.minorVersion == 0) ||
! (response.majorVersion == 1 && response.minorVersion == 0);
!
! int count = contentLength;
! int len = (count > -1) ? count : buffer.length;
! len = (len > buffer.length) ? buffer.length : len;
! while (len > -1)
! {
! len = in.read(buffer, 0, len);
! if (len < 0)
! {
! // EOF
! connection.closeConnection();
! break;
! }
! if (notify)
! {
! responseBodyReader.read(buffer, 0, len);
! }
! if (count > -1)
! {
! count -= len;
! if (count < 1)
! {
! if (doClose)
! {
! connection.closeConnection();
! }
! break;
! }
! }
! }
! if (notify)
! {
! responseBodyReader.close();
! }
! if (trailer != null)
! {
! response.getHeaders().putAll(trailer);
! notifyHeaderHandlers(trailer);
! }
}
boolean authenticate(Response response, int attempts)
--- 525,531 ----
contentCoding);
}
}
! return in;
}
boolean authenticate(Response response, int attempts)
Index: classpath/gnu/java/net/protocol/http/Response.java
===================================================================
RCS file:
/cvs/gcc/gcc/libjava/classpath/gnu/java/net/protocol/http/Response.java,v
retrieving revision 1.1.1.1
diff -c -p -r1.1.1.1 Response.java
*** classpath/gnu/java/net/protocol/http/Response.java 16 Jul 2005 00:33:57
-0000 1.1.1.1
--- classpath/gnu/java/net/protocol/http/Response.java 12 Sep 2005 19:43:58
-0000
*************** exception statement from your version. *
*** 38,43 ****
--- 38,44 ----
package gnu.java.net.protocol.http;
+ import java.io.InputStream;
import java.util.Date;
/**
*************** public class Response
*** 64,82 ****
protected final int code;
/**
- * The class of the response. This is the most significant digit of the
- * status code.
- * <dl>
- * <dt><code>1xx</code></dt> <dd>Informational response</dd>
- * <dt><code>2xx</code></dt> <dd>Success</dd>
- * <dt><code>3xx</code></dt> <dd>Redirection</dd>
- * <dt><code>4xx</code></dt> <dd>Client error</dd>
- * <dt><code>5xx</code></dt> <dd>Server error</dd>
- * </dl>
- */
- protected final int codeClass;
-
- /**
* Human-readable text of the response.
*/
protected final String message;
--- 65,70 ----
*************** public class Response
*** 87,104 ****
protected final Headers headers;
/**
* Constructs a new response with the specified parameters.
*/
protected Response(int majorVersion, int minorVersion, int code,
! int codeClass, String message,
! Headers headers)
{
this.majorVersion = majorVersion;
this.minorVersion = minorVersion;
this.code = code;
- this.codeClass = codeClass;
this.message = message;
this.headers = headers;
}
/**
--- 75,96 ----
protected final Headers headers;
/**
+ * An InputStream that returns the body of the response.
+ */
+ protected final InputStream body;
+
+ /**
* Constructs a new response with the specified parameters.
*/
protected Response(int majorVersion, int minorVersion, int code,
! String message, Headers headers, InputStream body)
{
this.majorVersion = majorVersion;
this.minorVersion = minorVersion;
this.code = code;
this.message = message;
this.headers = headers;
+ this.body = body;
}
/**
*************** public class Response
*** 129,140 ****
}
/**
! * Returns the class of the response.
! * @see #codeClass
*/
public int getCodeClass()
{
! return codeClass;
}
/**
--- 121,139 ----
}
/**
! * Returns the class of the response. This is the most significant
! * digit of the status code.
! * <dl>
! * <dt><code>1xx</code></dt> <dd>Informational response</dd>
! * <dt><code>2xx</code></dt> <dd>Success</dd>
! * <dt><code>3xx</code></dt> <dd>Redirection</dd>
! * <dt><code>4xx</code></dt> <dd>Client error</dd>
! * <dt><code>5xx</code></dt> <dd>Server error</dd>
! * </dl>
*/
public int getCodeClass()
{
! return code / 100;
}
/**
*************** public class Response
*** 173,178 ****
--- 172,186 ----
}
/**
+ * Returns the header value for the specified name as a long.
+ * @param name the header name
+ */
+ public long getLongHeader(String name)
+ {
+ return headers.getLongValue(name);
+ }
+
+ /**
* Returns the header value for the specified name as a date.
* @param name the header name
*/
*************** public class Response
*** 181,185 ****
--- 189,202 ----
return headers.getDateValue(name);
}
+ /**
+ * Returns an InputStream that returns the body of the response.
+ *
+ * @return the body of the response
+ */
+ public InputStream getBody()
+ {
+ return body;
+ }
}
Index: classpath/gnu/java/net/protocol/http/ResponseBodyReader.java
===================================================================
RCS file: classpath/gnu/java/net/protocol/http/ResponseBodyReader.java
diff -N classpath/gnu/java/net/protocol/http/ResponseBodyReader.java
*** classpath/gnu/java/net/protocol/http/ResponseBodyReader.java 16 Jul
2005 00:33:57 -0000 1.1.1.1
--- /dev/null 1 Jan 1970 00:00:00 -0000
***************
*** 1,70 ****
- /* ResponseBodyReader.java --
- Copyright (C) 2004 Free Software Foundation, Inc.
-
- This file is part of GNU Classpath.
-
- GNU Classpath is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation; either version 2, or (at your option)
- any later version.
-
- GNU Classpath is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with GNU Classpath; see the file COPYING. If not, write to the
- Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
- 02110-1301 USA.
-
- Linking this library statically or dynamically with other modules is
- making a combined work based on this library. Thus, the terms and
- conditions of the GNU General Public License cover the whole
- combination.
-
- As a special exception, the copyright holders of this library give you
- permission to link this library with independent modules to produce an
- executable, regardless of the license terms of these independent
- modules, and to copy and distribute the resulting executable under
- terms of your choice, provided that you also meet, for each linked
- independent module, the terms and conditions of the license of that
- module. An independent module is a module which is not derived from
- or based on this library. If you modify this library, you may extend
- this exception to your version of the library, but you are not
- obligated to do so. If you do not wish to do so, delete this
- exception statement from your version. */
-
-
- package gnu.java.net.protocol.http;
-
- /**
- * Callback interface for receiving notification of response body content.
- *
- * @author Chris Burdess (address@hidden)
- */
- public interface ResponseBodyReader
- {
-
- /**
- * Indicate whether this reader is interested in the specified response.
- * If it returns false, it will not receive body content notifications for
- * that response.
- */
- boolean accept(Request request, Response response);
-
- /**
- * Receive notification of body content.
- * @param buffer the content buffer
- * @param offset the offset within the buffer that content starts
- * @param length the length of the content
- */
- void read(byte[] buffer, int offset, int length);
-
- /**
- * Notifies the reader that the end of the content was reached.
- */
- void close();
-
- }
-
--- 0 ----
- [cp-patches] RFC: [Patch] rewrite of classpath/gnu/java/net/protocol/http/*,
David Daney <=