Strange Socket problem

I'm having a problem in some production code that I can't figure out.
I'll post the complete actual code below. This code is running in three
places and has the same problem in two of them at the same time. The
other I'm not sure, it may be that the personnel operating it are
restarting the program and so don't complain. This piece of code is a
simple client that connects via a Socket to a server. The server
supplies some data and the client reads that data and files it away. It
is supposed to restart itself if there is a connection failure or fault
for whatever reason. The problem is that at some random point in time
the Socket disconnects, the code logs the disconnect but never restarts.
It does print the "SportsWinClient Disconnected" message but never
executes the "fireconnectionEvent()" method after creating a new Socket.
It doesn't print any Exception message. I'm not sure how it gets out
of the try block without printing the "End of Stream" message or an
exception message.

The crazy part is that all night long when there is no activity from the
server it times out and restarts with no problems.

I'm hoping that somebody will see a fault in my code that could cause
the failure. It is not a compile problem so I left the formatting as it is.

Thanks for looking.

package com.knutejohnson.xyzcasinos.translux;

import java.io.*;
import java.net.*;
import java.util.*;

import com.knutejohnson.classes.*;

import static com.knutejohnson.xyzcasinos.translux.Constants.*;

public class SportsWinClient implements Runnable {
private final Thread thread;

private volatile boolean isConnected;
private volatile boolean runFlag = true;

private volatile Socket socket;

public SportsWinClient() {
thread = new Thread(this,"SportsWinClient");
}

public void start() {
thread.start();
}

public void run() {
// boolean serverFlag = true;

System.out.println("SportsWinClient: Started");
while (runFlag) {
// String serverAddress = serverFlag ? SPORTS_WIN_IP_PRIMARY :
// SPORTS_WIN_IP_SECONDARY;
try {
// socket = new Socket(serverAddress,SPORTS_WIN_PORT,
socket = new Socket(SPORTS_WIN_IP,SPORTS_WIN_PORT,
InetAddress.getByName(REMOTE_IP),0);
socket.setKeepAlive(true);
isConnected = true;

********* I know that the line below is not being executed **********

fireConnectionEvent(ConnectionEvent.CONNECTED);
socket.setSoTimeout(3600000); // one hour timeout
System.out.println("SportsWinClient: Connected");
InputStream is = socket.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);

String str;
while ((str = br.readLine()) != null) {
if (!str.matches("\\d+.*")) // not a sports record
continue;
SportsBet sb = new SportsBet(str);
SPORTS_BET_MAP.put(sb.betNumber,sb);
}

System.out.println("SportsWinClient: End of Stream");
} catch (IOException ioe) {
System.out.println("SportsWinClient: " + ioe.toString());
} finally {
isConnected = false;
if (socket != null)
try {
socket.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
fireConnectionEvent(ConnectionEvent.DISCONNECTED);
// serverFlag = !serverFlag;

*********** I know that the line below is being executed *************

System.out.println("SportsWinClient: Disconnected");
}
// stop interrupts this thread so this will be bypassed on
a stop
try {
Thread.sleep(10000);
} catch (InterruptedException ie) { }
}
System.out.println("SportsWinClient: Stopping");
}

public void disconnect() {
if (isConnected())
if (socket != null)
try {
socket.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}

public void stop() {
runFlag = false;
thread.interrupt();
if (socket != null)
try {
socket.close();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}

public boolean isConnected() {
return isConnected;
}

private final java.util.List<ConnectionListener> connectionListeners =
new ArrayList<ConnectionListener>();

public synchronized void addConnectionListener(ConnectionListener cl) {
connectionListeners.add(cl);
}

public synchronized void
removeConnectionListener(ConnectionListener cl) {
connectionListeners.remove(cl);
}

private synchronized void fireConnectionEvent(int id) {
ConnectionEvent ce = new ConnectionEvent(this,id);

for (ConnectionListener listener : connectionListeners)
listener.connState(ce);
}
}

On 01/03/12 19:49, Knute Johnson wrote:

Can you provide an exact trace of the messages that are visible in the
code you've shown (around the problematic event, of course), just so
there's no ambiguity about what you're seeing?


Catch Throwable here too, print it out, and rethrow, so we can be sure
no other exception is slipping through.

On 3/1/2012 12:59 PM, Steven Simpson wrote:

When the problem arrives, it prints the SportsWinClient: Disconnected
message and nothing else. It doesn't reconnect to the server.

I'll try putting in a catch for Throwable but it could take some time
before I see the problem again.

Thanks,

On 3/1/2012 11:49 AM, Knute Johnson wrote:


This code below strikes me as possibly being in the wrong order. If you
interrupt a thread, it should bail. But if you close the socket, it
should do the same thing, and close the socket too (I'm pretty sure
there's a handshake for TCP for "close me".)

If you interrupt the thread, then close the socket, the "close" might
never happen. I'd *just* close the socket, if that passes your testing.
Try to let the thread actually execute the close, and then unwind
naturally.

(I didn't look to see if your thread/runnable has other places it waits
besides the socket. If it does, those should be eliminated, replaced
with other sockets which are also closed, etc.)