首页 > itarticle > 深入分析broken pipe和connetion reset by peer

深入分析broken pipe和connetion reset by peer

admin 9月 30, 2020 0

这两个异常在网络编程里应该是比较常见的，之前对于他们的理解只知道是对端关闭了连接，但什么情况下会报Broken pipe，什么情况下会报Connetion reset by peer是一无所知的，今天专门就这个问题做一个调研，下面记录下整个调研过程。

TCP的连接的建立与终止

在讲具体原因之前，有必要说一下TCP的三次握手与四次挥手，下面用一张图来说明（图是网上找的）。

tcp_hank

上面三条是连接的建立，也就是说的三次挥手；下面四条是正常的连接终止，也就是四次挥手，关于为什么建立连接需要三次交互，而终止连接需要四次交互，这里不做过多解释，可以查询相关信息。正常关闭连接是发送FIN，还有一种异常关闭是发送RST，今天要讨论的就和这个有关系。

异常模拟

下面写了一段NIO的代码，Server端代码如下，代码很简单，看不懂的可以稍微补一点Nio的知识：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
import java.util.concurrent.TimeUnit;

public class  {
	public static void main(String[] args) throws IOException, InterruptedException {
		ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
		serverSocketChannel.socket().bind(new InetSocketAddress(9000));
		serverSocketChannel.configureBlocking(false);
		Selector selector = Selector.open();
		serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

		while (true) {
			int selected = selector.select();
			if (selected > 0) {
				Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
				while (iterator.hasNext()) {
					SelectionKey selectionKey = iterator.next();
					iterator.remove();
					if (selectionKey.isAcceptable()) {
						System.err.println("Acceptable");
						SocketChannel socketChannel = serverSocketChannel.accept();
						socketChannel.configureBlocking(false);
						socketChannel.register(selector, SelectionKey.OP_READ);
					} else if (selectionKey.isReadable()) {
						System.err.println("Readable");
						SocketChannel socketChannel = (SocketChannel) selectionKey.channel();
						ByteBuffer buffer = ByteBuffer.allocate(128);
						socketChannel.read(buffer);
						System.out.println("接收来自客户端的数据：" + new String(buffer.array()));
						selectionKey.interestOps(SelectionKey.OP_WRITE);
					} else if (selectionKey.isWritable()) {
						System.err.println("Writable");
						TimeUnit.SECONDS.sleep(3);
						SocketChannel channel = (SocketChannel) selectionKey.channel();
						String content = "向客户端发送数据 : " + System.currentTimeMillis();
						ByteBuffer bufferOne = ByteBuffer.wrap(content.getBytes());
						channel.write(bufferOne); 
						System.err.println("发送bufferOne");
						TimeUnit.SECONDS.sleep(3);
						ByteBuffer bufferTwo = ByteBuffer.wrap(content.getBytes());
						channel.write(bufferTwo); //@2
						System.err.println("发送bufferTwo");
						selectionKey.interestOps(SelectionKey.OP_READ);
					}
				}
			}
		}
	}
}

大家可以看到，在接收到客户端的数据以后，首先会往客户端写数据，在@1代码处，然后会再次往客户端写数据，在@2代码处。下面再看一下客户端的代码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.util.Iterator;

public class NioClient {
	public static void main(String[] args) throws IOException {
		SocketChannel socketChannel = SocketChannel.open();

		socketChannel.configureBlocking(false);
		Selector selector = Selector.open();
		socketChannel.register(selector, SelectionKey.OP_CONNECT);
		socketChannel.connect(new InetSocketAddress("192.168.2.60", 9000));
		//socketChannel.socket().setSoLinger(true,0);  //@3

		while (true) {
			int select = selector.select();
			if (select > 0) {
				Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
				while (iterator.hasNext()) {
					SelectionKey selectionKey = iterator.next();
					if (selectionKey.isConnectable()) {
						System.err.println("Connectable");
						SocketChannel clientChannel = (SocketChannel) selectionKey.channel();
						clientChannel.finishConnect();
						selectionKey.interestOps(SelectionKey.OP_WRITE);
					} else if (selectionKey.isReadable()) {
						System.out.println("Readable");
						SocketChannel channel = (SocketChannel) selectionKey.channel();
						ByteBuffer buffer = ByteBuffer.allocate(128);
						channel.read(buffer);
						selectionKey.interestOps(SelectionKey.OP_WRITE);
						System.out.println("收到服务端数据" + new String(buffer.array()));
					} else if (selectionKey.isWritable()) {
						SocketChannel clientChannel = (SocketChannel) selectionKey.channel();
						String str = "qiwoo mobile";
						ByteBuffer buffer = ByteBuffer.wrap(str.getBytes());
						clientChannel.write(buffer);
						selectionKey.interestOps(SelectionKey.OP_READ);
						System.out.println("向服务端发送数据" + new String(buffer.array()));
						clientChannel.close(); //@4
					}
					iterator.remove();
				}
			}
		}
	}
}

当客户端建立完连接往服务端发送数据后，会调close()方法，关闭连接，在代码@4处。我们先启动NioServer，然后再启动NioClient。会发现NioServer会报如下异常：

1
2
3
4
5
6
7
8
9
10
11
12
13
 
[[email protected] tmp]# java NioServer
Acceptable
Readable
接收来自客户端的数据：qiwoo mobile
Writable
发送bufferOne
Exception in thread "main" java.io.IOException: 断开的管道
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
	at NioServer.main(NioServer.java:48)

而此时，我们用tcpdump抓包看的情况如下：

1
2
3
4
5
6
7
8
9
 
17:35:38.819801 IP 192.168.2.11.33962 > 192.168.2.60.cslistener: Flags [S], seq 3601143964, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 0
17:35:38.820008 IP 192.168.2.60.cslistener > 192.168.2.11.33962: Flags [S.], seq 3163373872, ack 3601143965, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 0
17:35:38.820072 IP 192.168.2.11.33962 > 192.168.2.60.cslistener: Flags [.], ack 1, win 29, length 0
17:35:38.821801 IP 192.168.2.11.33962 > 192.168.2.60.cslistener: Flags [P.], seq 1:13, ack 1, win 29, length 12
17:35:38.821958 IP 192.168.2.60.cslistener > 192.168.2.11.33962: Flags [.], ack 13, win 58, length 0
17:35:38.822097 IP 192.168.2.11.33962 > 192.168.2.60.cslistener: Flags [F.], seq 13, ack 1, win 29, length 0
17:35:38.861551 IP 192.168.2.60.cslistener > 192.168.2.11.33962: Flags [.], ack 14, win 58, length 0
17:35:41.825209 IP 192.168.2.60.cslistener > 192.168.2.11.33962: Flags [P.], seq 1:41, ack 14, win 58, length 40
17:35:41.825223 IP 192.168.2.11.33962 > 192.168.2.60.cslistener: Flags [R], seq 3601143978, win 0, length 0

结合上面信息，我们可以看到，在NioClient调用close()方法后，NioClient会往NioServer发送一个FIN包，而NioServer还继续往NioClient写数据，此时会收到NioClient返回的RST；接着NioServer继续往NioClient写数据，这个时候，NioServer就报了Broken pipe；

现在我们将NioClient代码处@3的注释打开，然后执行同样的操作，会发现NioServer报如下异常：

1
2
3
4
5
6
7
8
9
10
11
12
 
[[email protected] tmp]# java NioServer
Acceptable
Readable
接收来自客户端的数据：qiwoo mobile
Writable
Exception in thread "main" java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
	at sun.nio.ch.IOUtil.write(IOUtil.java:65)
	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
	at NioServer.main(NioServer.java:44)

而此时，我们用tcpdump抓包看的情况如下：

1
2
3
4
5
6
 
17:37:12.234530 IP 192.168.2.11.34818 > 192.168.2.60.cslistener: Flags [S], seq 2138229963, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 0
17:37:12.234728 IP 192.168.2.60.cslistener > 192.168.2.11.34818: Flags [S.], seq 3174262802, ack 2138229964, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 9], length 0
17:37:12.234740 IP 192.168.2.11.34818 > 192.168.2.60.cslistener: Flags [.], ack 1, win 29, length 0
17:37:12.295824 IP 192.168.2.11.34818 > 192.168.2.60.cslistener: Flags [P.], seq 1:13, ack 1, win 29, length 12
17:37:12.296007 IP 192.168.2.60.cslistener > 192.168.2.11.34818: Flags [.], ack 13, win 58, length 0
17:37:12.296074 IP 192.168.2.11.34818 > 192.168.2.60.cslistener: Flags [R.], seq 13, ack 1, win 29, length 0

结合上面信息，我们可以看到，在NioClient调用close()方法后，因为配置了soLinger选项，也即，NioClient会往NioServer发送一个RST包，而NioServer还继续往NioClient写数据，这个时候，NioServer就报了Broken pipe。

结论

从上面的两个例子可以看出，对于被动关闭方来说，也就是上述例子的NioServer，有两种情况：

被FIN关闭

第一次write得到了RST响应，但不会抛异常，第二次write，就会报Broken pipe（针对Linux系统）

被RST关闭

无论读写都会报Connection reset by peer

RST的场景

什么场景下会发送RST呢，网上总结的也很多，这里就不在赘述了。

参考文章

微海报分享

近期文章

近期评论

标签

热门

文章归档

分类目录

功能