Egg.js 应用经常会出现大量的 socket 连接
发布于 4 年前 作者 mehunk 4327 次浏览 来自 问答

当前的一个 Egg.js 应用,之前是正常的,最近几天常常会出现大量的 socket 连接,频率不固定,大概1、2天左右就会出现,表现就是请求响应速度特别慢,但是 CPU 正常,SQL 也没有慢查询。使用命令 lsof -i:7001 就会发现大量的 socket 连接。这种情况该怎么解决呢?

以下是 nginx 的错误日志。

2020/07/24 08:03:44 [error] 20926#0: *131353 connect() failed (111: Connection refused) while connecting to upstream, client: 203.168.21.214, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/charge"
2020/07/24 08:03:44 [error] 20926#0: *131487 connect() failed (111: Connection refused) while connecting to upstream, client: 27.195.150.150, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://www.cn/platform/agent"
2020/07/24 08:03:44 [error] 20926#0: *131487 connect() failed (111: Connection refused) while connecting to upstream, client: 27.195.150.150, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://www.cn/platform/agent"
2020/07/24 08:03:44 [error] 20926#0: *131353 connect() failed (111: Connection refused) while connecting to upstream, client: 203.168.21.214, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/charge"
2020/07/24 08:03:44 [error] 20926#0: *131535 connect() failed (111: Connection refused) while connecting to upstream, client: 117.136.56.82, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/login"
2020/07/24 08:03:44 [error] 20926#0: *131534 connect() failed (111: Connection refused) while connecting to upstream, client: 223.104.16.16, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/charge"
2020/07/24 08:03:45 [error] 20926#0: *131534 connect() failed (111: Connection refused) while connecting to upstream, client: 223.104.16.16, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/charge"
2020/07/24 08:05:18 [error] 20926#0: *131726 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 112.97.59.250, server: xxx.cn, request: "POST /graphql HTTP/1.1", upstream: "http://127.0.0.1:7001/graphql", host: "xxx.cn", referrer: "http://xxx.cn/packages"
COMMAND   PID  USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
node    25611  root   27u  IPv4 349980918      0t0  TCP *:afs3-callback (LISTEN)
nginx   25622 nginx   23u  IPv4 350837668      0t0  TCP localhost:46782->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   24u  IPv4 350835359      0t0  TCP localhost:46234->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   27u  IPv4 350837451      0t0  TCP localhost:46320->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   28u  IPv4 350836524      0t0  TCP localhost:46460->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   34u  IPv4 350835463      0t0  TCP localhost:46484->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   35u  IPv4 350838859      0t0  TCP localhost:46944->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   38u  IPv4 350837405      0t0  TCP localhost:46254->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   39u  IPv4 350836513      0t0  TCP localhost:46426->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   43u  IPv4 350835409      0t0  TCP localhost:46358->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   55u  IPv4 350836498      0t0  TCP localhost:46396->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   57u  IPv4 350837654      0t0  TCP localhost:46752->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   63u  IPv4 350836536      0t0  TCP localhost:46492->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   65u  IPv4 350836538      0t0  TCP localhost:46500->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   66u  IPv4 350839848      0t0  TCP localhost:47026->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   74u  IPv4 350836709      0t0  TCP localhost:46792->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   79u  IPv4 350836617      0t0  TCP localhost:46644->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   81u  IPv4 350837758      0t0  TCP localhost:46916->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   82u  IPv4 350839809      0t0  TCP localhost:46938->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   84u  IPv4 350837702      0t0  TCP localhost:46828->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   85u  IPv4 350836647      0t0  TCP localhost:46704->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   88u  IPv4 350835569      0t0  TCP localhost:46716->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   92u  IPv4 350837714      0t0  TCP localhost:46854->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx   98u  IPv4 350838817      0t0  TCP localhost:46872->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  101u  IPv4 350837738      0t0  TCP localhost:46880->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  105u  IPv4 350839815      0t0  TCP localhost:46948->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  106u  IPv4 350839854      0t0  TCP localhost:47048->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  107u  IPv4 350838902      0t0  TCP localhost:47010->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  109u  IPv4 350838867      0t0  TCP localhost:46968->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  110u  IPv4 350838087      0t0  TCP localhost:46986->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  113u  IPv4 350839832      0t0  TCP localhost:spremotetablet->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  115u  IPv4 350835303      0t0  TCP localhost:46036->localhost:afs3-callback (ESTABLISHED)
nginx   25622 nginx  129u  IPv4 350836359      0t0  TCP localhost:46046->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   23u  IPv4 350838919      0t0  TCP localhost:47040->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   31u  IPv4 350837624      0t0  TCP localhost:46700->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   38u  IPv4 350837561      0t0  TCP localhost:46566->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   41u  IPv4 350835052      0t0  TCP localhost:45560->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   42u  IPv4 350837552      0t0  TCP localhost:46544->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   52u  IPv4 350837978      0t0  TCP localhost:46588->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   54u  IPv4 350838058      0t0  TCP localhost:46928->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   58u  IPv4 350837994      0t0  TCP localhost:46632->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   61u  IPv4 350837681      0t0  TCP localhost:46802->localhost:afs3-callback (ESTABLISHED)
nginx   25624 nginx   62u  IPv4 350836628      0t0  TCP localhost:46660->localhost:afs3-callback (ESTABLISHED)
node    25644  root   55u  IPv4 350837387      0t0  TCP localhost:afs3-callback->localhost:46234 (ESTABLISHED)
node    25644  root   59u  IPv4 350837284      0t0  TCP localhost:afs3-callback->localhost:46046 (ESTABLISHED)
node    25644  root  155u  IPv4 350836438      0t0  TCP localhost:afs3-callback->localhost:46254 (ESTABLISHED)
node    25644  root  156u  IPv4 350835560      0t0  TCP localhost:afs3-callback->localhost:46704 (ESTABLISHED)
node    25644  root  157u  IPv4 350838847      0t0  TCP localhost:afs3-callback->localhost:46916 (ESTABLISHED)
node    25644  root  158u  IPv4 350834506      0t0  TCP localhost:afs3-callback->localhost:45560 (ESTABLISHED)
node    25644  root  159u  IPv4 350835643      0t0  TCP localhost:afs3-callback->localhost:46944 (ESTABLISHED)
node    25644  root  160u  IPv4 350836719      0t0  TCP localhost:afs3-callback->localhost:46802 (ESTABLISHED)
node    25644  root  161u  IPv4 350836653      0t0  TCP localhost:afs3-callback->localhost:46716 (ESTABLISHED)
node    25644  root  162u  IPv4 350838795      0t0  TCP localhost:afs3-callback->localhost:46828 (ESTABLISHED)
node    25644  root  163u  IPv4 350839839      0t0  TCP localhost:afs3-callback->localhost:47010 (ESTABLISHED)
node    25644  root  164u  IPv4 350838879      0t0  TCP localhost:afs3-callback->localhost:46986 (ESTABLISHED)
node    25644  root  165u  IPv4 350837470      0t0  TCP localhost:afs3-callback->localhost:46358 (ESTABLISHED)
node    25644  root  166u  IPv4 350835435      0t0  TCP localhost:afs3-callback->localhost:46426 (ESTABLISHED)
node    25644  root  167u  IPv4 350837934      0t0  TCP localhost:afs3-callback->localhost:46460 (ESTABLISHED)
node    25644  root  168u  IPv4 350836558      0t0  TCP localhost:afs3-callback->localhost:46544 (ESTABLISHED)
node    25644  root  169u  IPv4 350835470      0t0  TCP localhost:afs3-callback->localhost:46500 (ESTABLISHED)
node    25644  root  170u  IPv4 350836701      0t0  TCP localhost:afs3-callback->localhost:46782 (ESTABLISHED)
node    25644  root  171u  IPv4 350838888      0t0  TCP localhost:afs3-callback->localhost:spremotetablet (ESTABLISHED)
node    25644  root  172u  IPv4 350835528      0t0  TCP localhost:afs3-callback->localhost:46660 (ESTABLISHED)
node    25644  root  173u  IPv4 350835679      0t0  TCP localhost:afs3-callback->localhost:47040 (ESTABLISHED)
node    25644  root  179u  IPv4 350837815      0t0  TCP localhost:afs3-callback->localhost:46036 (ESTABLISHED)
node    25645  root   56u  IPv4 350836475      0t0  TCP localhost:afs3-callback->localhost:46320 (ESTABLISHED)
node    25645  root   60u  IPv4 350837673      0t0  TCP localhost:afs3-callback->localhost:46792 (ESTABLISHED)
node    25645  root  156u  IPv4 350837523      0t0  TCP localhost:afs3-callback->localhost:46484 (ESTABLISHED)
node    25645  root  157u  IPv4 350837917      0t0  TCP localhost:afs3-callback->localhost:46396 (ESTABLISHED)
node    25645  root  158u  IPv4 350835675      0t0  TCP localhost:afs3-callback->localhost:47026 (ESTABLISHED)
node    25645  root  159u  IPv4 350835508      0t0  TCP localhost:afs3-callback->localhost:46588 (ESTABLISHED)
node    25645  root  160u  IPv4 350837531      0t0  TCP localhost:afs3-callback->localhost:46492 (ESTABLISHED)
node    25645  root  161u  IPv4 350835628      0t0  TCP localhost:afs3-callback->localhost:46928 (ESTABLISHED)
node    25645  root  162u  IPv4 350835639      0t0  TCP localhost:afs3-callback->localhost:46938 (ESTABLISHED)
node    25645  root  163u  IPv4 350836574      0t0  TCP localhost:afs3-callback->localhost:46566 (ESTABLISHED)
node    25645  root  164u  IPv4 350838073      0t0  TCP localhost:afs3-callback->localhost:46968 (ESTABLISHED)
node    25645  root  165u  IPv4 350837589      0t0  TCP localhost:afs3-callback->localhost:46632 (ESTABLISHED)
node    25645  root  166u  IPv4 350837600      0t0  TCP localhost:afs3-callback->localhost:46644 (ESTABLISHED)
node    25645  root  167u  IPv4 350835647      0t0  TCP localhost:afs3-callback->localhost:46948 (ESTABLISHED)
node    25645  root  168u  IPv4 350836681      0t0  TCP localhost:afs3-callback->localhost:46752 (ESTABLISHED)
node    25645  root  169u  IPv4 350835557      0t0  TCP localhost:afs3-callback->localhost:46700 (ESTABLISHED)
node    25645  root  170u  IPv4 350837729      0t0  TCP localhost:afs3-callback->localhost:46872 (ESTABLISHED)
node    25645  root  171u  IPv4 350835683      0t0  TCP localhost:afs3-callback->localhost:47048 (ESTABLISHED)
node    25645  root  172u  IPv4 350838805      0t0  TCP localhost:afs3-callback->localhost:46854 (ESTABLISHED)
node    25645  root  173u  IPv4 350838824      0t0  TCP localhost:afs3-callback->localhost:46880 (ESTABLISHED)
2 回复

nginx 和 egg 应用建立的连接?你这还有很多 root 用户建立的,看看是什么应用,nginx 用户的应该是正常的

@hyj1991 应该是和nginx建立的正向和反向的连接吧,你看端口号都是成对出现的,nginx->node,node->nginx

回到顶部