ZooKeeper connection refused

[shacky]

Hi.
I have a SolrCloud cluster with 3 nodes Solr + Zookeeper.

My solr.in.sh file is configured as following:
ZK_HOST="zk1,zk2,zk3"

All worked good but now I cannot start SOLR nodes and the command exit
with the following errors:

***@index1:~# service solr restart
Sending stop command to Solr running on port 8983 ... waiting 5
seconds to allow Jetty process 32087 to stop gracefully.
Waiting to see Solr listening on port 8983 [\] Still not seeing Solr
listening on 8983 after 30 seconds!
WARN - 2015-06-17 10:18:37.158; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:37.823; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:38.990; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:40.543; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:42.174; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

I can telnet to the ZooKeeper port:

***@index1:~# telnet zk1 2181
Trying 192.168.70.31...
Connected to index1.dc.my.network.
Escape character is '^]'.

Could you help me please?

Thank you very much!
Bye

[Shalin Shekhar Mangar]

You are asking telnet to connect to zk1 on port 2181 but you have not
specified the port to Solr. You should set
ZK_HOST="zk1:2181,zk2:2181,zk3:2181" instead.

Post by shacky
Hi.
I have a SolrCloud cluster with 3 nodes Solr + Zookeeper.
ZK_HOST="zk1,zk2,zk3"
All worked good but now I cannot start SOLR nodes and the command exit
Sending stop command to Solr running on port 8983 ... waiting 5
seconds to allow Jetty process 32087 to stop gracefully.
Waiting to see Solr listening on port 8983 [\] Still not seeing Solr
listening on 8983 after 30 seconds!
WARN - 2015-06-17 10:18:37.158; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:37.823; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:38.990; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:40.543; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
WARN - 2015-06-17 10:18:42.174; [ ]
org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server
null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Trying 192.168.70.31...
Connected to index1.dc.my.network.
Escape character is '^]'.
Could you help me please?
Thank you very much!
Bye

[shacky]

Post by Shalin Shekhar Mangar
You are asking telnet to connect to zk1 on port 2181 but you have not
specified the port to Solr. You should set
ZK_HOST="zk1:2181,zk2:2181,zk3:2181" instead.

I modified the ZK_HOST instance with the port, but the problem is not solved.
Do you have any ideas?

[Shalin Shekhar Mangar]

Is ZK healthy? Can you try the following from the server on which Solr
is running:

echo ruok | nc zk1 2181

Post by shacky

Post by Shalin Shekhar Mangar
You are asking telnet to connect to zk1 on port 2181 but you have not
specified the port to Solr. You should set
ZK_HOST="zk1:2181,zk2:2181,zk3:2181" instead.

I modified the ZK_HOST instance with the port, but the problem is not solved.
Do you have any ideas?

[shacky]

Post by Shalin Shekhar Mangar
Is ZK healthy? Can you try the following from the server on which Solr
echo ruok | nc zk1 2181

Thank you very much Shalin for your answer!
My ZK cluster was not ready because two nodes was dead and only one
node was running.
I fixed the two nodes and now all works good.
Thank you very much!