Created
May 23, 2017 14:04
-
-
Save mfenniak/d1bfa2c1c94ee9980113e70ba19ca563 to your computer and use it in GitHub Desktop.
4.9.27-14.31.amzn.x86_64 ena issue
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
May 21 12:05:37 ip-172-16-15-126 ec2net: [get_meta] Trying to get http://169.254.169.254/latest/meta-data/network/interfaces/macs/06:1e:aa:71:34:ca/local-ipv4s | |
May 21 12:05:37 ip-172-16-15-126 ec2net: [rewrite_aliases] Rewriting aliases of eth0 | |
May 21 12:06:50 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 125920ms. | |
May 21 12:07:59 ip-172-16-15-126 consul[3614]: 2017/05/21 12:07:59 [INFO] consul: New leader elected: i-0f14d98ae903e7de2 | |
May 21 12:08:56 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 114310ms. | |
May 21 12:10:51 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 119760ms. | |
May 21 12:12:51 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 115570ms. | |
May 21 12:13:36 ip-172-16-15-126 consul[3614]: 2017/05/21 12:13:36 [INFO] serf: attempting reconnect to i-0bd3ab67982f1fe55 172.16.17.249:8301 | |
May 21 12:14:46 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 114980ms. | |
May 21 12:15:54 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:54 [INFO] memberlist: Suspect i-0cc0de42055d7454f has failed, no acks received | |
May 21 12:15:55 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:55 [INFO] memberlist: Suspect i-06bc61f26c1304e21 has failed, no acks received | |
May 21 12:15:56 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:56 [INFO] memberlist: Suspect i-03dc80424912d45e2 has failed, no acks received | |
May 21 12:15:57 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:57 [INFO] memberlist: Suspect i-01a218a01be0b7228 has failed, no acks received | |
May 21 12:15:58 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:58 [INFO] memberlist: Suspect i-08b27d38fd68fa803 has failed, no acks received | |
May 21 12:15:59 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:59 [INFO] memberlist: Suspect i-0668c4279c75c6276 has failed, no acks received | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.005610] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 533. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.010071] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 905. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.014421] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 906. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.019248] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 416. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.024266] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 417. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.028839] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 418. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.033007] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 419. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.037131] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 420. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.041450] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 421. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.045691] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 83. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.049838] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 86. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.054012] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 87. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.058086] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 89. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.062460] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 94. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.066849] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 95. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.071118] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 98. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.075747] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 99. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.080056] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 100. | |
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.084511] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 102. | |
May 21 12:16:00 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:00 [INFO] memberlist: Suspect i-0d3f1ddeac019e354 has failed, no acks received | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.093646] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 21. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.097647] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 92. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.101552] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 96. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.105449] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 237. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.109379] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 238. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.126336] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 244. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.130579] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 245. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.134763] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 246. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.139039] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 247. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.147302] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 249. | |
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.151355] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 250. | |
May 21 12:16:01 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:01 [INFO] memberlist: Suspect i-0f6372101a741e520 has failed, no acks received | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.181701] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 993. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.186420] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 994. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.190616] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 995. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.194761] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 996. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.199496] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 997. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.204210] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1000. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.208920] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 809. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.213519] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 810. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.218263] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 811. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.222886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 812. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.227431] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 813. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.231836] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 815. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.236406] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 171. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.241056] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 172. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.245571] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 173. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.249998] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 174. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.254277] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 175. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.258414] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 176. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.262361] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 177. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.266272] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 178. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.270505] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 525. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.274667] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 526. | |
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.278761] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 531. | |
May 21 12:16:02 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:02 [INFO] memberlist: Suspect i-08e477c727ca9b536 has failed, no acks received | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.301733] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 532. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.305903] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 535. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.310489] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 536. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.315117] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 907. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.319414] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 908. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.323696] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 909. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.327831] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 910. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.332092] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 911. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.336569] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 422. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.340603] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 423. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.344782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 424. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.349244] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 425. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.353324] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 426. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.357547] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 427. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.361792] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 428. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.366003] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 429. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.370183] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 430. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.374445] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 431. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.378399] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 97. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.382585] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 110. | |
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.387010] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 113. | |
May 21 12:16:03 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:03 [INFO] memberlist: Suspect i-0803418647953526e has failed, no acks received | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.421767] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 104. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.426092] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 112. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.430411] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 115. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.434730] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 251. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.439046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 252. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.443171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 253. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.447194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 255. | |
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.451137] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 256. | |
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] memberlist: Marking i-0cc0de42055d7454f as failed, suspect timeout reached | |
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] serf: EventMemberFailed: i-0cc0de42055d7454f 172.16.16.88 | |
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] memberlist: Suspect i-0a0a5bfcdf25cf714 has failed, no acks received | |
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [ERR] yamux: keepalive failed: i/o deadline reached | |
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [ERR] agent: failed to sync remote state: rpc error: EOF | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.477819] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1001. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.482124] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1002. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.486288] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1003. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.490459] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1004. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.494432] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 814. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.498505] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 816. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.503028] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 817. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.507216] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 818. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.511233] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 819. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.515394] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 179. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.519865] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 180. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.524532] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 181. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.528870] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 182. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.533590] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 183. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.538120] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 184. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.542559] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 185. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.547110] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 186. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.551720] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 537. | |
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.555983] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 538. | |
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] memberlist: Marking i-06bc61f26c1304e21 as failed, suspect timeout reached | |
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] serf: EventMemberFailed: i-06bc61f26c1304e21 172.16.14.71 | |
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] memberlist: Suspect i-0ee769a39584ed7d7 has failed, no acks received | |
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] memberlist: Marking i-03dc80424912d45e2 as failed, suspect timeout reached | |
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] serf: EventMemberFailed: i-03dc80424912d45e2 172.16.14.41 | |
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] memberlist: Suspect i-040d2e16273351cff has failed, no acks received | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.565866] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 539. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.570171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 912. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.574194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 913. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.578084] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 914. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.582171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 915. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.586198] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 916. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.590397] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 917. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.594609] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 918. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.598782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 919. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.603094] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 432. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.607003] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 433. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.610941] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 434. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.628329] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 435. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.632294] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 93. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.636191] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 101. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.640116] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 103. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.644036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 105. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.647949] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 106. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.651876] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 107. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.655777] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 109. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.659686] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 111. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.663608] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 114. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.667519] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 116. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.671506] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 117. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.675415] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 118. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.679329] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 119. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.683233] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 120. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.687148] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 121. | |
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.691075] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 122. | |
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [ERR] agent: coordinate update error: rpc error: failed to get conn: dial tcp 172.16.14.67:8300: getsockopt: no route to host | |
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] memberlist: Marking i-01a218a01be0b7228 as failed, suspect timeout reached | |
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] serf: EventMemberFailed: i-01a218a01be0b7228 172.16.16.114 | |
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] memberlist: Suspect i-0f0f87b95a476583d has failed, no acks received | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.721917] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 108. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.725956] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 123. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.729955] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 125. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.733997] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 127. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.737989] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 128. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.741943] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 129. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.745902] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 257. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.749889] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 258. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.753801] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 259. | |
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.757649] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 260. | |
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] memberlist: Marking i-08b27d38fd68fa803 as failed, suspect timeout reached | |
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] serf: EventMemberFailed: i-08b27d38fd68fa803 172.16.16.243 | |
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] memberlist: Suspect i-0de2d29024b43cb6d has failed, no acks received | |
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] consul: removing server i-08b27d38fd68fa803 (Addr: 172.16.16.243:8300) (DC: dc1) | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.773947] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1005. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.778595] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1008. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.782871] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 820. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.787066] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 821. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.791196] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 823. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.795361] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 824. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.799631] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 825. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.803782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 826. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.807906] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 827. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.812074] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 828. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.816276] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 187. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.820616] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 188. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.824935] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 189. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.829714] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 190. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.834064] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 191. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.838281] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 540. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.842491] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 541. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.846633] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 542. | |
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.850898] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 543. | |
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] memberlist: Marking i-0668c4279c75c6276 as failed, suspect timeout reached | |
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] serf: EventMemberFailed: i-0668c4279c75c6276 172.16.16.47 | |
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] memberlist: Suspect i-09013c5adb8f3f726 has failed, no acks received | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.861980] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 544. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.866109] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 546. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.870014] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 547. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.873937] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 920. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.877856] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 921. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.881790] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 922. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.885673] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 923. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.889606] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 436. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.893514] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 437. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.897422] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 438. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.901323] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 439. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.905224] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 440. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.909145] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 441. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.913088] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 91. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.916959] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 124. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.920873] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 126. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.924787] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 130. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.928700] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 131. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.932622] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 132. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.936531] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 133. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.940467] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 134. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.944394] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 135. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.948308] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 136. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.952280] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 139. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.956239] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 144. | |
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.960202] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 146. | |
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] memberlist: Marking i-0d3f1ddeac019e354 as failed, suspect timeout reached | |
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] serf: EventMemberFailed: i-0d3f1ddeac019e354 172.16.17.176 | |
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] memberlist: Suspect i-0552ac87d19ad1bad has failed, no acks received | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.982034] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 138. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.986592] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 140. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.990966] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 141. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.995277] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 143. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.999570] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 145. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.003865] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 148. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.008139] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 149. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.012055] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 150. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.015886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 254. | |
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.019808] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 261. | |
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] memberlist: Marking i-0f6372101a741e520 as failed, suspect timeout reached | |
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] serf: EventMemberFailed: i-0f6372101a741e520 172.16.14.160 | |
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] memberlist: Suspect i-06675422c58ce18ce has failed, no acks received | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.038051] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1009. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.042121] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 822. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.046064] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 829. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.050036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 830. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.053996] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 831. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.057926] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 832. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.061876] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 833. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.065850] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 192. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.082613] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 193. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.086549] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 194. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.090484] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 195. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.094422] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 196. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.098372] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 197. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.102305] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 198. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.106209] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 199. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.110129] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 201. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.114052] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 202. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.118006] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 545. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.122035] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 548. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.126067] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 549. | |
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.130059] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 550. | |
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] memberlist: Marking i-08e477c727ca9b536 as failed, suspect timeout reached | |
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] serf: EventMemberFailed: i-08e477c727ca9b536 172.16.17.83 | |
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] memberlist: Suspect i-0280f96a4564c8ccd has failed, no acks received | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.158099] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 552. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.162157] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 924. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.166143] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 925. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.170090] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 926. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.173990] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 927. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.177906] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 928. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.181828] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 929. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.185730] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 932. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.189657] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 442. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.193560] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 443. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.197454] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 444. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.201354] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 445. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.205246] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 142. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.209141] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 147. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.213037] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 152. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.216943] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 153. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.220847] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 154. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.224782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 155. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.228685] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 156. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.232587] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 157. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.236474] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 158. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.240378] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 159. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.244261] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 160. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.248162] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 161. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.252090] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 165. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.255992] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 168. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.259904] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 170. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.263837] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 171. | |
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.267754] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 172. | |
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] memberlist: Marking i-0803418647953526e as failed, suspect timeout reached | |
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] serf: EventMemberFailed: i-0803418647953526e 172.16.17.61 | |
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] memberlist: Suspect i-03c4d88e2854d047b has failed, no acks received | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.278157] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 162. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.282180] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 163. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.285973] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 169. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.289771] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 173. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.293592] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 262. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.297369] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 263. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.301155] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 264. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.304923] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 265. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.308688] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 266. | |
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.312457] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 267. | |
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] memberlist: Marking i-0a0a5bfcdf25cf714 as failed, suspect timeout reached | |
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] serf: EventMemberFailed: i-0a0a5bfcdf25cf714 172.16.14.141 | |
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] memberlist: Suspect i-0af21f21158ce7c92 has failed, no acks received | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.334181] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1006. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.338200] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1010. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.342124] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1011. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.346053] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 834. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.349942] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 835. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.353824] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 836. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.357717] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 837. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.361651] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 838. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.365539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 839. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.369402] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 840. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.373280] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 200. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.377153] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 203. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.381036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 204. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.384922] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 205. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.388803] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 206. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.392706] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 207. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.396588] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 208. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.400490] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 209. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.404393] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 553. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.408259] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 554. | |
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.412169] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 555. | |
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] memberlist: Marking i-0ee769a39584ed7d7 as failed, suspect timeout reached | |
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] serf: EventMemberFailed: i-0ee769a39584ed7d7 172.16.14.199 | |
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] memberlist: Suspect i-0f14d98ae903e7de2 has failed, no acks received | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.422253] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 933. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.426747] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 934. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.431097] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 935. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.435456] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 446. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.439700] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 447. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.443966] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 448. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.448352] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 449. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.452577] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 450. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.456734] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 137. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.461016] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 151. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.465300] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 164. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.469591] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 166. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.473890] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 167. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.478065] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 176. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.482228] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 177. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.486393] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 178. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.490539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 179. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.507563] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 181. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.511724] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 182. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.515860] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 185. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.520151] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 188. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.524560] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 191. | |
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.529148] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 192. | |
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] memberlist: Marking i-040d2e16273351cff as failed, suspect timeout reached | |
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] serf: EventMemberFailed: i-040d2e16273351cff 172.16.17.130 | |
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] memberlist: Suspect i-02b2968ab933cf778 has failed, no acks received | |
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] serf: attempting reconnect to i-040d2e16273351cff 172.16.17.130:8301 | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.542273] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 174. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.546530] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 183. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.550760] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 194. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.554962] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 201. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.559094] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 268. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.563218] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 269. | |
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.567367] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 270. | |
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] memberlist: Marking i-0f0f87b95a476583d as failed, suspect timeout reached | |
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] serf: EventMemberFailed: i-0f0f87b95a476583d 172.16.14.143 | |
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] memberlist: Suspect i-055abe2ce793ae2c8 has failed, no acks received | |
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [ERR] agent: failed to sync remote state: rpc error: failed to get conn: dial tcp 172.16.16.243:8300: i/o timeout | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.598324] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1007. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.602886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1013. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.607268] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 841. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.611594] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 842. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.615959] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 843. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.620320] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 844. | |
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] memberlist: Marking i-0de2d29024b43cb6d as failed, suspect timeout reached | |
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] serf: EventMemberFailed: i-0de2d29024b43cb6d 172.16.14.47 | |
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] memberlist: Suspect i-0757cbd488bd58eb4 has failed, no acks received | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.624799] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 845. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.629358] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 846. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.633809] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 847. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.638172] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 848. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.642539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 849. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.646840] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 850. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.651040] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 851. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.655200] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 210. | |
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.659342] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 211. | |
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] memberlist: Marking i-09013c5adb8f3f726 as failed, suspect timeout reached | |
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] serf: EventMemberFailed: i-09013c5adb8f3f726 172.16.16.225 | |
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] memberlist: Suspect i-0b10c1368ac9bd218 has failed, no acks received | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.686391] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 937. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.691116] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 938. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.695751] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 939. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.700366] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 940. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.704947] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 451. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.709635] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 452. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.714111] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 184. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.718668] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 187. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.723125] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 189. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.727549] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 190. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.731936] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 193. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.736257] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 195. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.740642] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 197. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.745014] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 198. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.749349] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 199. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.753895] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 200. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.758412] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 202. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.762935] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 204. | |
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.767428] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 211. | |
May 21 12:16:20 ip-172-16-15-126 kernel: [335609.771942] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 212. | |
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] memberlist: Marking i-0552ac87d19ad1bad as failed, suspect timeout reached | |
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] serf: EventMemberFailed: i-0552ac87d19ad1bad 172.16.16.153 | |
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] memberlist: Suspect i-06c86172cdacb3d1f has failed, no acks received | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.806392] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 175. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.811407] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 180. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.816001] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 205. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.820605] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 206. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.825100] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 207. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.829034] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 208. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.832944] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 209. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.836835] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 210. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.840719] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 214. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.844627] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 215. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.848511] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 271. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.852405] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 272. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.856249] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 273. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.860091] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 274. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.863936] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 275. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.867754] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 276. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.871603] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 277. | |
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.875487] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 278. | |
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] memberlist: Marking i-06675422c58ce18ce as failed, suspect timeout reached | |
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] serf: EventMemberFailed: i-06675422c58ce18ce 172.16.15.131 | |
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] memberlist: Suspect i-01b6b77ca63499415 has failed, no acks received | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.894421] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1016. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.898375] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 852. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.902204] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 853. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.906010] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 854. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.909805] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 856. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.913607] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 857. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.917583] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 858. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.921533] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 859. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.925438] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 860. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.929378] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 861. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.933288] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 212. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.937194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 213. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.941101] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 214. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.944953] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 217. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.948899] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 218. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.952971] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 219. | |
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.957173] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 551. | |
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] memberlist: Marking i-0280f96a4564c8ccd as failed, suspect timeout reached | |
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] serf: EventMemberFailed: i-0280f96a4564c8ccd 172.16.14.51 | |
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] memberlist: Suspect i-04cf3f2b71f631e5a has failed, no acks received | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.982482] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 941. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.986922] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 453. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.991221] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 454. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.995516] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 186. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.999790] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 203. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.004065] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 213. | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.008321] ena 0000:00:03.0 eth0: The number of lost tx completion is above the threshold (129 > 128). Reset the device | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.026469] ena 0000:00:03.0 eth0: Trigger reset is on | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.028757] ena 0000:00:03.0 eth0: tx_timeout: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.030839] ena 0000:00:03.0 eth0: io_suspend: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.032901] ena 0000:00:03.0 eth0: io_resume: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.034940] ena 0000:00:03.0 eth0: wd_expired: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.037010] ena 0000:00:03.0 eth0: interface_up: 1 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.039134] ena 0000:00:03.0 eth0: interface_down: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.041332] ena 0000:00:03.0 eth0: admin_q_pause: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.043485] ena 0000:00:03.0 eth0: queue_0_tx_cnt: 54560759 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.045893] ena 0000:00:03.0 eth0: queue_0_tx_bytes: 109427757914 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.048495] ena 0000:00:03.0 eth0: queue_0_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.050982] ena 0000:00:03.0 eth0: queue_0_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.053537] ena 0000:00:03.0 eth0: queue_0_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.056212] ena 0000:00:03.0 eth0: queue_0_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.058706] ena 0000:00:03.0 eth0: queue_0_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.061394] ena 0000:00:03.0 eth0: queue_0_tx_napi_comp: 140035994 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.064115] ena 0000:00:03.0 eth0: queue_0_tx_tx_poll: 140035997 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.066821] ena 0000:00:03.0 eth0: queue_0_tx_doorbells: 46403547 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.069507] ena 0000:00:03.0 eth0: queue_0_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.072137] ena 0000:00:03.0 eth0: queue_0_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.074634] ena 0000:00:03.0 eth0: queue_0_rx_cnt: 120515032 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.077147] ena 0000:00:03.0 eth0: queue_0_rx_bytes: 611095745645 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.079817] ena 0000:00:03.0 eth0: queue_0_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.082434] ena 0000:00:03.0 eth0: queue_0_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.084841] ena 0000:00:03.0 eth0: queue_0_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.087511] ena 0000:00:03.0 eth0: queue_0_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.090134] ena 0000:00:03.0 eth0: queue_0_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.092775] ena 0000:00:03.0 eth0: queue_0_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.095317] ena 0000:00:03.0 eth0: queue_0_rx_rx_copybreak_pkt: 18805624 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.098269] ena 0000:00:03.0 eth0: queue_1_tx_cnt: 34543466 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.100759] ena 0000:00:03.0 eth0: queue_1_tx_bytes: 90804732689 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.103396] ena 0000:00:03.0 eth0: queue_1_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.105866] ena 0000:00:03.0 eth0: queue_1_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.108407] ena 0000:00:03.0 eth0: queue_1_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.111057] ena 0000:00:03.0 eth0: queue_1_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.113515] ena 0000:00:03.0 eth0: queue_1_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.116212] ena 0000:00:03.0 eth0: queue_1_tx_napi_comp: 79958517 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.118918] ena 0000:00:03.0 eth0: queue_1_tx_tx_poll: 79958518 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.121528] ena 0000:00:03.0 eth0: queue_1_tx_doorbells: 27611647 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.124197] ena 0000:00:03.0 eth0: queue_1_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.126855] ena 0000:00:03.0 eth0: queue_1_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.129362] ena 0000:00:03.0 eth0: queue_1_rx_cnt: 62402699 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.131839] ena 0000:00:03.0 eth0: queue_1_rx_bytes: 288495267776 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.134539] ena 0000:00:03.0 eth0: queue_1_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.137119] ena 0000:00:03.0 eth0: queue_1_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.139532] ena 0000:00:03.0 eth0: queue_1_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.142167] ena 0000:00:03.0 eth0: queue_1_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.144776] ena 0000:00:03.0 eth0: queue_1_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.147412] ena 0000:00:03.0 eth0: queue_1_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.149994] ena 0000:00:03.0 eth0: queue_1_rx_rx_copybreak_pkt: 12404633 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.152903] ena 0000:00:03.0 eth0: queue_2_tx_cnt: 45547747 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.155387] ena 0000:00:03.0 eth0: queue_2_tx_bytes: 88046639394 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.158037] ena 0000:00:03.0 eth0: queue_2_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.160529] ena 0000:00:03.0 eth0: queue_2_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.163069] ena 0000:00:03.0 eth0: queue_2_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.165704] ena 0000:00:03.0 eth0: queue_2_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.168150] ena 0000:00:03.0 eth0: queue_2_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.170836] ena 0000:00:03.0 eth0: queue_2_tx_napi_comp: 188870192 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.173538] ena 0000:00:03.0 eth0: queue_2_tx_tx_poll: 189138904 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.176199] ena 0000:00:03.0 eth0: queue_2_tx_doorbells: 38910085 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.178879] ena 0000:00:03.0 eth0: queue_2_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.181550] ena 0000:00:03.0 eth0: queue_2_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.184019] ena 0000:00:03.0 eth0: queue_2_rx_cnt: 301701153 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.186534] ena 0000:00:03.0 eth0: queue_2_rx_bytes: 510479422410 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.189203] ena 0000:00:03.0 eth0: queue_2_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.191789] ena 0000:00:03.0 eth0: queue_2_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.194206] ena 0000:00:03.0 eth0: queue_2_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.196846] ena 0000:00:03.0 eth0: queue_2_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.199453] ena 0000:00:03.0 eth0: queue_2_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.202085] ena 0000:00:03.0 eth0: queue_2_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.204630] ena 0000:00:03.0 eth0: queue_2_rx_rx_copybreak_pkt: 10178455 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.207532] ena 0000:00:03.0 eth0: queue_3_tx_cnt: 33417773 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.210018] ena 0000:00:03.0 eth0: queue_3_tx_bytes: 91776922876 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.212668] ena 0000:00:03.0 eth0: queue_3_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.215158] ena 0000:00:03.0 eth0: queue_3_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.217710] ena 0000:00:03.0 eth0: queue_3_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.220355] ena 0000:00:03.0 eth0: queue_3_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.222809] ena 0000:00:03.0 eth0: queue_3_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.225473] ena 0000:00:03.0 eth0: queue_3_tx_napi_comp: 72533647 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.228139] ena 0000:00:03.0 eth0: queue_3_tx_tx_poll: 72533647 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.230776] ena 0000:00:03.0 eth0: queue_3_tx_doorbells: 25996045 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.233450] ena 0000:00:03.0 eth0: queue_3_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.236093] ena 0000:00:03.0 eth0: queue_3_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.238584] ena 0000:00:03.0 eth0: queue_3_rx_cnt: 58050897 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.241073] ena 0000:00:03.0 eth0: queue_3_rx_bytes: 291255301924 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.243745] ena 0000:00:03.0 eth0: queue_3_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.246316] ena 0000:00:03.0 eth0: queue_3_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.248725] ena 0000:00:03.0 eth0: queue_3_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.251406] ena 0000:00:03.0 eth0: queue_3_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.254014] ena 0000:00:03.0 eth0: queue_3_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.256649] ena 0000:00:03.0 eth0: queue_3_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.259243] ena 0000:00:03.0 eth0: queue_3_rx_rx_copybreak_pkt: 12242740 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.262194] ena 0000:00:03.0 eth0: queue_4_tx_cnt: 97417136 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.264660] ena 0000:00:03.0 eth0: queue_4_tx_bytes: 83594367057 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.267301] ena 0000:00:03.0 eth0: queue_4_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.269779] ena 0000:00:03.0 eth0: queue_4_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.272324] ena 0000:00:03.0 eth0: queue_4_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.274984] ena 0000:00:03.0 eth0: queue_4_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.277440] ena 0000:00:03.0 eth0: queue_4_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.280142] ena 0000:00:03.0 eth0: queue_4_tx_napi_comp: 204885202 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.283215] ena 0000:00:03.0 eth0: queue_4_tx_tx_poll: 204885203 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.286199] ena 0000:00:03.0 eth0: queue_4_tx_doorbells: 91866448 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.289228] ena 0000:00:03.0 eth0: queue_4_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.292209] ena 0000:00:03.0 eth0: queue_4_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.294725] ena 0000:00:03.0 eth0: queue_4_rx_cnt: 146586586 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.297239] ena 0000:00:03.0 eth0: queue_4_rx_bytes: 656436382527 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.299929] ena 0000:00:03.0 eth0: queue_4_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.302513] ena 0000:00:03.0 eth0: queue_4_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.304936] ena 0000:00:03.0 eth0: queue_4_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.307582] ena 0000:00:03.0 eth0: queue_4_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.310194] ena 0000:00:03.0 eth0: queue_4_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.312846] ena 0000:00:03.0 eth0: queue_4_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.315409] ena 0000:00:03.0 eth0: queue_4_rx_rx_copybreak_pkt: 27715663 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.318308] ena 0000:00:03.0 eth0: queue_5_tx_cnt: 54269388 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.320797] ena 0000:00:03.0 eth0: queue_5_tx_bytes: 95150507647 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.323441] ena 0000:00:03.0 eth0: queue_5_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.325911] ena 0000:00:03.0 eth0: queue_5_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.328456] ena 0000:00:03.0 eth0: queue_5_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.331109] ena 0000:00:03.0 eth0: queue_5_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.333563] ena 0000:00:03.0 eth0: queue_5_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.336240] ena 0000:00:03.0 eth0: queue_5_tx_napi_comp: 111649875 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.338951] ena 0000:00:03.0 eth0: queue_5_tx_tx_poll: 111649875 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.341601] ena 0000:00:03.0 eth0: queue_5_tx_doorbells: 48317709 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.344272] ena 0000:00:03.0 eth0: queue_5_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.346918] ena 0000:00:03.0 eth0: queue_5_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.349396] ena 0000:00:03.0 eth0: queue_5_rx_cnt: 77300362 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.351895] ena 0000:00:03.0 eth0: queue_5_rx_bytes: 326773596123 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.354578] ena 0000:00:03.0 eth0: queue_5_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.357156] ena 0000:00:03.0 eth0: queue_5_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.359595] ena 0000:00:03.0 eth0: queue_5_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.362279] ena 0000:00:03.0 eth0: queue_5_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.364932] ena 0000:00:03.0 eth0: queue_5_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.367617] ena 0000:00:03.0 eth0: queue_5_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.370200] ena 0000:00:03.0 eth0: queue_5_rx_rx_copybreak_pkt: 18377680 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.373122] ena 0000:00:03.0 eth0: queue_6_tx_cnt: 34298113 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.375628] ena 0000:00:03.0 eth0: queue_6_tx_bytes: 64209391587 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.378292] ena 0000:00:03.0 eth0: queue_6_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.380798] ena 0000:00:03.0 eth0: queue_6_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.383413] ena 0000:00:03.0 eth0: queue_6_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.386087] ena 0000:00:03.0 eth0: queue_6_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.388576] ena 0000:00:03.0 eth0: queue_6_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.391285] ena 0000:00:03.0 eth0: queue_6_tx_napi_comp: 101257578 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.394031] ena 0000:00:03.0 eth0: queue_6_tx_tx_poll: 101257578 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.396711] ena 0000:00:03.0 eth0: queue_6_tx_doorbells: 30137999 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.399417] ena 0000:00:03.0 eth0: queue_6_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.402083] ena 0000:00:03.0 eth0: queue_6_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.404597] ena 0000:00:03.0 eth0: queue_6_rx_cnt: 87915741 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.407142] ena 0000:00:03.0 eth0: queue_6_rx_bytes: 352158714788 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.409846] ena 0000:00:03.0 eth0: queue_6_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.412446] ena 0000:00:03.0 eth0: queue_6_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.414918] ena 0000:00:03.0 eth0: queue_6_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.417613] ena 0000:00:03.0 eth0: queue_6_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.420262] ena 0000:00:03.0 eth0: queue_6_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.422927] ena 0000:00:03.0 eth0: queue_6_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.425486] ena 0000:00:03.0 eth0: queue_6_rx_rx_copybreak_pkt: 20796029 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.428442] ena 0000:00:03.0 eth0: queue_7_tx_cnt: 52485410 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.430976] ena 0000:00:03.0 eth0: queue_7_tx_bytes: 67669867898 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.433672] ena 0000:00:03.0 eth0: queue_7_tx_queue_stop: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.436175] ena 0000:00:03.0 eth0: queue_7_tx_queue_wakeup: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.438750] ena 0000:00:03.0 eth0: queue_7_tx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.441428] ena 0000:00:03.0 eth0: queue_7_tx_linearize: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.443895] ena 0000:00:03.0 eth0: queue_7_tx_linearize_failed: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.446612] ena 0000:00:03.0 eth0: queue_7_tx_napi_comp: 153127460 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.449352] ena 0000:00:03.0 eth0: queue_7_tx_tx_poll: 153127463 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.452053] ena 0000:00:03.0 eth0: queue_7_tx_doorbells: 48526263 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.454758] ena 0000:00:03.0 eth0: queue_7_tx_prepare_ctx_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.457433] ena 0000:00:03.0 eth0: queue_7_tx_bad_req_id: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.459963] ena 0000:00:03.0 eth0: queue_7_rx_cnt: 140231578 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.462513] ena 0000:00:03.0 eth0: queue_7_rx_bytes: 674949155576 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.465205] ena 0000:00:03.0 eth0: queue_7_rx_refil_partial: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.467806] ena 0000:00:03.0 eth0: queue_7_rx_bad_csum: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.470275] ena 0000:00:03.0 eth0: queue_7_rx_page_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.472931] ena 0000:00:03.0 eth0: queue_7_rx_skb_alloc_fail: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.475581] ena 0000:00:03.0 eth0: queue_7_rx_dma_mapping_err: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.478254] ena 0000:00:03.0 eth0: queue_7_rx_bad_desc_num: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.480832] ena 0000:00:03.0 eth0: queue_7_rx_rx_copybreak_pkt: 24570422 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.483797] ena 0000:00:03.0 eth0: ena_admin_q_aborted_cmd: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.486375] ena 0000:00:03.0 eth0: ena_admin_q_submitted_cmd: 44 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.489046] ena 0000:00:03.0 eth0: ena_admin_q_completed_cmd: 44 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.505136] ena 0000:00:03.0 eth0: ena_admin_q_out_of_space: 0 | |
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.507742] ena 0000:00:03.0 eth0: ena_admin_q_no_completion: 0 | |
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] memberlist: Marking i-03c4d88e2854d047b as failed, suspect timeout reached | |
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] serf: EventMemberFailed: i-03c4d88e2854d047b 172.16.16.159 | |
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] memberlist: Suspect i-0caf05d83d7af272a has failed, no acks received | |
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] memberlist: Marking i-0af21f21158ce7c92 as failed, suspect timeout reached | |
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] serf: EventMemberFailed: i-0af21f21158ce7c92 172.16.14.227 | |
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] memberlist: Suspect i-0d2a89cac90d1b339 has failed, no acks received | |
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] memberlist: Marking i-0f14d98ae903e7de2 as failed, suspect timeout reached | |
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] serf: EventMemberFailed: i-0f14d98ae903e7de2 172.16.14.67 | |
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] consul: removing server i-0f14d98ae903e7de2 (Addr: 172.16.14.67:8300) (DC: dc1) | |
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] memberlist: Suspect i-07d37224cc2e12f36 has failed, no acks received | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.670560] ena: Reset indication didn't turn off | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.673353] ena 0000:00:03.0: Device reset failed | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.676424] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 0 idx 0x3e1 | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.682126] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 1 idx 0x329 | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.687797] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 2 idx 0xab | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.693431] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 3 idx 0x20d | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.699073] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 4 idx 0x389 | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.704681] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 5 idx 0x1a0 | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.710284] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 6 idx 0x15 | |
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.715969] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 7 idx 0xed | |
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [ERR] agent: coordinate update error: rpc error: failed to get conn: dial tcp 172.16.14.66:8300: getsockopt: no route to host | |
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] memberlist: Marking i-02b2968ab933cf778 as failed, suspect timeout reached | |
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] serf: EventMemberFailed: i-02b2968ab933cf778 172.16.15.32 | |
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] memberlist: Suspect i-0edd0cf9d6f372a8c has failed, no acks received | |
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] memberlist: Marking i-055abe2ce793ae2c8 as failed, suspect timeout reached | |
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] serf: EventMemberFailed: i-055abe2ce793ae2c8 172.16.14.149 | |
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] memberlist: Suspect i-001ba44e32334f963 has failed, no acks received | |
-- snip -- lost logs, weren't stored in /var/log/messages, but recovered from console: -- | |
_bm_flush+0x1c/0x20 [dm_persistent_data] | |
[365557.729461] [<ffffffffa08bffc3>] dm_tm_commit+0x33/0x40 [dm_persistent_data] | |
[365557.735149] [<ffffffffa08d8d39>] __commit_transaction+0x359/0x360 [dm_thin_pool] | |
[365557.740735] [<ffffffffa08d9f47>] dm_pool_commit_metadata+0x37/0x60 [dm_thin_pool] | |
[365557.746227] [<ffffffffa08d3967>] commit+0x27/0x90 [dm_thin_pool] | |
[365557.750505] [<ffffffffa08d3cf2>] pool_status+0x322/0x6b0 [dm_thin_pool] | |
[365557.755227] [<ffffffff811cd5c6>] ? __memcg_kmem_charge+0x46/0x90 | |
[365557.759749] [<ffffffffa0889c96>] retrieve_status+0xa6/0x1b0 [dm_mod] | |
[365557.764474] [<ffffffffa088ae14>] table_status+0x54/0x80 [dm_mod] | |
[365557.768939] [<ffffffffa088adc0>] ? dm_get_live_or_inactive_table.isra.3+0x30/0x30 [dm_mod] | |
[365557.775238] [<ffffffffa088aae6>] ctl_ioctl+0x246/0x4b0 [dm_mod] | |
[365557.778569] [<ffffffff8107932b>] ? recalc_sigpending+0x1b/0x50 | |
[365557.781825] [<ffffffffa088ad63>] dm_ctl_ioctl+0x13/0x20 [dm_mod] | |
[365557.785157] [<ffffffff811e772d>] do_vfs_ioctl+0x2cd/0x4b0 | |
[365557.788226] [<ffffffff8107c27b>] ? __set_current_blocked+0x4b/0x60 | |
[365557.791642] [<ffffffff811e7989>] SyS_ioctl+0x79/0x90 | |
[365557.794518] [<ffffffff8107c4a7>] ? SyS_rt_sigprocmask+0x67/0xb0 | |
[365557.797936] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71 | |
[365557.801516] INFO: task rs:main Q:Reg:3401 blocked for more than 120 seconds. | |
[365557.805308] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365557.808942] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365557.813403] rs:main Q:Reg D ffff881e6a3bf968 0 3401 1 0x00000000 | |
[365557.817403] ffff881e6a3bf968 ffff881e77ab0000 ffff881e71ddba80 ffff881e6a3c0000 | |
[365557.822043] 0000000000000000 7fffffffffffffff ffff881efff92a08 ffffffff814ecd30 | |
[365557.840036] ffff881e6a3bf980 ffffffff814ec5f5 ffff881e84695340 ffff881e6a3bfa20 | |
[365557.844684] Call Trace: | |
[365557.846326] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365557.850013] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365557.853482] [<ffffffff814eef01>] schedule_timeout+0x231/0x2b0 | |
[365557.857646] [<ffffffff8142c315>] ? netif_rx_internal+0x35/0xf0 | |
[365557.861887] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365557.866587] [<ffffffff810d2f7a>] ? ktime_get+0x3a/0x90 | |
[365557.870317] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365557.874015] [<ffffffff814ebcf6>] io_schedule_timeout+0xa6/0x110 | |
[365557.878223] [<ffffffff814ecd4b>] bit_wait_io+0x1b/0x60 | |
[365557.881947] [<ffffffff814ec990>] __wait_on_bit+0x60/0x90 | |
[365557.885801] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365557.889444] [<ffffffff814eca32>] out_of_line_wait_on_bit+0x72/0x80 | |
[365557.893829] [<ffffffff810acf20>] ? autoremove_wake_function+0x40/0x40 | |
[365557.898380] [<ffffffffa00ccdb9>] do_get_write_access+0x279/0x4c0 [jbd2] | |
[365557.903081] [<ffffffff8120918e>] ? __find_get_block+0xae/0x120 | |
[365557.907299] [<ffffffffa00cd031>] jbd2_journal_get_write_access+0x31/0x60 [jbd2] | |
[365557.912781] [<ffffffffa0136511>] __ext4_journal_get_write_access+0x31/0x70 [ext4] | |
[365557.918323] [<ffffffffa01080da>] ext4_reserve_inode_write+0x6a/0x90 [ext4] | |
[365557.923190] [<ffffffffa010b888>] ? ext4_dirty_inode+0x48/0x70 [ext4] | |
[365557.927728] [<ffffffffa010813f>] ext4_mark_inode_dirty+0x3f/0x1f0 [ext4] | |
[365557.932491] [<ffffffffa010b888>] ext4_dirty_inode+0x48/0x70 [ext4] | |
[365557.936911] [<ffffffff812009e1>] __mark_inode_dirty+0x181/0x370 | |
[365557.941163] [<ffffffff811ee887>] generic_update_time+0x87/0xd0 | |
[365557.945363] [<ffffffff811edfd9>] file_update_time+0x99/0xf0 | |
[365557.949398] [<ffffffff810dc784>] ? get_futex_key+0x194/0x270 | |
[365557.953487] [<ffffffff81163f1c>] __generic_file_write_iter+0x9c/0x1e0 | |
[365557.958070] [<ffffffff810dcd41>] ? futex_wake+0x81/0x150 | |
[365557.961918] [<ffffffffa00feb76>] ext4_file_write_iter+0xf6/0x360 [ext4] | |
[365557.966561] [<ffffffff810dec50>] ? do_futex+0x100/0xad0 | |
[365557.970384] [<ffffffff81411c68>] ? sock_sendmsg+0x38/0x50 | |
[365557.974311] [<ffffffff814120e9>] ? SYSC_sendto+0x139/0x170 | |
[365557.978286] [<ffffffff811d4dba>] __vfs_write+0xaa/0xe0 | |
[365557.982034] [<ffffffff811d53e2>] vfs_write+0xa2/0x1a0 | |
[365557.985716] [<ffffffff811d60e6>] SyS_write+0x46/0xa0 | |
[365557.989368] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71 | |
[365557.993848] INFO: task agent:43861 blocked for more than 120 seconds. | |
[365557.998607] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.003736] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.009987] agent D ffff880c4fdf7860 0 43861 13580 0x00000000 | |
[365558.015615] ffff880c4fdf7860 ffff881e77ab0000 ffff881d07de57c0 ffff880c4fdf8000 | |
[365558.022129] ffff881e70ff6558 ffff881e70ff6570 ffff880c4fdf7990 0000000000000000 | |
[365558.028637] ffff880c4fdf7878 ffffffff814ec5f5 ffff881d07de57c0 ffff880c4fdf78c8 | |
[365558.035142] Call Trace: | |
[365558.037302] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.041067] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120 | |
[365558.045637] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30 | |
[365558.050637] [<ffffffff814ee2d7>] ? down_read+0x17/0x20 | |
[365558.054694] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool] | |
[365558.060370] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool] | |
[365558.065245] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod] | |
[365558.069759] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod] | |
[365558.075500] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod] | |
[365558.080323] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0 | |
[365558.084977] [<ffffffff812aa1e8>] submit_bio+0x68/0x130 | |
[365558.089021] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4] | |
[365558.094118] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120 | |
[365558.098800] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4] | |
[365558.103485] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200 | |
[365558.108597] [<ffffffff81164714>] filemap_fault+0x334/0x3e0 | |
[365558.113007] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4] | |
[365558.118027] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70 | |
[365558.122134] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870 | |
[365558.126725] [<ffffffff810b3501>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 | |
[365558.133188] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0 | |
[365558.137660] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365558.142909] [<ffffffff8105efc2>] do_page_fault+0x22/0x30 | |
[365558.147324] [<ffffffff814f2058>] page_fault+0x28/0x30 | |
[365558.151830] INFO: task consul:15336 blocked for more than 120 seconds. | |
[365558.157299] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.162777] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.169393] consul D ffff881e4f6a3860 0 15336 15293 0x00000000 | |
[365558.175451] ffff881e4f6a3860 ffffffff81a11500 ffff881e49d2ba80 ffff881e4f6a4000 | |
[365558.182413] ffff881e70ff6558 ffff881e70ff6570 ffff881e4f6a3990 0000000000000000 | |
[365558.189346] ffff881e4f6a3878 ffffffff814ec5f5 ffff881e49d2ba80 ffff881e4f6a38c8 | |
[365558.196075] Call Trace: | |
[365558.198259] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.202177] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120 | |
[365558.206997] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30 | |
[365558.212061] [<ffffffff814ee2d7>] ? down_read+0x17/0x20 | |
[365558.216155] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool] | |
[365558.221855] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool] | |
[365558.226781] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod] | |
[365558.231363] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod] | |
[365558.237145] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod] | |
[365558.241819] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0 | |
[365558.246294] [<ffffffff812aa1e8>] submit_bio+0x68/0x130 | |
[365558.250354] [<ffffffff8117028e>] ? lru_cache_add+0xe/0x10 | |
[365558.254594] [<ffffffff81162c1d>] ? add_to_page_cache_lru+0x4d/0x80 | |
[365558.259375] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4] | |
[365558.264526] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120 | |
[365558.269243] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4] | |
[365558.273910] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200 | |
[365558.278920] [<ffffffff81164714>] filemap_fault+0x334/0x3e0 | |
[365558.283247] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4] | |
[365558.288152] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70 | |
[365558.292114] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870 | |
[365558.296529] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365558.301583] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0 | |
[365558.306036] [<ffffffff8105efc2>] do_page_fault+0x22/0x30 | |
[365558.310156] [<ffffffff814f2058>] page_fault+0x28/0x30 | |
[365558.314101] INFO: task consul:15339 blocked for more than 120 seconds. | |
[365558.318987] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.324015] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.330096] consul D ffff881e4f69bd98 0 15339 15293 0x00000000 | |
[365558.335620] ffff881e4f69bd98 ffff881e77a9ba80 ffff881e51918000 ffff881e4f69c000 | |
[365558.341683] ffff881e749dbc80 ffff881e749dbc68 ffffffff00000000 ffffffff00000002 | |
[365558.347735] ffff881e4f69bdb0 ffffffff814ec5f5 ffff881e51918000 ffff881e4f69be30 | |
[365558.354237] Call Trace: | |
[365558.356452] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.360397] [<ffffffff814ee9fa>] rwsem_down_write_failed+0x1da/0x320 | |
[365558.365314] [<ffffffff810a03b5>] ? pick_next_entity+0xa5/0x160 | |
[365558.369767] [<ffffffff812dac03>] call_rwsem_down_write_failed+0x13/0x20 | |
[365558.374620] [<ffffffff814ee304>] ? down_write+0x24/0x40 | |
[365558.378673] [<ffffffff8117d53b>] vm_mmap_pgoff+0x6b/0xc0 | |
[365558.382693] [<ffffffff811955b3>] SyS_mmap_pgoff+0x103/0x260 | |
[365558.386914] [<ffffffff810d240a>] ? ktime_get_ts64+0x4a/0xe0 | |
[365558.391291] [<ffffffff8101b292>] SyS_mmap+0x22/0x30 | |
[365558.395186] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71 | |
[365558.400120] INFO: task consul:15340 blocked for more than 120 seconds. | |
[365558.405105] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.410223] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.416474] consul D ffff881e4f657860 0 15340 15293 0x00000000 | |
[365558.422156] ffff881e4f657860 ffff881e77aeba80 ffff881e69c9ba80 ffff881e4f658000 | |
[365558.428666] ffff881e70ff6558 ffff881e70ff6570 ffff881e4f657990 0000000000000000 | |
[365558.435155] ffff881e4f657878 ffffffff814ec5f5 ffff881e69c9ba80 ffff881e4f6578c8 | |
[365558.441440] Call Trace: | |
[365558.443518] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.447315] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120 | |
[365558.452132] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30 | |
[365558.457138] [<ffffffff814ee2d7>] ? down_read+0x17/0x20 | |
[365558.461221] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool] | |
[365558.466932] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool] | |
[365558.471825] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod] | |
[365558.476386] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod] | |
[365558.482138] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod] | |
[365558.486966] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0 | |
[365558.491617] [<ffffffff812aa1e8>] submit_bio+0x68/0x130 | |
[365558.495686] [<ffffffff8117028e>] ? lru_cache_add+0xe/0x10 | |
[365558.499933] [<ffffffff81162c1d>] ? add_to_page_cache_lru+0x4d/0x80 | |
[365558.504756] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4] | |
[365558.509903] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120 | |
[365558.514668] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4] | |
[365558.519387] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200 | |
[365558.524455] [<ffffffff81164714>] filemap_fault+0x334/0x3e0 | |
[365558.528782] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4] | |
[365558.533708] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70 | |
[365558.537682] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870 | |
[365558.541997] [<ffffffff8147670a>] ? tcp_poll+0x6a/0x240 | |
[365558.545909] [<ffffffff810b3501>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 | |
[365558.552179] [<ffffffff8121a631>] ? SyS_epoll_ctl+0xd1/0xca0 | |
[365558.556523] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0 | |
[365558.574325] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365558.579398] [<ffffffff8105efc2>] do_page_fault+0x22/0x30 | |
[365558.583584] [<ffffffff814f2058>] page_fault+0x28/0x30 | |
[365558.587630] INFO: task jbd2/dm-10-8:16273 blocked for more than 120 seconds. | |
[365558.593052] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.598254] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.604597] jbd2/dm-10-8 D ffff881df5e17b08 0 16273 2 0x00000000 | |
[365558.610381] ffff881df5e17b08 ffffffff81a11500 ffff881e51bcba80 ffff881df5e18000 | |
[365558.617026] 0000000000000000 7fffffffffffffff ffff881efff94478 ffffffff814ecd30 | |
[365558.623543] ffff881df5e17b20 ffffffff814ec5f5 ffff881e84615340 ffff881df5e17bb8 | |
[365558.630097] Call Trace: | |
[365558.632304] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365558.636368] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.640199] [<ffffffff814eef01>] schedule_timeout+0x231/0x2b0 | |
[365558.644499] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365558.649548] [<ffffffff810d2f7a>] ? ktime_get+0x3a/0x90 | |
[365558.653643] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365558.657688] [<ffffffff814ebcf6>] io_schedule_timeout+0xa6/0x110 | |
[365558.662327] [<ffffffff814ecd4b>] bit_wait_io+0x1b/0x60 | |
[365558.666432] [<ffffffff814ec990>] __wait_on_bit+0x60/0x90 | |
[365558.670655] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50 | |
[365558.674676] [<ffffffff814eca32>] out_of_line_wait_on_bit+0x72/0x80 | |
[365558.679505] [<ffffffff810acf20>] ? autoremove_wake_function+0x40/0x40 | |
[365558.684492] [<ffffffff812088c7>] __wait_on_buffer+0x27/0x30 | |
[365558.688881] [<ffffffffa00cf2f6>] jbd2_journal_commit_transaction+0xf66/0x1bb0 [jbd2] | |
[365558.695040] [<ffffffffa00d3c90>] kjournald2+0x120/0x340 [jbd2] | |
[365558.699622] [<ffffffff810acee0>] ? prepare_to_wait_event+0xf0/0xf0 | |
[365558.704476] [<ffffffffa00d3b70>] ? commit_timeout+0x10/0x10 [jbd2] | |
[365558.709312] [<ffffffff8108a7f9>] kthread+0xc9/0xe0 | |
[365558.713179] [<ffffffff8108a730>] ? kthread_park+0x60/0x60 | |
[365558.717631] [<ffffffff814f02cf>] ret_from_fork+0x3f/0x70 | |
[365558.721865] [<ffffffff8108a730>] ? kthread_park+0x60/0x60 | |
[365558.726156] INFO: task java:16402 blocked for more than 120 seconds. | |
[365558.731055] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.736209] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.742194] java D ffff881e6bc37e08 0 16402 16351 0x00000000 | |
[365558.747635] ffff881e6bc37e08 ffff881e77a99d40 ffff881e61288000 ffff881e6bc38000 | |
[365558.754131] ffff881e75cd9340 ffff881e75cd9328 ffffffff00000000 ffffffff00000001 | |
[365558.760631] ffff881e6bc37e20 ffffffff814ec5f5 ffff881e61288000 ffff881e6bc37ea0 | |
[365558.767086] Call Trace: | |
[365558.769269] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.773139] [<ffffffff814ee9fa>] rwsem_down_write_failed+0x1da/0x320 | |
[365558.778030] [<ffffffff812dac03>] call_rwsem_down_write_failed+0x13/0x20 | |
[365558.783083] [<ffffffff814ee304>] ? down_write+0x24/0x40 | |
[365558.787210] [<ffffffff811987d1>] SyS_mprotect+0xc1/0x240 | |
[365558.791414] [<ffffffff810cefd1>] ? posix_ktime_get_ts+0x11/0x20 | |
[365558.796010] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71 | |
[365558.800940] INFO: task java:16445 blocked for more than 120 seconds. | |
[365558.805799] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1 | |
[365558.810897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. | |
[365558.817115] java D ffff881debbd3e10 0 16445 16351 0x00000000 | |
[365558.822756] ffff881debbd3e10 ffffffff81a11500 ffff881e6bd2ba80 ffff881debbd4000 | |
[365558.829278] ffff881e75cd9328 ffff881e75cd9340 ffff881debbd3f58 ffff881e6bd2ba80 | |
[365558.835788] ffff881debbd3e28 ffffffff814ec5f5 ffff881e6bd2ba80 ffff881debbd3e78 | |
[365558.842069] Call Trace: | |
[365558.844150] [<ffffffff814ec5f5>] schedule+0x35/0x80 | |
[365558.847945] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120 | |
[365558.852755] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30 | |
[365558.857772] [<ffffffff814ee2d7>] ? down_read+0x17/0x20 | |
[365558.861861] [<ffffffff8105ef39>] __do_page_fault+0x389/0x3f0 | |
[365558.866301] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20 | |
[365558.871390] [<ffffffff8105efc2>] do_page_fault+0x22/0x30 | |
[365558.875588] [<ffffffff814f2058>] page_fault+0x28/0x30 | |
[365665.548153] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 858. | |
[365703.557562] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 229. | |
[365704.561666] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 567. | |
[365704.566122] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1021. | |
[365705.569628] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 647. | |
[365705.573997] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 648. | |
[365706.577713] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 230. | |
[365707.581763] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 876. | |
[365708.585837] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 649. | |
[365709.593807] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 809. | |
[365709.598182] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 231. | |
[365711.601901] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 650. | |
[365712.605951] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 810. | |
[365714.610007] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 651. | |
[365724.614383] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 232. | |
[365730.618610] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 87. | |
[365737.622898] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 872. | |
[365739.626961] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 839. | |
[365742.631063] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 88. | |
[365742.635070] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 89. | |
[365742.639040] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 840. | |
[365742.643015] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 841. | |
[365742.647046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 811. | |
[365742.651030] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 812. | |
[365742.655302] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 813. | |
[365742.659545] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 814. | |
[365742.663826] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 815. | |
[365742.668055] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 233. | |
[365743.671120] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1022. | |
[365743.675192] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1023. | |
[365744.683098] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 652. | |
[365744.687117] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 653. | |
[365746.691242] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 0. | |
[365747.695279] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 654. | |
[365748.699313] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 842. | |
[365749.703345] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 568. | |
[365749.707929] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 569. | |
[365749.712383] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1. | |
[365749.716774] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 2. | |
[365763.723946] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 816. | |
[365894.752829] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 899. | |
[366004.772985] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 14. | |
[366006.777073] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 849. | |
[366007.781113] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 15. | |
[366007.785109] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 911. | |
[366012.789315] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 850. | |
[366013.793349] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 16. | |
[366025.797802] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 919. | |
[366027.801877] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 248. | |
[366030.805996] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 851. | |
[366031.810023] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 586. | |
[366031.814046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 587. | |
[366044.830516] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 669. | |
[366060.835104] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 102. | |
[366060.839915] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 822. | |
[366061.843146] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 588. | |
[366063.847294] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 249. | |
[366072.863541] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 823. | |
[366074.863683] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 924. | |
[366100.868637] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 589. | |
[366103.872750] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 17. | |
[366103.876745] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 18. | |
[366105.880828] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 824. | |
[366105.884859] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 825. | |
[366109.888993] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 23. | |
[366117.893301] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 827. | |
[366283.903580] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 962. | |
[366305.908408] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 682. | |
[366313.912713] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 596. | |
[366313.917037] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 597. | |
[366314.920749] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 683. | |
[366377.372692] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null) | |
[366377.602016] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null) | |
[366377.782775] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null) | |
[391563.883504] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null) | |
[391564.067606] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null) | |
[391564.222983] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null) | |
[395856.136657] audit: type=1305 audit(1495033308.746:1058): audit_pid=0 old=3379 auid=4294967295 ses=4294967295 res=1 | |
[395856.229645] audit: type=1305 audit(1495033308.838:1059): audit_enabled=0 old=1 auid=500 ses=91 res=1 | |
[395856.251799] audit: type=1305 audit(1495033308.862:1060): audit_pid=109278 old=0 auid=500 ses=91 res=1 | |
/dev/fd/10: line 1: /sbin/plymouthd: No such file or directory | |
initctl: Event failed | |
Stopping docker: [395889.810592] docker0: port 1(vethd31d3a8) entered disabled state | |
[395889.813485] veth6d92c3f: renamed from eth0 | |
[395889.874456] docker0: port 1(vethd31d3a8) entered disabled state | |
[395889.880996] device vethd31d3a8 left promiscuous mode | |
[395889.883901] docker0: port 1(vethd31d3a8) entered disabled state | |
[ OK ] | |
Stopping sshd: [ OK ] | |
Shutting down sm-client: [ OK ] | |
Shutting down sendmail: [ OK ] | |
Stopping crond: [ OK ] | |
Shutting down ntpd: [ OK ] | |
Stopping block device availability: Deactivating block devices: | |
[MD]: deactivating raid0 device md0... skipping | |
[LVM]: deactivating Volume Group docker... done | |
[ OK ] | |
Stopping auditd: [395901.470975] audit: type=1305 audit(1495033354.081:1097): audit_pid=0 old=109278 auid=500 ses=91 res=1 | |
[ OK ] | |
[395901.565585] audit: type=1305 audit(1495033354.177:1098): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1 | |
Shutting down system logger: [ OK ] | |
Shutting down interface eth0: [ OK ] | |
Shutting down loopback interface: [ OK ] | |
Stopping cgconfig service: [ OK ] | |
Stopping monitoring for VG docker: WARNING: Failed to connect to lvmetad. Falling back to device scanning. | |
WARNING: Failed to connect to lvmetad. Falling back to device scanning. | |
[ OK ] | |
Stopping rngd: [ OK ] | |
Sending all processes the TERM signal... init: ecs post-stop process (114746) killed by TERM signal | |
[ OK ] | |
Sending all processes the KILL signal... [ OK ] | |
Saving random seed: [ OK ] | |
Unmounting file systems: [ OK ] | |
init: Re-executing /sbin/init | |
[395907.704988] EXT4-fs (xvda1): re-mounted. Opts: (null) | |
Please stand by while rebooting the system... | |
[395908.933545] xenbus: xenbus_dev_shutdown: device/pci/0: Initialising != Connected, skipping | |
[395908.938776] reboot: Restarting system | |
[395908.941265] reboot: machine restart |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment