Lately from time to time on one of my Zimbra servers I was getting A network service error has occurred error message. Restarting the Zimbra service was fixing the issue but it was time to find out why this is happening. This is the the screen I’m getting and nothing in my log files:  Now my problem was that I’m using a nginx proxy to route the traffic to my server and I changed the proxy server IP address and forgot about the Zimbra DosFilter Throttling Mechanism The denial-of-service filter or DoSFilter was added to the mailbox server in ZCS 8.0 to throttle clients sending a large number of requests over a very short period of time. The DoSFilter is applied to all requests for service, mailbox and admin. I spent all day working on this problem with help from zimbra IRC yesterday with no resolution. We made sure the proxy was properly configured but it seems to intermittently have issues connecting to mailboxd. I thought that zimbra-proxy was required for 8.7+ as well, but it seem that it is only required that it is installed, not that it is enabled. This bug is what is driving the requirement in 8.7+, but it doesn't actually say why it is required - does anyone know why?I've researched this problem quite extensively and cannot find any more debug information for why it was happening. There's not much in the error logs, just this: /opt/zimbra/log/nginx.log: 2016/08/29 13:50:12 [error] 28997 /opt/zimbra/log/mailbox.log: 2016-08-29 13:50:22,421 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving VirtualConversation (id=-368187) to Folder Trash (id=3). Affected message ids: 368187. 2016-08-29 13:50:22,423 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving VirtualConversation (id=-368186) to Folder Trash (id=3). Affected message ids: 368186. 2016-08-29 13:50:22,425 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving Conversation (id=366405) to Folder Trash (id=3). Affected message ids: 366236,366431. 2016-08-29 13:50:22,427 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving VirtualConversation (id=-366239) to Folder Trash (id=3). Affected message ids: 366239. 2016-08-29 13:50:22,429 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving VirtualConversation (id=-367760) to Folder Trash (id=3). Affected message ids: 367760. 2016-08-29 13:50:22,430 INFO [qtp1684106402-1162:http://zimbra.example.com/service/soap/ ... ionRequest] [[email protected];mid=82;ip=192.168.1.10;port=45304;ua=ZimbraWebClient - GC52 (Win)/8.7.0_GA_1659;] mailop - Moving VirtualConversation (id=-367761) to Folder Trash (id=3). Affected message ids: 367761. talks about what the 502 error means in regards to the proxy and mailboxd communcation: 2. 502 Bad Gateway: The server was acting as a gateway or proxy and received an invalid response from the upstream server. This is a single-node instance, so the proxy and mailboxd are on the same server. The same action will fail with the 502 error, but then if you try it again (e.g. click on the same folder or message again), it will then work. This type of intermittent behavior makes me think it's some kind of throttling or timeout problem rather than a complete misconfiguration (which I would expect to work completely or not at all). |
