A syntax error in our GDNSD configuration caused the wikis to go down for 7 minutes.
- 17:35 Southparkfan: commited 14afa4fd3bf1362e320e11c97ccce025a0f88982 "Add DNS for publictestwiki.com"
- 17:40 Southparkfan: puppet runs automatically. At this point GDNSD crashed due to the syntax error.
- 17:42 Southparkfan: Icinga alerts on IRC that DNS crashed
- 17:47 Southparkfan: revert DNS config to good version
- 17:47 Southparkfan: deployed change on DNS servers, site confirmed back online
Southparkfan commited a file to GDNSD that contained a syntax error, and he takes full responsibility for this outage. It has made clear that we should treat DNS even more carefully!
- #64 "GDNSD should not crash upon reload if config contains syntax errors" - Done