Previous incidents

We have resolved the issue.

cPanel has been moved back to it's home IP address after running on the backup for approximately 3 hours.

RCA will follow, however we believe this to be a fault of the NBN NTDs, as we have experienced this kind of behaviour before where a single IP address gets "blackholed" until the NTD is forcefully rebooted.

In the interim, we will continue to monitor but will be working on an automation to keep this kind of downtime to a minimum. We will also be providing a...

We've resolved the issue, and all systems have recovered.

RCA; a severe lack of sleep and lack of caffeine. Don't press shutdown and ignore the warnings telling you you're shutting down a node instead of a VM.

Sorry about that, folks!

We're back on track. Systems have returned to normal operation!

Thanks, and sorry again!

Our provider has implemented mitigations to their AS, which has resulted in services coming back online.

We're going to continue to monitor over the coming 24 hours, but as it stands, all systems are running as normal now.