Email me when new posts are made to this blog

Australian servers

Written by Courtney Brown on December 20th, 2012.      0 comments

Update 11.08am: Orcon, who hosts some of our servers, is having an issue routing customers using certain ISPs to their websites, which is directly affecting our websites. They're working on fixing this now

We're currently experiencing issues with our Australian servers. It looks like this is only interrupting some Australian websites, not all of them. We're currently in communication with our provider to get this fixed as soon as possible.


 

Australian server issue - some websites down

Written by Courtney Brown on December 11th, 2012.      0 comments

There is an issue with our Australian servers, which may cause some websites to be offline. Our systems analyst is working on this now and we're hoping they will be up again very soon. We apologize for any inconvenience, please bear with us.

UPDATE - the issue has been fixed by Orcon. They were running an upgrade which had some unintended affects which have since been resolved. Thank you for your understanding.
 

Moving offices - possible interruptions with phoning the team

Written by on December 7th, 2012.      0 comments

Hi everyone, we're packing up the office in preparation for moving after 5pm and over the weekend. This won't affect any websites as these are hosted offsite, however our phone service may be interrupted - if you have trouble getting through to the Support team, please email support@zeald.com or try again after a few minutes.
 

Fire alarm interruption

Written by on November 30th, 2012.      0 comments

Hi everyone, we apologise if you've been unable to get through to our team as we've just had a fire alarm go off in the building. Everything is back to normal now.
 

Australian Hosted Websites Issue

Written by on November 18th, 2012.      0 comments

Zeald Australian hosted websites are having a connectivity issue with our provider. We have notified our provider and hopefully they can fix the problem and have it up and running as soon as possible. We apologize for the downtime

Update from Provider;

In the midst of severe thunderstorms our data centre lost mains power at approx. 10:33am AEST (GMT+10) on Sunday 18'th of November, 2012. Due to a technical fault with the power switching equipment the generator did not automatically start and the UPS’s protecting the data centre discharged, resulting in a loss of power to all systems.

Staff immediately responded to the outage manually starting the backup generator and then started working on getting all systems back online. Most customer services had been restored by approx 12:00pm with some services taking longer to get back online.

Under normal circumstances in the event of mains failure load is briefly supplied by UPS's and the diesel generator automatically starts within a few seconds.

Entity Data is working with our suppliers to identify and correct the issue.

We would like to apologise to the customers impacted by this outage.

Warm Regards,
Entity Data
 

Internal Server Errors

Written by on November 16th, 2012.      0 comments

We've been having several customers getting the error of "Internal Server Error". Our systems administrator has been working on the issue and fixed up most of the affected websites. If you are seeing this error please hold tight as this will be working again soon.

UPDATE: All issues have been resolved and all websites are up and running now.

We have a report on the issues, from our products and infrastructure manager:

On friday we completed a migration of our inhouse release deployment tool's revision control system from subversion to github.  
 
This release, while well tested in our test environment,  and even on a smaller scale in product, had some unanticipated consequences when rolled out across our 100+ production servers, all of which compounded to cause cascading errors across our network:
 
(a) Since all servers needed to pull the complete version of our software again from the github servers, a massive spike in disk IO and network usage, correlated across all servers, which slowed everything down to the point of crashing
 
(b) A number of servers failed to install the release, due to differing server versions/operating systems having subtle incompatibility with the software used by the new release manager
 
(c) Due to (a), some release installs timed out
 
(d) Due to all these effects, our monitoring system (and the human's on the other end of it) were maxed out and took some time to discern all the factors that were causing errors (no single factor was responsible for all errors, and they were masking each other)
 
(e) The combination of these effects, combined with the high load throughout the system from (a), made it extremely time consuming to roll back the changes (normally a near-instant operation) and led to  race conditions where it was impossible to roll back since the release hadn't completed and likely was never going to.
 
Essentially, everything that could go wrong did go wrong.  Thankfully, since the github deployment has completed, it has performed well.  All these issues were one-off problem's related to the changeover and won't happen again.
 
 

Scheduled Server Maintenance - 27 September 9pm

Written by Zeald Support Team on September 26th, 2012.      1 comments

There will be scheduled server maintenance at our data centre on Thursday the 27th of September at 9pm. This is necessary maintenance which will improve our hosting infrastructure for current and future clients.
The maintenance is scheduled to last up to three hours, and during this time your website may have reduced capacity - meaning that it could run slower than normal however it shouldn't go down. The last maintenance, a few weeks ago, went smoothly and no websites went down. We expect the same for Thursday.

We'll be monitoring this so we'll be aware straight away if anything does happen.
 

Configuration error for one server - resolved

Written by Zeald Support Team on September 26th, 2012.      0 comments

This has been resolved, all affected websites are up and running.

Websites hosted on one of our servers are seeing an "Undefined catalog:" error message. This is affecting about 20 websites. A configuration error caused these sites to go offline, and we're bringing them back online now. Already a few are back up and running. We will update this when all sites are back online.
 

Telecom customers unable to access Zeald sites - Resolved

Written by on September 18th, 2012.      4 comments

This has been resolved.

It appears that those who are using Telecom as their ISP maybe having issues accessing any Orcon hosted sites (including any Zeald sites).
It appears to be out of our control but we have contacted Telecom and Orcon (our provider) so they are aware of the problem and hopeful it can be fixed ASAP. 
 

US Server problem

Written by on September 13th, 2012.      0 comments

3.26pm: Resolved. The US server is back up and running.

Our US-based hosting provider Rimu is currently having issues, which is affecting our US websites. This will only have an affect on your website if it is pointed to our US server, which has the IP address 72.249.182.114. You can find more status updates at their website here

We are moving away from Rimu as a US provider soon, to a more reliable provider. We already have a new US server up and running, and as a solution to this issue we encourage you to point your domain name (website address) to our new US server. You'll need to do this eventually anyway, so it won't hurt to make this change to your domain now, if you're able to. The new IP address is 50.56.72.179 and you can find pointing instructions at http://help.zeald.com/Hosting+Domains++Email/Pointing+a+domain+to+your+Zeald+website.html
 

RESOLVED - Issues with Credit Card payments

Written by on September 11th, 2012.      0 comments

Hi guys,

We seem to be having some issues when the visitor is selecting 'Credit Card' on the Checkout page and receiving an error message. The error message is "Payment failed. Please try again or select another payment method." and it will not let the visitor continue. Our developers are aware of the problem and are working on this issue as a matter of urgency.

We will keep you updated of the progress, and apologise for the inconvenience this is causing you and your customers.

UPDATE - The problem was resolved around 9.45am by an update by our Developers. Once again, we apologise for the inconvenience.
 

Scheduled Server Maintenance - 6 September 9pm

Written by Courtney Brown on September 6th, 2012.      3 comments

There will be scheduled server maintenance at our data centre tonight (6 September) at 9pm. This is necessary maintenance which will improve our hosting infrastructure for current and future clients.
The maintenance is scheduled to last up to three hours, and during this time your website may have reduced capacity - meaning that it could run slower than normal. However it shouldn't go down, we expect all sites to still be up and running this evening. We'll be monitoring this so we'll be aware straight away if anything does happen.
 

Webdrive Control Panel Issue - Affecting Zeald Domain Manager

Written by Courtney Brown on September 5th, 2012.      0 comments

This has been resolved and domains.zeald.com is working again.

10:14am. Webdrive, who provide some of our hosting, is having issues with their Control Panel. This is affecting our Domain Manager (domains.zeald.com) which means you may not be able to log in to your domain name at this time. Note that this will not affect your website or stop the domain name itself from working - it only applies if you're trying to make changes to it's configuration or settings. The latest post from Webdrive was at 9.50am today, and they are looking at fixing the issue as soon as possible. They have a Network Status link at the bottom of their website http://www.webdrive.co.nz/.
 

Orcon outages

Written by Courtney Brown on August 24th, 2012.      1 comments

Hi everyone, Orcon has been having outages this morning and it looks like this hasn't yet been resolved. This is affecting our Australian servers so your Australian website may be down. Our phone's are also run through Orcon and this is causing a lot of our calls to drop unexpectedly.
Unfortunately this is out of our control, but you can keep up to date with the latest status at http://www.orcon.net.nz/support/network_status
 

UK Server Upgrades

Written by Courtney Brown on August 22nd, 2012.      0 comments

UK clients: We will be performing server maintenance tomorrow, Thursday at 11am (12am, midnight UK time) on some of our UK servers. Websites that are running on these servers will be offline temporarily while we are working on upgrades to increase the disk space capabilities, so that you can continue to add content and files to your website. We're hoping that by upgrading at midnight, this will cause minimal interruption. This is a necessary upgrade and we apologise for any inconvenience.