This is the status page archive for tapestryjournal.com, the online learning journal. It contains incidents that occurred more than 12 months ago.
You can see more recent issues on our main page.
Once again, between 1349 BST and 1448 BST we saw increased error rates for some people using Tapestry. Once again this seemed to be related to an issue with one of our database servers. We have upgraded the capacity on the relevant servers.
Between 1315 and 1345 BST we saw increased error rates for some people using Tapestry. The cause is heavy load on the same database that was causing problems yesterday. We have some further fixes that we will deploy once the load drops off this evening. We are sorry for the inconvenience this is causing.
Between 1342 BST and 1439 BST some users may have seen errors around changing passwords, reseting passwords and activating new users. The cause was a high load on our database due to a large number of new users hitting code that had recently been changed and was not working optimally. The high load meant that some requests were timing out, leading to an error. We have deployed a fix to the code which has reduced the load and are monitoring the situation.
There was an increased rate of errors between 13:30 GMT and 14:10 GMT due to increased load on the database. This issue has been resolved.
From 13:10 to 13:23 BST up to about 30% of requests to our servers resulted in an error. This is because one of our database servers slowed down. We are investigating the cause of why this happened.
From 12:35 to 12:38 GMT up to about 4% of requests to our servers resulted in an error. This is because an update introduced an error. The update was reversed. We are looking into how the error got through our testing. We are sorry for the fault - a bad day for us.
From 0600 to 0605 GMT about 1% of requests to our servers resulted in an error. This is because a maintenance update caused one of our database servers to run out of memory. The system switched to the backup database server at which point normal service resumed. We will investigate how the maintenance led to the server running out of memory. We are sorry for the fault.
From 13:57 until 14:38 on 5 March up to 10% of requests to our servers resulted in an error. The cause was that a query on one of our database ran slowly. Because this is a busy time of day, this led to a 'traffic jam' of queries to pile up behind it. At a certain point, the traffic jam got too long, which meant that our other database servers got clogged up. Once that happened, the servers started erroring on some requests. We are working on improvements to fix this problem. Sorry for the fault, the errors and the inconvenience this will have caused.
Between 13:45 UTC & 13:59 UTC Tapestry was responsing with increased error rates due to a database connection issue. This has now been resovled.
Planned maintenance between 0600-0700 UTC to update our database servers. Actual downtime of < 1 minute at c. 0624.
From the completion of the maintenance, till around 0826 UTC Tapestry was running, but much slower than we'd like, which may have cause some people to see timeout errors.
Between 10:50 UTC and 10:54 we again experienced a burst of errors from one of our database servers. The cause is still not clear.
Between 10:56 UTC and 10:59 we experienced a burst of errors from one of our database servers. The cause is not yet clear.
Between 08:20 BST (07:20 UTC) and 11:09 BST (10:09 UTC) on 26 September 2018 we experienced bursts of errors every 10 minutes or so due to a connection problem between some of our servers. Our hosting provider traced and fixed the fault.
For just under 1 minute between 3:14pm and 3:15pm UK time, some schools may have experienced errors as one of our cache servers failed and was replaced by its backup.
For just under 2 minutes between 9:22pm and 9:24pm UK time, some schools may have experienced errors as one of our database servers failed and was replaced by its backup.
For just under 50 minutes between 1:32pm and 2:17pm UK time, Tapestryjournal.com was working, but attempting to connect using a modern web browser or the apps would show a security warning due to an expired certificate. Read the incident report.