On Sunday at 12:00PM UTC (8:00AM Eastern) we had an internal SSL certificate expire. This certificate is only used for internal load balancers (not our data ingest, API, or UI endpoints) causing internal communication issues. Our API service was not able to communicate with some of our backend services resulting in an unusable user interface.
We discovered the issue at 9:40AM Eastern and updated the certificate, restoring access to the UI at 10:00AM Eastern.
We recognize that behind our firewall API endpoints which are not public do not require SSL and therefore have decided to remove them in this specific case, which prevents us from being exposed to certificate expirations. For all public-facing endpoints we will continue to force the use of SSL and will monitor them more closely for expiration.