June Update from the Leadership Team

Nonprofit News

The big news item is still a waiting game. We’ve filed our paperwork to become a 501(c)(3) in the United States. Usually, this process takes a month or two however it’s taking an extra long time to get our application processed. Joe contacted the Internal Revenue Service to inquire about the status of our application and was informed the application has been assigned to an agent for review on May 29th. This means we should be hearing back within the next 30-45 days.

Hosted Service News

The hosted service suffered some rough performance issues in late May which unfortunately affected a few events. The Leadership Team dug into the issues and worked with our hosting provider Platform.sh to get the issues resolved. We have also moved to a newer region of their service so that we’ll have fewer issues. So far we have seen much improvement in the hosted service performance.

Open Source Project News

The Leadership Team has scheduled an internal Product Discussion meeting focused on defining what direction the hosted service needs to push towards to serve our users as best we can. The outcome of this meeting should provide the next 4-6 months of priority work around revamping the hosted service to better facilitate our users.

This past week was the 2019 Dutch PHP Conference in Amsterdam which featured a hackathon event where Joind.in was represented by Andreas Heigl & Jos Elstgeest from the Leadership Team. We had several contributors share pull requests and we’re excited to see new community contributors!

Hosted Service Outages May 21 & 22

On May 21st the Joind.in hosted service began experiencing connectivity issues between parts of their application hosting tier, which ultimately was caused by a mix of timeouts and TLS handshake failures when our web2 frontend requested data from our API. This resulted in many requests seeing a 522 error from Cloudflare, an application error from the website, or a Slim error message while we were attempting to troubleshoot.

We’ve been in contact with Platform.sh (our host) since shortly after seeing the issue crop up. They confirmed that the region we’re currently hosting the application in was having some infrastructure performance issues shortly thereafter. This morning they reached out to us indicating that these issues have been resolved.

As part of this discussion, Platform.sh staff noticed that our account is currently provisioned in their older, less advanced US1 region. They’ve recommended that we migrate to the newer US2 region as soon as we can manage to do so. Our plan, now that the fires are out, is to perform that migration next week.

 

The unfortunate part of this site reliability issue was that it hit smack in the middle of at least one big event in the US. We don’t want this to happen again and are working to ensure upcoming events will have a smooth experience with our platform. The region move is part of this effort, but we’ve also implemented additional performance tweaks at the database level to ensure API responsiveness, as well as increased logging, monitoring, and alerting across the board to ensure that we’re (nearly) the first ones to notice another issue like this.

 

If you’d like to ask any questions or chat with the Joindin Leadership Team, feel free to join us in our Discord chat group: https://discordapp.com/invite/fWa9fu9. Thanks to the Platform.sh team, including our primary contact Larry Garfield, for advising us on measures to avoid another repeat of this issue.