We currently run a cluster to supplement the Status network because we don’t have enough people running them themselves. This is a pain point for us as it is against our principles, is extra cost and time we spend to support our network, and goes against the whole idea of what we’re trying to do (based on us depending on it).
@Bruno has spent quite a bit of time creating guides and shipping at-cost machines to run a status node yourself. I have planned on doing this, but have put it off because it’s additional funds that aren’t high on my spending priorities (US taxes are rough this year, don’t buy a house using crypto). My wife came up with this idea and I ran with it, she is smarter than me.
We are clearly trying to find ways of making the network more resilient and less dependent on the few of us that monitor it. Chaos day is a testament to this, and our ability to get around things quickly.
We currently pay around ~1380 USD/month (eth.beta) for hosting this cluster (confirmed by @jakubgs). We have eth.staging and eth.testing but we should keep those.
I propose the following:
- Status subsidizes the purchase of the required hardware for core contributors, so that they can all run their own mailserver and status-node.
- Core contributors then build and run Status nodes as part of their employment (when feasible of course). It is important that contributors go through the steps to do it, and not rely on others to do it for them. We are not building things that emphasize off-loading responsibility.
- This currently requires some technical ability, we can get around this by doing it in tranches, each building tutorials and processes that make it easier for the next. This makes the next Chaos Unicorn Day a breeze already. Simple tranches are as follows:
- technical crew
- semi-technical crew
- non-technical crew
Benefits of this:
- We offload the dependance of the Status network from our cluster, if it fails, the network stands. In fact, we potentially drastically boost its resiliency by increasing the total number of nodes being run, which ends up growing with the company.
- The people who work for Status have the requisite knowledge to help those around them to bolster the network. In my opinion, anyone who works full time should at least understand how to run a node. If they can’t, then we’re failing.
- We reduce the financial burn of running the entire cluster. Of course we should run at least a few nodes (bootnode, statistics gathering, etc).
- We want people to run nodes. Why would they when we don’t? This dog-foods the concept and helps us build what needs to be built to make that an easy job, and we continue to behave in a manner ask others to follow.
- cost: at ~66 people and 360 Euro (Bruno’s no OS build), we’re sitting at ~24,000 (being rough). This is still potentially less than the cloud services. There will be a few folks in the company that simply don’t have a good way to sustainably keep one running (maybe). And we severely downgrade our running cloud-hosted cluster. Could end up being positive in the long-run.
- time: The time it takes to get everyone to get the hardware, go through the steps, troubleshoot, write guides, upgrade, etc takes away from them building the final release. I argue that who cares if we have an app in the main store if it doesn’t have a resilient network to support it. If we get a massive on-ramp of people who want to know how to run their own node, and we can’t on-board them quickly to the right way of doing it, when we’ve failed.
- These things will change: With incentivization work being done to try and get peopel to run their own nodes, this tech may change in the process, which will require everyone to upgrade. This could be drastically more annoying than throwing it on @jakubgs shoulders (lol).
I understand this won’t be a walk in the park, but it feels like the efforts put into this are required for us to deliver what we idealize in our principles and thoughts. I’d appreciate your criticism where I’ve forgotten something or glazed over why this won’t work, or your support in getting it done.