REFUNITE behind-the-scenes: Tech Team Reveals Upgrades
The past five months have been a particularly exciting time for the tech team at REFUNITE. Given the increased number of displaced people globally, we saw an urgent need to upgrade our backend platform to better adjust and scale to the numbers of displaced persons we were seeing.
Our previous platform was a monolithic api that handled everything on our platform. The touch points (SMS, USSD and WEB) would then make calls to this service to handle registration, notifications and communication between users. The monolithic API proved problematic to maintain, test and re-deploy especially as demands for new features from our users increased. At the same time, as the technical team grew, there was increased pressure on it to break up the monolithic api to smaller, strictly defined services that would do one thing and one thing well.
Combined these changes will make it easier for REFUNITE to respond to emergencies faster and more efficiently from a technological perspective.
Even more important was the increased need for greater visibility into the numbers and statistics on our platform. The previous REFUNITE platform made generation of insights into data computationally expensive and time consuming. There was need for a redesign of all our data stores with business intelligence and metrics easy to build.
It gradually became clear that in order to achieve the intended goals for REFUNITE’s new platform, a lot had to change regarding our existing culture and methods of working. During the past months, we have not only changed our architecture. We have also changed our work culture.
We’re currently inspired by agile principles and scrum methodology. Agile sets forth that:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Scrum methodology aims to achieve these principles through iterative planning taken at two week intervals. We also use cross functional teams, which include all disciplines needed to complete features.
Whereas SCRUM and agile methodologies were only used as task division and workload distribution tools for the tech team, we discovered that in order to effect such a large scale migration of users, databases and services, we had to apply SCRUM’s more salient features, (KANBAN, sprints) to finish testing and migration of our services and data by mid October.
We split out our monolithic platform into a set of core functionalities, each of which was to be handled by a single API. The design goal for each API was that it would be simple to deploy and understand, and more importantly, easily scalable. Each API was assigned to a lead, who was to present the initial blueprint for the API to the team, after which the team would criticise it, leading to and addition/reduction of functionality in the API. After consensus was reached, work on the API itself would begin, with each core feature addition presented to the entire team for review.
Sandboxes, which are isolated testing environments, were set up for testing migration of the API. The touch points the integrated test instances of the API’s on the sandboxes. The entire organisation was involved in testing the features of the new API’s, and most importantly, that no existing feature would be broken on migration. After we exhausted all possible tests, and a few migration dry-runs and simulated roll backs, we migrated all our applications to the new micro service architecture. The downtime experienced by our users was only 3 hours…quite a feat considering the amount of work and possible failure scenarios typically present during such large scale migrations.
After the tech freeze was lifted, most of our time is now spent fine tuning the performance of each API. There has been heated discussions on feature additions to the existing API’s, with each feature addition request subjected to debate and review from the entire tech team. Discussions like these are an integral part of our culture. As REFUNITE, we are now at a point where we have great faith in the existing platform and its ability to stand up to the challenges it will be faced with as we scale with the overarching goal in mind: To help displaced families and communities reconnect all over the world.
We’re hiring! Send an unsolicited resumé to REFUNITE’s Tech Lead, Laban Mwangi at lm@refunite.org.
