Turning off the LAMP, part two
This is the second part of the story series describing our journey from early 2017 to present day. If you haven't read the first part - I suggest you do that first.
So, now we had it figured out the why and what, and at least starting to figure out how, the next big question was "Who?". We had a small team of PHP coders, who needed to learn a few new tricks and two pythonistas ready to mentor. Not exactly having the full-stack in-house, as one could say. We had a mobile app - built by a 3rd party contractor - that was pretty decent and used our public API, but our web shops and the back-office part were still umbilically connected to the PHP implementation running either server-side or using another totally undocumented API - which really wasn't even RESTful by a long shot.
Being a start-up with no name and located in one of the business parks in Vantaa, recruiting top talent was not going to be easy - in fact it took us a better part of a year to hire a single senior front-end dev, but that's a whole story by itself that needs to be told another time. Since we also needed a new front-end implementation right now, or preferably last year, and were burning a scarce resource called investor money, we were pretty much left with only one option for our front-end development: go full military grade. In other words - get something just barely good enough, built by the lowest bidder.
In hindsight, it would have been nice to be able to say that it was a conscious decision and that we knew that we're getting yet another unmaintainable mess of a codebase, but yeah we totally knew we did. Nevertheless, in 3 months or so, we actually had a web shop as a if not technically a single page app, at least something that could be served from a S3 bucket without any server side rendering - and more importantly - using the public API.
In the meantime, we started slowly doing things in a more organized manner, without going full scrum just yet. While the old team was learning Python by slowly re-implementing the API one endpoint at a time using Flask, Flask-RESTplus and SQLalchemy they were doing it in feature branches with pull requests and code reviews by the more experienced ones.
Part of me wanted to go for an asynchronous web framework of the REST era, such as aiohttp - you have to remember that this was still the spring of 2017 and Responder (et al.) were not a thing yet - but you had to draw a line somewhere how many new concepts you introduce at once.
There were, of course, some old habits that died hard, but the amount of copy-pasta dropped to zero within weeks. However the general result was that the code quality improved tremendously and TDD was slowly gaining ground.
The new implementations were deployed to AWS Lambda using Zappa and paths were mapped one-by-one at CloudFront to point to them instead of the default legacy implementation. Around this time AWS finally enabled Python 3.6 as a runtime, so we could drop six and Python2.7 almost immediately, which I was -and am - really happy about.
When I said earlier our API was pretty decent, I was being generous - it suited the purpose, but it had all the hallmarks of "My First REST API" on it. For example the documentation was a completely separately maintained website and it was evident that it wasn't as much designed but implemented from the monolithic database's point of view.
So while the API - now dubbed legacy API - was being re-implemented, we started designing the next version of the API on a parallel track - focusing on a more resource oriented way. Since the old data model already had duplication wherever it made sense (and everywhere else) we realized that in a multi-channel online merchant network the data really is not that relational and SQL was going to be more of a problem than solution, the new API was based on a key-value storage. Redis would have been the obvious choice - me personally being a fanboy, but as we were planning to go fully serverless, we opted for DynamoDB instead.