I am techie, in fact I am a textbook techie, who thrives on challenging work, this project of creating an eCommerce platform for selling all kinds of baby products is close to the heart!
You will wonder why when the whole world is shopping online (1.61bn people to be exact) is this particular one special? Well, this is not your run of the mill eCommerce platform – the requirement was an eCommerce platform which also supported social features. This was one of a kind platform where parents or soon to be parents specifically mothers could follow each other and learn about each product from each’s experience – so – think a community of buyers, reviewers (word of mouth or written) and experience sharers all on one eCommerce platform!
With eCommerce growing at an unprecedented pace in India, there has been a boom of online stores available for users to buy from. With so many options, it is sometimes hard to figure the right store to buy from. Even within the same store, it is not easy to decide which brand or product is the right choice for you.
As a parent, it can sometimes be scary to choose just the right product for your bundle of joy.
What better way to choose the right product for your children than asking other parents? That is how parents would make decisions in the past and we saw no reason why they shouldn’t be able to do so online and too in a non intrusive way.
Users would be able to like and comment on products, and these activities would appear in the feeds of those who were following them. This eCommerce platform would also let a user create list or collections of products as well as stories of their children using some of these products, Baby Destination was going to leverage such User Generated or UGC to engage and share information easily with new or soon to be parents.
So, such depth of social integration on this project – “Baby Destination” – was one of a kind and ergo one of the more challenging projects that I have handled. It’s been fun, the team and I really enjoyed the process of learning new tech and exploring how to get different tech working in perfect harmony – in this case we leveraged neo4j, ReactJS, gulpjs, and the best part? – how to integrate them with complex eCommerce platform such as Magento.
Magento seemed like the logical choice for an eCommerce platform, but it did not seem like a good choice for building a dynamic application out of. Less so, a social application that needed to generate recommendations in real time.
Granted that Magento could host a rather robust eCommerce store, even allow for a lot of changes to be made to its core features with plugins, building something like a recommendations engine the magento way was simply too much of a task. Moreover, mysql was not the logical choice for mapping social data, or for generating recommendations.
Building the eCommerce store from scratch was not an effort we wanted to undertake. Magento came with a good set of features that were well tested and the vast community system support was not something that we could ignore.
On the other hand, doing a custom build on say nodeJS allowed us the freedom to use the tools that we wanted while also saving development time.
We finally decided to use both. Magento for the eCommerce sections of the application and a nodeJS application for the social features.
This was pretty much our first polyglot project where both applications had an equally important role. The Magento engine delivered features that were solely eCommerce related such as display of products, admin backend and Vendor management.
The Social Server was responsible for delivering services such as likes, comments, feed management, engagement features such as creation of Collections and User Generated Stories.
The social server would use neo4j as the database. The neo4j database would have a copy of the data.
The frontend code to render features of social services was written in react.js with some jquery for ajax features.
1. Session Management
We didn’t want to build an additional authentication layer for the Social API since the applications were supposed to seamlessly integrate into each other. We decided to share the session cookie over subdomain for apis and use redis to share the session info. This way the sessions would be shared seamlessly over the two applications
2. Data Syncrhonization
Another challenge was to share data between the magento and the nodejs api consistently. For this purpose, we created hooks in the nodejs server to sync products, users and blogs. Whenever a new resource was created, or an old one was updated, it would send the data to the nodeJS server which would then store it in the neo4j database.
There was a problem however: this approach didn’t work when the user did a bulk upload.
To handle this, we decided that when the user made a request for a particular action on a resource, the node js would, if it did not find the resource, check via the SOAP api if the resource existed and then sync the data if it was found.
There were also some hooks on Magento to update for the updation of data relating to social features such as likes, comments, etc.
3. Social Frontend
We chose react as the frontend framework because it allowed us to create widgetized UI modules that could easily be injected into the Magento Frontend when we needed to and didn’t quite make assumptions about the rest of the page.
React was quite different from the earlier UI Frameworks such as angular that we were familiar with as it wasn’t very opinionated. We had to build our own library for things like events, ajax requests, etc. It also didn’t support two way data binding out of the box either which was a bummer.
This resulted in a bit of harassment at first, but as we got used to it, it was a pleasure working with react since there was very little mystery about how things were working.
It provided the bare minimum required to build a stateful modern application and left the rest to the developer.
We were maintaining the react code in a separate repository and would insert the files into the magento repository when we wanted to upload changes. Eventually, this started resulting in dependency between the magento and UI Team for uploading changes.
So we decided, it was time to form a separate deployment procedure.
We used gulp to compile the code for the UI repository (as we should have done much earlier) and then went a step further to allow publishing the code to s3 from where it could be served over cloudfront. This, of course saved us quite a bit of time.
There was another problem however: since we were pushing changes directly to s3, and without a build control system in place, developers sometimes forgot to commit/push changes.
This would result in their work being overwritten the next time someone pushed changes and being left clueless.
So we went a step further and installed safeguard into the publish task to check if changes were committed/pushed.
4. Recommendations Engine
Neo4j was our database of choice for the social application since we needed to do quite a few traversals and that too against multiple inheritance types. Feed generation is one example where cypher query language of neo4j really came through.
Using it, we were able to accomplish some very complicated ‘joins’ across several ‘tables’ with smartly designed queries. To describe in brief what the feed component generated (in one response):
For a particular user, all the objects liked, commented, viewed, created or user followed by all the users that he is following with each object appearing only once in the feed sorted by ascending order and each object also displaying the latest activity committed on the object by a person the user is following.
It does sound complicated, but using neo4j we were able to generate this feed in milli seconds without requiring any post processing on the part of the nodeJS server.
Using a polyglot architecture was certainly challenging at many times, but it was also highly rewarding as it allowed us to designate responsibilities of the application to whichever component was best at handling it.
It provided us with a solid, tested backend to base the product on while at the same time, it allowed us the freedom to use the right tools that could do the job best.