At Zyro, we are developing a next-generation website builder to allow our users to achieve online success faster and cheaper than ever before.
All Zyro teams contribute to this goal with different processes/development cycles.
In this article, I would like to talk about the AI team development cycle and how we create new AI features.
The Ideation Stage
It all starts with an idea.
At Zyro, we have a huge list of ideas that have been submitted to the AI team by our colleagues, uncovered in user testing sessions, and our own regular market research sessions.
There are no rules about who can suggest ideas or what form those ideas should take. Anyone who sees an opportunity to improve the end user’s experience can ping the AI team, put their idea into the mix, and add it to our list for consideration.
Next up are user testing sessions. As of Q1 2021, the AI team participates in every single user testing session, and the outcome has been great.
User testing allows us to get closer to our users, learn about their struggles and find innovative ways to help them.
Our final source of ideas is market research. Zyro’s AI team is always trawling the market for tools, pre-trained models, datasets, and techniques that could be used to help solve our users’ problems.
With all these avenues for inspiration, there are naturally a lot of ideas in our backlog. To organize that list effectively, we order each idea according to two simple parameters:
- The value it would create for our users
- The project’s feasibility at first glance
Proof of Concept (PoC)
Once we have the idea, it’s time to start by developing a proof of concept (PoC). This is designed to help us to answer some basic questions about the idea:
- How much time is needed to implement the idea?
- Is the idea feasible?
- What are the best techniques to use?
- What is the fastest and most effective way to achieve the idea?
It is entirely possible that the idea could be implemented in many different ways. So, in the PoC stage, we evaluate all the main possible solutions and we set a time limit of 40 hours per PoC to make sure that the process keeps moving and the AI team do not get stuck on one PoC for too long.
Over time, the AI team has defined a four step flow that should increase the success rate of PoC development.
- Analyze research papers
- Analyze open source tools, models, and data
- Rapid development and experimentation
- Record PoC learnings in a handbook for further study
Every PoC that we develop is recorded in the handbook and we use the following structure to do that.
We always collect the main information about each PoC:
- The approaches tried with source code
- The results achieved and learnings made
- Ideas on how the results from each approach might be improved
Real Data Evaluation
After that, each PoC is evaluated with real Zyro website builder data.
This evaluation allows us to see how our PoC is performing with real data and as such, what improvements would be needed for a PoC to work in a live environment.
Every month, the PoCs that the AI team has worked on are presented in a brief 15-minute presentation which we use to gather even more feedback. This process of feedback and evaluation of results and learnings allows us to decide if we want to move forward with any of our PoCs.
When an idea is selected that is worth investing our time in, a refinement meeting is organized with a cross-functional team.
In this meeting, we plan the resources needed to bring the idea to life (frontend developers, QA engineers, designers, etc.)
After that, the standard development process is as follows:
- Minimum viable product (MVP) development
- Deployment to production
For the first version of the feature, “good enough” is sufficient.
We do not need to polish the first iteration because this version is primarily for gathering data about features. Once we have that data, we can iterate accordingly.
Our flow is: build, measure, learn, repeat.
As we are working with tasks at the “idea level,” there is often a large degree of uncertainty and as such, a few additional steps are needed in order to understand the scope and feasibility of the task.
By placing time constraints on the PoC stage, we can iterate fast, while recording all the knowledge we learned in the handbook. Alongside this, our monthly presentations to other teams bring transparency to the work of the AI team.
It is important to note that different development cycles have different code bases and, therefore, different deployment processes.
For example, from a code perspective, the AI team does not write code for the frontend nor the backend of the builder itself, and as such, it should be understood that the AI team is not participating in the direct development of the Zyro website builder.
Instead, the output of our team is the network resources that perform specific tasks and are executed by Zyro website builder, such as third-party add-ons (typically with a RESTful API endpoint.)
This allows the AI team to focus on R&D tasks, experimentation, and research into innovations that might be applied within the Zyro website builder, rather than focusing on the integrations of one thing or another into the current builder ecosystem.
We are not bound to a stack that is used for builder development, such as programming languages, frameworks, technology, or tools.
We are always experimenting and searching for ways to improve our effectiveness.
Currently, we are using the model described above, but as with everything we do, this model will also be evaluated so that we can find the weak points and continue to improve.