The best way to start an AI project? Don’t think about the models

Did you know that 85% of all AI projects fail to reach the production or operation stage? Why is this the case?

It’s very common for businesses to come up with creative ideas to use AI to improve customer experience or simplify workflows. The barrier to success for these projects often resides in the time and resources it takes to get them into development and then into production. But, as we’ve seen with OpenAI’s new ChatGPT, AI can be as entertaining as it can be problematic.

With so many projects failing, or worse, being inaccurate, chances are that many of these companies are making the same mistakes. The following are some tips that will optimize your chances of success.

Start off on the right foot

The process of AI development suffers from poor planning, project management, and engineering problems. Most business leaders today learn about AI from the media, which often describes the value of AI as magic or as something that can be put into production with just a few sprinkles.

They believe implementing AI can help lower costs, improve margins and boost revenue. With competitors already in motion, it creates AI “FOMO” and executives are pushed to take action quickly despite not having a clear understanding of the overall impact, plan, cost and resources involved in creating a successful and accurate AI project.

With little understanding of the engineering environment, the first logical step should be hiring data scientists to map and plan the challenges that the team may face. However, these data scientists usually have no domain knowledge.

A data scientist entering a new organization with the goal of automating and improving the business will usually try to manually collect enough data to first prove there is value in creating AI. Once a successful proof of concept is made, the team often hits a wall regarding its data management. The organization may not collect, store or manage the data in a way that is “AI friendly.”

For example, a factory that wishes to embed smart fault inspection on a production assembly line will be able to demo the AI project pretty fast by using a single camera on a machine for a few minutes. However, for the project to get put into production and be used daily, it must transition the single-camera demo to 500 cameras operating 24/7. This will require many months or even years to bring the value the AI provides in the demo across the finish line.

Executives should, of course, have in mind a clear idea of the problem they want to solve as well as a business case. But the AI core team should include at least three personas, all of which will be equally important for the success of the project: data scientist, data engineer and domain expert.

A data-centric kickoff

While it may sound strange, the first AI project that is successful within an organization should not involve algorithms or fancy AI models. AI is essentially an effort to automate knowledge. Like all automation efforts, it’s good to start by showing the value of a few examples in a manual, slow and non-scalable manner.

In the kickoff, the data engineer will create a few cases using data and the domain expert will transform these case studies into examples. This step exposes the core of the AI process. The business will have raw data and a goal, and then they will want to get an example, which is the output data as it is ideally imagined.

This process is called data development, and the approach is data-centric by nature since no modeling is involved. This approach has many advantages over the model-first approach, such as

it requires less investment;
it utilizes the organization’s strengths (process, data, domain expertise);
it is faster; and
it lowers risk.

Once a few examples are completed manually, the business can start planning the AI’s path to production.

The best way to start an AI project? Don’t think about the models by Ram Iyer originally published on TechCrunch