Artificial Intelligence: what was once a pipe dream is now capturing the world’s attention as it begins to deliver on its more outlandish promises. More than that, AI is increasingly considered integral to the so-called fourth industrial revolution that is currently underway, as every aspect of our lives is transformed by the surge of advanced and seamlessly interconnected technologies.
Although the concept is as old as the computer itself, it is only in the last few years that AI has been thrust into the limelight, reaching a breakthrough moment with the hysteria surrounding the recent release of OpenAI’s ChatGPT and its image-generating cousin DALL.E, as well as other models of their ilk.
It's no wonder so many are enthralled, given the jaw-dropping feats such systems have accomplished. Microsoft has been a huge driver in its success, investing billions in OpenAI. Now, it is starting to commercialize its potential by seemingly jamming its AI models into every one of its products and services conceivable, with varying degrees of success (look no further than the rollercoaster ride Bing has already taken the world on).
Other big players like Google and Meta have built their own AI systems too, and it is of course very tempting for everyone else to follow their lead, and many have already - but there are numerous obstacles associated with adopting AI technologies.
For one, the practical applications are still in the relatively nascent stages. There are plenty of errors and mistakes that even advanced AI models can make, as many have found with ChatGPT. And although big tech is keen to roll out commercially to individuals and businesses, there are dangers in doing so prematurely, as Microsoft and Google - with its rival Bard chatbot - have found out to their reputational and financial cost.
In order to improve, spending will have to continue and even increase beyond the already huge outlay that it has taken to build and maintain these impressive machines. And although the titans of the industry may have the resources to dedicate to R&D, is AI worth the investment for SMBs and firms that are anything less than behemoths? And will costs come down soon or remain exorbitantly high for the foreseeable future?
Development costs
Firstly, it's worth getting things into perspective and looking at how much it costs to develop, implement and improve state-of-the-art AI. Large Language Models, such as ChatGPT, are the showstoppers, and the most advanced, taking a huge amount of time, effort and money to launch and operate.
Research and consulting firm SemiAnalysis has also been crunching the numbers in trying to determine the compute costs of those looking to develop similar LLMs. MosiacML, for instance, currently offers the training of AI models of a purported similar quality to ChatGPT for less than $500,000.
Taking the running costs of a single Nvidia A100 GPU as an example - the current favorite used by the big cloud services, such as Azure, which hosts ChatGPT - SemiAnalysis worked out that the baseline compute cost to run just one of these for AI training purposes is $1.50 per hour, when used in systems with clusters of 256 of the GPUs. It arrived at this figure by surveying many startups and enterprises.
It also notes that some companies will have better deals than this with the big cloud companies, such as AWS and Azure, and that entering into three-year contracts will also improve costs - Azure’s price is $1.36 per hour to use a single A100, but SemiAnalysis points out that many will not want to enter into contracts this long given that the A100 is already three years old, and developments always move fast in the world of computing technology.
In calculating the theoretical overall training cost of popular models using A100s, the aforementioned MosaicML with its GPT-30B model will cost $325,855. OpenAI’s GPT-3 model, of ChatGPT fame, costs $841,346 according to SemiAnalysis’ estimation. Google’s PaLM model - “the most advanced dense model that has been trained and publicly detailed” - tops the scales of the popular LLM training costs, at an eye-watering $6,750,000.
And as SemiAnalysis is keen to point out, these are only the training costs - the other costs to consider include: “people required, ML Ops tools, data gathering/preprocessing, failure restoration, one-shot/few-shot learning examples, inference, etc. Many of these components are incredibly costly.”
Various reports speculate that ChatGPT costs OpenAI around $100,000 a day just to run, again based on the prices Azure charges for running the A100s. But the biggest portion of an LLMs cost is devoted to that last one mentioned above - inference. This is the model's ability to make predictions as to the best possible output based on a user's input - essentially, it's how it operates, and accounts for 90% of the overall compute costs. Currently, ChatGPT costs $36bn in inference costs according to SemiAnalysis figures.
However, these costs are coming down as new hardware and techniques are developed. Chibeza Agley, co-founder and CEO at AI-powered learning platform OBRIZUM, believes that “people are finding clever ways to use existing hardware in a parallel way. For example, most Hugging Face endpoints are now powered by CPUs rather than the almost ubiquitous GPUs of two years ago.”
Agley also emphasizes the importance of parameter size, which determines how many variables an AI has, and therefore, in essence, how sophisticated its outputs can be:
“The single biggest factor in cost though is the size of models. If you have to run a calculation using 175Bn parameters, that’s always going to be more expensive than using 1Bn parameters. The exploding size of models (sometimes referred to as a model Moore’s law) is causing us problems here. I expect the next revolutions to be using a smaller number of parameters better, rather than just more parameters”.
Usage costs
So that’s how much it costs to build an LLM, but what about using one? ChatGPT can be used for free at the moment, but this will end soon and users will then have to pay for the privilege. Sam Altman, CEO of OpenAI, wrote a frank tweet in December stating that "we will have to monetize it somehow at some point; the compute costs are eye-watering" - as we have seen above.
Whether some kind of free version will continue to be available has not yet been confirmed, but a premium version dubbed ChatGPT Plus was announced at the beginning of February 2023, which will leverage a subscription-based model costing $20 per month to use. The advantages include unfettered access to the chatbot, even during peak times - which is something a lot of users currently run into issues with. It also promises faster response times, as well as preferential access to new features and improvements as they arrive.
ChatGPT Plus is already available to US customers, and roll out to regions worldwide is to be expected soon. In the same blog post, the company also stated that it is “actively exploring options for lower-cost plans, business plans, and data packs for more availability.” So it looks like it is keen to be of use to your firm, albeit at a price.
And if your business wants to integrate AI power within its own software, then there is also an API on offer from OpenAI, using the same models that have been and are currently used by ChatGPT. The most advanced of these, Davinci, is priced at $0.02 for every 1,000 tokens produced for the base model. A token is the data that makes up the words processed by the AI, and 1,000 of them roughly equates to 750 words. To put that into perspective, OpenAI says that “the collected works of Shakespeare are about 900,000 words or 1.2M tokens” - that’s only $24!
It is worth noting, however, that experimenting with the API in the platform’s Playground mode counts towards your token usage, and if you want to fine-tune Davinci with your own training data, it will cost you $0.0300 for a thousand tokens used in training - and this includes both the number of tokens in your dataset and your training epochs (which refers to the process of completing one full cycle of the training process), and $0.1200 per thousand tokens thereafter for its actual use.
Marketing potential
LLMs are of course a huge potential boon for those in marketing, where generating written content en masse and to tight deadlines is key. If your firm is prepared to pay for their use, then models like ChatGPT do appear to excel in this regard, being able convert short prompts on just about any given topic into a detailed and lengthy reply in appropriate tones and styles that you desire.
However, Edward Coram-James, CEO and founder of SEO company Go Up, believes that there are some major problems with using LLMs like ChatGPT for this purpose, particularly regarding Google’s perspective on such content:
“Businesses looking to cut their marketing costs may want to hold fire on swapping out the humans at the helm of their written content. Google has made its stance on automatically generated content clear, and could penalize content produced by AI, potentially decimating a site’s search rankings and undoing years of hard work to build web traffic.”
“In fact, we could be well on track for the single largest mass-penalisation of websites in Google’s history. The businesses that have hastily employed ChatGPT could see their websites removed entirely from the search engine’s database, if Google chooses to prioritize high-quality, informative, human-written copy.”
“Google is smart, and will only get better at spotting the watermarks of AI-generated content — such as the repetitive stylistic patterns and derivative talking points being regurgitated by the current iteration of ChatGPT. So, companies should think twice before putting AI to work to produce their content.”
“As we know, AI gathers its information online, just as a human researcher typically would — but the way that it synthesizes a narrative from this data errs on the side of “extremely unoriginal”. This is largely down to the fact that ChatGPT seems to simply take the most common talking points on the web and regurgitate those same insights back to the user. So, feeding the ChatGPT algorithm your query will prompt a relevant but highly formulaic response — and certainly not one that Google will be looking to reward.”
He also adds another issue with using AI-writers: “AI-produced content is not fact-checked — and if misinformation proliferates through Google’s search rankings, the tech giant will be keen to make an example of any spammy websites and preserve its authority as the top-dog search engine.”
In-house systems
So what about in other industries - is investment in AI worth it? What are the costliest aspects?
“I would argue that the greatest cost is structural and architectural though; building scalable systems that can support AI is a very tricky business, which even big companies can get wrong,” Agley notes.
When it comes to firms developing their own in-house systems rather than using a pre-packaged alternative, Agley ran the numbers:
“The OpenAI endpoints are very competitively priced. I did a calculation of how much it would cost to have a server to serve a similar open-source model to GPT-3, and it was more per request to run it yourself than just to use the AWS offering, even if you were using it every second of every day 24/7.”
However, he did mention some pitfalls to using an off-the-shelf model, namely that “they are very general in their intelligence, and your ability to control the tasks you give them is restricted to writing freeform text templates”, and that “they are absolutely not explainable and as a startup, your competitive moat is pretty shallow if you only use these models.”
On the question of whether non-AI alternatives to solutions are cheaper, he gives a somewhat surprising answer:
“It’s almost always more expensive to use AI. My team are probably sick of hearing me say 'if you can do it any other way, don’t use AI'. That said, there are times when you can’t write traditional software “rules”, you can only describe the problem in data and train a model to reproduce what a human would. If it’s a choice between being able to do something valuable, and not, it’s worth the cost!”
However, Gleb Gusev, CTO and Co-Founder of Artec 3D, a manufacturer of 3D scanners, had a different take:
“In many cases, using AI can be more cost-effective than other methods such as standard computing systems or manual human-generated work. Using standard computing systems for complex tasks such as image or speech recognition can be very time-consuming and computationally expensive. AI models can often perform these tasks more accurately and quickly, with less human intervention and at a lower cost.”
He added that “using AI can provide cost savings by increasing productivity and reducing errors. For example, an AI-powered assembly line can operate more efficiently and with fewer errors than a manual assembly line, reducing labor costs and increasing productivity.”
However, Gusev did provide the caveat that: “it is important to note that the cost-effectiveness of AI depends on the specific use case and the quality of the AI model. Developing and deploying effective AI models can require a significant investment of time, expertise, and computational resources. In some cases, the cost of developing and deploying an AI model may outweigh the potential cost savings.”
Will everyone be using AI soon?
While the major AI models used in LLMs are extremely expensive to develop and run, it seems as if many expect their prices to come down, at least to some extent. How much this will be passed on to the customer is not yet known, but given the competition between all the big tech companies to develop rival systems, it should help to bring consumer and B2B costs down.
The question is, though, is it really worth it? When it comes to marketing and written content, LLMs may not be the magic bullet that the industry may hope for. Since it is an industry where potential customers can smell inauthenticity a mile away, getting a computer to write your content in the only way it knows how - anodyne - then perhaps it isn’t the best answer. Not to mention the SEO problems it can cause too.
In other industries, though, it seems AI can be worth the investment, providing that other alternatives are explored first. Automating complex processes certainly has its obvious benefits, but firms will have to make sure that costs don’t spiral out of control. Using in-house systems is rarely worth the benefit, according to both Agely and Gusev, over off-the-shelf alternatives.
Both were also in agreement on the eventual need for most businesses to adopt AI to some degree at least. “As the technology and its application becomes more mature, we’ll see it featuring in all areas of business”, says Agely.
“It's an inevitability that in some form businesses will need to embrace this new tech or run the risk of being left behind.”
- Here are the best AI writers if you want to see what they are capable of
