Link copied to the clipboard.
- Dr. Theresa Bick
Advancements in artificial intelligence (AI) are a game-changer for the economy, but also a double-edged sword. The current hype has motivated many companies in the past two years to invest in AI technologies, yet not all initiatives lead to success. Impressive results can be quickly achieved, especially with large language models (LLMs) – but how sustainable are these really? Too often, AI projects in this context fail due to hidden costs, lack of integration, unmet data protection requirements, or poor problem-solution fit.
In this article, we examine the impact of using LLMs – and why it is crucial to critically question the technology.
With the seemingly easy availability of powerful generative AI technologies, particularly LLMs, it seems natural for many companies to use these approaches as universal tools to solve various business problems. Thanks to easily adaptable interfaces from well-known providers like OpenAI, quick results can be shown in the proof-of-concept (PoC) stage. However, when the insights gained need to be transferred into usable solutions, several challenges arise:
Deployment: Large language models are extremely resource-intensive, both in their development and operation. Even after training, the operational costs are so high that for most companies, it is nearly uneconomical to host LLMs on their own. Therefore, most rely on cloud providers that offer the necessary infrastructure and scalability. However, this dependence on the cloud brings additional challenges:
Uncertainty of training data: It is often unclear which data was used for training, leading to legal risks, especially regarding copyright and intellectual property. Companies that use such models risk unintentionally reproducing protected content5. Another issue is the potential political bias hidden in the training data. In the case of DeepSeek, we see how political influence can negatively affect the model's output in terms of factual accuracy, neutrality, and reliability of results6. To eliminate these risks, it is essential to rely on models that have been trained transparently, ideally using self-selected, company-owned datasets.
Problem-solution fit: An LLM is often selected too hastily without adequately analyzing the underlying problem. As a result, the chosen AI solution may be technically impressive but not address the actual issue. Without a solid problem definition, there is a risk of investing resources into a solution that is either ineffective or creates new challenges (see previous points). A thorough focus on the problem, data, and requirements is crucial to ensure that AI actually delivers added value. Experience shows that an LLM is not always the solution that offers the best fit for a problem.
Many companies use LLMs in chatbot applications, for example, to automate customer support or streamline internal processes. However, the actual added value in practice is often limited: Many chatbots fail to effectively handle complex inquiries, leading to frustration among users. Instead of saving time or costs, these systems often create additional effort, such as necessary revisions or the need for human intervention. Additionally, companies often overestimate the benefits of a chatbot without thoroughly analyzing the needs of the target user group or the specific requirements of the application: in most cases, users do not want to chat but are simply looking for information. This leads to the technology being suboptimally utilized, and the expected efficiency gains do not materialize.
However, beyond chatbots, there are a variety of other AI-supported applications that often allow companies to achieve a significantly higher degree of process automation and can be implemented with classic natural language processing (NLP) technologies. NLP models are established algorithms that can be tailored to a wide range of automated document processing tasks. These technologies often rely on statistical and linguistic methods, which are less resource-intensive than modern LLMs.
Often, the underlying problem revolves around the (re)findability of documents and document content. We would like to explain two exemplary use cases – text classification and information extraction – in more detail below. Additionally, we aim to demonstrate where LLMs truly offer meaningful applications.
"Document classification" refers to the categorization of texts into predefined categories, such as "invoice," "delivery note," or "termination letter." Automated classification can be used at various points to increase efficiency, for example, for pre-sorting incoming mail and assigning it to the responsible departments, for automatically filing documents into a desired folder structure to improve organization and retrievability of documents, or for prioritizing the urgency of requests. In a previous article, we explained how retraining pipelines in the context of automated document classification can be efficiently operated.
For information extraction, methods are often used that can identify specific information such as names, dates, or amounts from texts. This can be utilized to extract relevant contract or membership data from documents, such as membership numbers, names, addresses, and contract/policy numbers. This can then be used to automatically fill out forms, perform completeness checks, or carry out anonymization.
Such well-established methods are so resource-efficient in terms of computing power that they can easily be run on standard on-premise computers. This has the particular advantage of keeping costs and data sovereignty under the company's control.
In addition to large, cloud-based LLMs, smaller LLMs that can be hosted locally present an interesting alternative for companies. These smaller LLMs can be tailored to specialized applications, such as creating summaries or performing semantic content searches. This allows employees to quickly get an overview of document contents without having to read the entire document. For the results of smaller LLMs to be convincing, it is important to train the models with domain-specific data. Additionally, suitable IT infrastructure and the necessary expertise are required to integrate models efficiently and ensure high performance. When used correctly, such models can not only be data protection-compliant but also more cost-efficient than their larger, cloud-based counterparts.
The use of LLMs in companies often presents more challenges than benefits. High costs, enormous resource demands that complicate meaningful local use, and data privacy issues with cloud solutions – often hosted in the USA – diminish their attractiveness. Additionally, it becomes apparent that the use of chatbots often does not deliver real added value: the problem-solution fit is frequently inadequate, and instead of saving time or costs, additional efforts arise due to revisions and human intervention.
We have shown that using "classic" NLP technologies and smaller LLMs, which can be hosted locally and used for specific tasks, often provides a better option for automation and efficiency gains. These approaches enable data protection-compliant and resource-efficient automation solutions, such as for text summarization, classification, data extraction, and anonymization, and deliver significantly better results through domain specialization. Companies should, therefore, focus on streamlined, tailored NLP solutions to promote both efficiency and sustainability.
1 Tokens are the building blocks of text in the context of natural language processing. They are units into which a text is broken down in order to process it machine-wise. A token can be a word, a number, a punctuation mark, or a fragment of a word.
2 Data, privacy, and security for Azure OpenAI Service
3 DeepSeek privacy policy
4 STACKIT Website
5 Copyright lawsuit against Meta
6 Censorship and political bias in DeepSeek's LLM