Generative AI tagging in pharmaceuticals

AI-driven document tagging system
David Drummond

Lead Data Scientist

Summary
Our AI-driven tagging system transforms pharmaceutical document management. It automates and enhances tagging processes. It’s scalable, precise, and adaptable, meeting diverse industry needs. Read on to discover how the system harnesses advanced AI by enabling cost-effective, efficient, and accurate content categorization.
Recommended Reads
Recommended Reads
The Future of

Business efficiency and human collaboration

download
GenAI in Pharma:

Revolutionizing efficiency, innovation, and patient care

download
AI’s role in:

Advancing patient-centric pharma

download
Summary
Our AI-driven tagging system transforms pharmaceutical document management. It automates and enhances tagging processes. It’s scalable, precise, and adaptable, meeting diverse industry needs. Read on to discover how the system harnesses advanced AI by enabling cost-effective, efficient, and accurate content categorization.

Background:

Recognizing the potential of Generative AI in transforming document tagging processes within the pharmaceutical industry, our team embarked on a journey to develop a revolutionary solution. The solution was envisioned to be universally applicable, catering to various pharmaceutical companies, from small startups to large multinationals. It is designed to address each entity’s unique challenges and workflows in this diverse industry.

The current state of tagging content varies widely across the industry. Some companies rely on manual processes, while others have adopted semi-automated or fully automated systems. However, these existing methods often fall short in efficiency, flexibility, and scalability. The new system represents a significant leap forward, overcoming the limitations of earlier methodologies and establishing a new standard in the industry.

The system is a comprehensive answer to several key industry challenges, including dramatic cost reduction by automating the tagging process, enhancing the accuracy and relevance of content, and providing scalability and flexibility to adapt to changing business needs. The development process involved collaboration with diverse stakeholders, including internal teams and external partners. This inclusive approach ensured the system was not only technologically robust but also aligned with the practical needs of its users.

Objective

The primary objectives of our AI-driven tagging system are strategically designed to meet the unique needs and challenges of the pharmaceutical industry:

1. Enhanced efficiency in document tagging: Automate the document tagging process to improve efficiency significantly, reducing manual effort and minimizing errors.

2. Scalability and flexibility for diverse needs: Provide a scalable and flexible solution tailored for content management across pharmaceutical companies of varying sizes, ensuring that the system adapts to the evolving needs of both small startups and large multinationals.

3. Precision and relevance in tagged content: Improve the precision and relevance of tagged content, enhancing the quality and utility of data analysis and content strategies, which is crucial for informed decision-making.

4. Adaptability to industry-specific challenges: Tailor the system to address the unique challenges in the pharmaceutical sector, including varying document types, complex data structures, and the need for rapid adaptation to new research and market developments.

These objectives are aimed at revolutionizing content management within the pharmaceutical sector, leveraging advanced AI technology to offer a solution that is industry efficient, cost-effective, compliant, and responsive to industry specific demands.

Methodology:

The development of our document tagging system represents a pioneering approach in the pharmaceutical industry, blending the most advanced elements of AI with proven traditional methods. At its core, the system employs a sophisticated mix of Large Language Models (LLMs) and Natural Language Processing (NLP) techniques. This hybrid approach is effective in generating accurate tags and versatile enough to handle the complex and diverse nature of pharmaceutical content.

During the development process, we encountered and overcame two significant challenges:

1. Structured output generation: Initially, the model faced difficulties in providing an appropriate structure to parse outputs. This critical challenge was addressed by incorporating advancements in ‘Structured Outputs’ as described in research by Bumgardner et al1. By adapting these methodologies, we enhanced the model’s ability to generate structured, coherent outputs that align better with the complex requirements of pharmaceutical content tagging.

2. Understanding the model and specific taxonomy: To resolve this, we implemented a strategy of directly inputting tag value definitions into the taxonomy. This approach significantly improved the model’s accuracy in recognizing and applying the correct tags, ensuring it aligns with the intricate and specialized taxonomy of the pharmaceutical industry.

User interaction remains a pivotal aspect of our methodology. By engaging users in providing tag definitions and refining AI-generated tags, we ensure improved accuracy and customization to meet specific user needs.

The system’s API interface facilitates efficient communication between the tagging system and the digital asset manager (DAM), ensuring consistent and reliable integration. Regular data synchronization between the cloud and DAM reflects the latest user inputs and industry trends, maintaining the system’s relevance and effectiveness.

Use case – impact on content management:

The enhanced document tagging system significantly transforms content management in the pharmaceutical industry by marrying cost-effectiveness, flexibility, scalability, and precision. This system marks a departure from traditional, labor-intensive tagging methods, dramatically reducing operational costs. Compared to manual tagging, this methodology is 1/100th of the cost.

The design of this system is notably flexible, effortlessly adapting to changes in taxonomies and evolving business needs. This adaptability ensures that the tagging remains relevant and aligned with current standards. Based on our pilot, a taxonomy change estimated to take 1 month of manual retags took 30 minutes of computing. In more recent iterations, this processing has been cut to just 30 seconds for 1000’s of documents. This allows stakeholders like next-best action, analytics, and measurement to have more input in tagging and enables them to move faster to experiment and implement new ways of working.

Furthermore, the system excels in handling complex and detailed content without requiring constant updates to training sets used in traditional ML methodologies. Considering the ever-growing volume of documents in the pharmaceutical sector, this scalability is crucial. The system’s precision in tagging is especially advantageous, allowing for the accurate categorization of diverse and specialized content. Based on our survey of SMEs, the results of the Generative AI-based approach were preferred to the traditional ML method 95% of the time, which supports adoption. Alignment to historic manually tagged values was 80%, with some variability in the preference towards the AI-generated values versus the manual tags when SMEs were surveyed. This precision ensures that users can quickly locate specific documents and that the data used for analysis is reliably categorized, supporting more accurate insights and informed decision-making.

Conclusion

The Generative AI Tagging system represents a significant advancement in content management within the pharmaceutical sector. Leveraging AI introduces a new level of cost-effectiveness, flexibility, precision, and scalability. This system can enhance operational efficiency and content tagging quality, thereby revolutionizing industry content strategy and data analysis.

Reference

1. Bumgardner, V. K. C., Mullen, A., Armstrong, S., Hickey, C., & Talbert, J. (2023). Local Large Language Models for Complex Structured Medical Tasks. Preprint submitted to Pacific Symposium on Biocomputing. arXiv:2308.01727. Available at https://doi.org/10.48550/arXiv.2308.01727.

Harness the potential of AI to revolutionize pharma
Contact Us