AI-Powered Content Tagging: Automating Taxonomy Generation for Web and Video Content

Creating a well-structured taxonomy for content organization can be quite a time-consuming and error-prone process. With the explosion of content across websites, blogs, and video platforms, maintaining consistent and meaningful tags has become increasingly challenging. In this post, I'll share one approach to automating content tagging using AI, based on a recent implementation that leverages ChatGPT and Azure AI services.

The Challenge of Content Tagging

Content teams often struggle with:

Maintaining consistent taxonomy across different content types
Manual tagging that's time-consuming and error-prone
Ensuring SEO-friendly tags that drive discoverability
Handling multilingual content and regional variations
Processing video content effectively

A Modern Approach to Content Tagging

This solution combines several AI technologies to create an automated tagging system. Here's how it works:

High-level Architecture

Below is a simplified architecture diagram illustrating the flow from content sources through AI-powered taxonomy generation and tagging, resulting in structured tagging output:

AI Powered Content Tagging1

1. Defining Your Taxonomy

The first step is establishing a solid taxonomy foundation. While you can work with your content teams to define this manually, we've found that AI can provide valuable input. Here's an example of how we structure our taxonomy:

{
  "SitecoreProducts": {
    "ContentCloud": [
      "XM Cloud",
      {
        "Content Hub": ["DAM", "Operations", "ONE"]
      },
      {
        "Search": ["Sitecore Search"]
      }
    ],
    "EngagementCloud": ["CDP", "Personalize", "Send"],
    "CommerceCloud": ["Discover", "OrderCloud"]
  }
}

This taxonomy can be generated or enhanced using ChatGPT by asking it to suggest relevant categories and tags for your domain.

2. Processing Text Content

For website content, our solution:

1. Downloads content from your CMS or website
2. Uses ChatGPT to analyze the content and select appropriate tags from your taxonomy
3. Optionally generates new tags if needed
4. Maps the content to the selected tags

Here's a real example from our implementation:

def assign_tags_to_url(url, tags_taxonomy):
    # Extract text content from URL
    page_content = extract_text_from_url(url)

    # Use ChatGPT to select relevant tags
    tags = select_tags_with_ai(page_content, tags_taxonomy)

    return tags

3. Processing Video Content

Video content requires a different approach. Our solution:

Use YouTube video trsanscripts, when available, and that would be the easiest option. With video transcripts, we can treat videos same as text content.

When video transcripts are not available, we can use Azure AI Video Indexer to generate them. This is a powerful tool that provides rich metadata for video analysis.

Here's how it works:

1. Upload videos to Azure Video Indexer
2. Processes videos to generate transcripts using Azure AI Video Indexer.
3. Once the transcripts are generated, we can use them to tag videos.

Here's code snippet to get video transcripts from YouTube:

def get_video_transcript(video_id):
    try:
        # Retrieve available transcripts
        transcript_list = YouTubeTranscriptApi.list_transcripts(video_id)
        transcript = transcript_list.find_transcript(['en'])
        transcript_data = transcript.fetch()

        # Combine transcript segments
        transcript_text = ""
        for entry in transcript_data:
            transcript_text += entry['text'] + "\n"
        return transcript_text.strip()
    except Exception as e:
        print("An error occurred:", e)

Example Results

Let's look at some actual results from our implementation. Here's how the system tagged a video about digital commerce:

{
  "video_id": "[your video ID]",
  "tags": ["Commerce", "Customer Experience", "E-Commerce", "Salesforce"],
  "title": "Commerce 360 OMS Demo"
}

And here's how it tagged a blog post about Sitecore development:

{
  "https://blogs.xcentium.com/blogs/time-saving-tip-for-creating-xm-cloud-renderings-templates-and-nextjs-components": [
    "XM Cloud",
    "Development Efficiency",
    "Front-end Development",
    "Best Practices",
    "JavaScript Development",
    "React",
    "Web Development"
  ]
}

Notes on Azure Video Indexer

For enhanced video processing, Azure Video Indexer can come very handy. This service provides:

Automatic transcription
Speaker identification
Topic extraction
Sentiment analysis
Visual content analysis

This gives us rich metadata to work with when generating tags.

Best Practices and Lessons Learned

1. Start with a Strong Taxonomy: Whether manually created or AI-generated, a well-structured taxonomy is crucial. Involve your content teams in reviewing and refining it.
2. Validate AI-Generated Tags: While AI is powerful, it's not perfect. Implement a review process for the first few weeks of implementation.
3. Consider Multilingual Content: If you're dealing with multiple languages, ensure your taxonomy and AI processing can handle them appropriately.
4. Monitor and Refine: Keep track of how your tags are being used in search and navigation. Use this data to refine your taxonomy and AI prompts.
5. Balance Automation and Control: While automation is powerful, maintain the ability for content editors to override or adjust tags when needed.

Conclusion

AI-powered content tagging is no longer just a nice-to-have – it's becoming essential for organizations dealing with large amounts of content across multiple channels. By combining ChatGPT for text analysis and Azure AI for video processing, we've created a robust solution that saves time, ensures consistency, and improves content discoverability.

The best part? This is just the beginning. As AI technology continues to evolve, we'll see even more sophisticated approaches to content classification and organization. The key is building flexible systems that can adapt to these changes while maintaining the human oversight necessary for high-quality content management.

Want to learn more about implementing AI-powered content tagging in your organization? Feel free to reach out to discuss your specific needs and challenges.