“AI Tagging” is an extension for the 4ALLPORTAL  based on Microsoft Azure – Cognitive Services, which enables “computer vision” and “face recognition” for images in the 4ALLPORTAL. Image content is precisely analyzed and the extracted information is automatically stored as metadata in the 4ALLPORTAL.


Summary of AI tagging functions


Connection to Microsoft Azure – Cognitive Services

Configuration of the Microsoft service via the 4ALLPORTAL administration

  • Data center URL
  • Subscription Key – Face (face recognition)
  • Subscription Key – Vision (computer vision)
  • Activate/Deactivate connection to the service 

Face recognition:

  • Face recognition (people) in images

  • Marking faces in images using an editable markup (image preview in detail view)

  • Extracting metadata of recognized faces: Age, gender, hair present on head, facial hair present, hair colors, glasses present, eye makeup present, lip makeup present, smile present, mood


Example: Face recognition – detail view DAM


Updated Face Recocgnition

To prevent stereotyping and discrimination, the Azure Face Recognition API stops supporting the following attributes by the end of June 2023:

  • age
  • gender
  • bald
  • eye make-up
  • lips make-up
  • smile
  • mood
  • facial hair
  • hair color


As a consequence, these attributes are no longer automatically recognized as metadata, nor are they set (details about this decision can be found here).

In the user interface, the fields for these attributes are no longer displayed. Exception: If the fields for these attributes already contain values, they will still be displayed.

Note: Action "Refresh preview" also updates the Face Recognition. As a result, values already set for the attributes no longer supported, will be cleared.


These changes affect AI Tagging versions 5.0 (supported from Core Engine version 4.0) and 4.0.1.

Computer vision:

  • Recognition of image content/properties

  • Extracting metadata on the image content: Categories, keywords, description.

Example: Computer vision – detail view DAM


Language support:

Computer vision: Single language support for metadata, in the product standard English (US). No other languages are currently supported. 

Face recognition: Multiple language support for metadata, in the product standard German and English (US).


Files module:

AI Tagging is an extension for the Files module (DAM). 



Product standard

FEATURE OR APP IS SUBJECT TO AN EXTRA CHARGE

This feature is part of the 4ALLPORTAL product standard and available as 4App in the Marketplace, subject to the license terms.

Technical condition – Microsoft Azure Cognitive Services

Availability of a Microsoft Azure server provided by the customer for Cognitive Services.


Connection data:

  • Subscription Key – Face service
  • Subscription Key – Vision service
  • Data center URL


Data protection terms

Due to data protection terms, customers must order the Microsoft Azure server for connecting to their 4ALLPORTAL themselves.
Microsoft Azure servers ordered by 4ALLPORTAL must not be connected to a customer system.


Customer data/files must not be transferred to 4ALLPORTAL’s Microsoft Azure servers. 



Installation and operation

Installation is carried out through App Management in the 4ALLPORTAL. When you add the AI Tagging app from the marketplace, the repository “4allportal-azure-cognitive” is added to App Management. It is then possible to activate it in the system and get app updates via App Management.


The installation of the AI Tagging app includes:

  • API connection to Microsoft Azure – Cognitive Services
  • “Microsoft Cognitive Services” admin snapin under category DAM  
  • Implementation of the modules "Face Recognition Markup", "Object Markup", "Objects (AI)
  • Implementation of the subpanels "Files where this contact can be found", field "ai_celevrity" and layout configuration in the contact module
  • Implementation of the module Objects (AI)
  • Implementation of AI-specific metadata fields and object renderers in the files module


Manual installation:

  • Configuration of metadata fields (if needed)
  • Configuration of role rights

Role rights:

  • “Objects (AI)” module: Allow/limit access to the module

  • “Face Recognition Mark-up” module: Allow/limit access for the face recognition mark-ups on the image previews

  • “Object Mark-ups” module: Allow/limit access to the object recognition mark-ups on the image preview
    Access to “View” needed to display mark-ups.

    NOTE: The object recognition system for AI tagging uses the same mark-up module as the “Approvals” module.






Customizing


The following customization options are available:

  • Control visibility of metadata and mark-ups via role rights
  • Customizing the standard metadata fields
  • Activating and deactivating individual Azure Cognitive services






Licensing model


FEATURE OR APP IS SUBJECT TO AN EXTRA CHARGE

FEATURE OR APP INCLUDES COSTS FOR THE INITIAL CONFIGURATION