Content import

General architecture

The CMS provides two ways to import scheduled broadcast events and VOD content metadata from another system:

• A Content Import interface (CIM) that takes the form of XML files, which is documented below
• A REST API (which also supports content export and other functionality) that is documented in  Ingesting using the CPM APIs.
  See also  Managing products in the Subscriber Management integration guide.

The CMS supports the definition of VOD and Live metadata. The providerId is provided by the system integrator to identify the source of ingestion within CMS system

Data processing

File ingest is triggered automatically at regular intervals as defined by a CRON expression present in the configuration of the importer. This interface supports ingests of the following entities.

Live

The following items can be created or updated using the import:

• Editorial Channels
• Technical Channels
• Channel Events
• Events
• Product
• ProductLink (Marketable supported: TechnicalChannel)

The following items can be explicitly deleted using the import:

• Editorial Channels
• Technical Channels
• Product
• ProductLink

For more information see  Deleting content.

The following items can be implicitly deleted using the import:

• Events using the PeriodToDelete element

The following items are not imported:

• Profiles
• Drm Information on Technical Channels

On Demand

The following items can be created or updated using the import:

• Editorial Contents with Reference Assets
• Technical Contents with Playable Assets
• Trailers (promotion)
• Content Groups

The following items can be explicitly deleted using the import:

• Editorial Contents
• Technical Contents
• Reference Assets
• Playable Assets
• Drm Information on Technical contents
• Content Groups
• Trailers (promotion)
• TrailerLinks

For more information see Deleting content.
The following items are not imported:

• Profiles
• Image (Images must be ingested with respective entities as preformed URLs)
• AssetData

Catalogue

The following set of data can be created/updated using the import:

• Content publishing windows
• Node
• Product
• ProductLink (Marketable supported: Node, ContentPublishingWindow)

The following set of data can be explicitly deleted using the import:

• Content publishing windows 
• Node
• Product 
• ProductLink

The CMS commits all the data changes implied by the contents of a file to its database using multiple transactions. If an inconsistency is detected, either within the data of the file, or between the file’s contents and data already existing in the CMS database (configurable through business rules):

• An error will be added to the error list
• The change of the concerned entity will not be committed
• The ingest of other entities will continue

When such a case occurs, the file provider must correct the error or inconsistency by submitting a new modified xml file or updating the existing data.

File processing

The CMS import is deployed as a Content Importer (CIM) that runs on a regular basis. When launched, the CIM importer retrieves the list of files to be processed by examining a specific configured directory, looking for data files whose name match a specific pattern (generally *.xml). This directory is located on a file server accessible to the CIM. FTP/SFTP protocol is supported as well as direct file access.

Files are processed in order according to case sensitive alpha-numeric sorting of the file names. To ensure the CMS processes files in the same order they were generated, the beginning of the file name contains the date and time of the file generation in a numeric format, year first. This behaviour can be changed via the setting appropriate setting in the CIM. For details of CIM configuration, see the CIM User Guide.

To prevent the file access errors that would occur if the importer tries to process a file while it is still being generated or copied by the data provider, the data provider performs a two-stage provisioning operation. It first generates or copies the file into a working directory and/or with a file extension that is different from that the importer examines. It then changes directory and/or changes the file extension to correspond with what the importer examines. If different directories are used, it is essential that they are on the same physical disk and file system so that the operating system can perform the change in an atomic action without provoking an extra data copy. In this way, the importer will only see the files once they are complete.

When the CMS has processed the file, it returns an acknowledgement to the data provided by renaming the source file. The parameters for file extensions (filePollingPattern, processingFileExtension, successFileExtension, failedFileExtension, errorFileExtension) can be configured in the APS.properties file. The .progress suffix is removed, and the suffix .success is added if the import is successful, or .failed when errors are detected during the import. In case of failure, an error log file with the same name as the source file and an .error extension will be created and written into the same directory as the source file. The file structure is described in Error file format, below.

When processing multiple files, if an error is detected for one file, the import process will still try to import the subsequent files. 

The data provider deletes the processed files once it has taken note of the acknowledgement.

If an error occurs while processing any ingest file, any data that has already been persisted data in a previous transaction will not be rolled back (in the same ingest XML file). CIM-NIS will create an error file and carry on processing any other ingest XML files.

The following diagram shows the import file life cycle.

XML file format

The XSD does not enforce maximum or minimum lengths for string values.

The complete XML schema for this interface is available as the reference document Nagravision-Import-Full-Specification-v5.x.xsd.

The schema file name should not be included in the supplied data files. The CMS validates all incoming source files using the referenced schema file located within the application resources.

File encoding and data formats

The character encoding of the source file must correspond to the “encoding” attribute of the first line of the XML file.

The XML files ingested by the CMS should be encoded in UTF-8.

Error file format

Where the CMS import process detects an error causing it to reject the input xml file, an error file with the same name as the source file and an .error extension is generated and written in the acknowledgement failure directory. 

Multiple <error> elements may be supplied, one for each error detected.

The error xml schema is available as the reference document Nagravision-Error-Specification-v5.x.xsd.

Error file sample

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<cmsErrors xmlns="error.framework.modules.cms.sp.ml.nagra.com" scheduleDate="2004-07-01T22:00:00Z">
    <error>{"technicalMessage":null,"businessRuleMessage":"validation error message"}
    </error>
    <error>{"businessRuleMessage":"Editorial Channel editorialChannelDeletionFailure_echProviderId_editorialChannelDeletionFailure_echProviderResourceId cannot be deleted since it is used by at least one Technical Channel"}
    </error>
    <error>{"technicalMessage":"Resource ID does not match its Provider IDs","businessRuleMessage":null}
    </error>
</cmsErrors>

Related documents

The supporting documents are as listed below.

Editorial model

Technical model

Sample files