Test Plan and Environment Specification

Introduction

Purpose

This document outlines the certification test plan for integration partners integrating our new and improved OpenTV Analytics SDK. The purpose of this test plan is to validate that integrations comply with our standards, ensuring accurate and efficient data reporting.

Scope

The scope of this test plan covers the integration of the OpenTV Analytics SDK with an application to validate its functionality and compatibility. The objective is to ensure that application integrators can successfully implement the SDK and achieve seamless interaction with its APIs and features.

Audience

This document is intended for the following audiences:

Application Integrators: individuals responsible for integrating the SDK into their applications
Operators: entities that are responsible for deploying the application and SDK into a production environment
Product Manager: individuals overseeing the development and lifecycle of the product
Engineering Team: personnel who will verify that the integration and SDK usage meet the required standards and work as expected

References

Activity definition: https://docs.nagra.com/opentv-docs/latest/Default/reporting-activity-and-event-metrics-to-uav

Test Objective

Goal

An application provider integrating TV Analytics SDK must execute the test plan and ensure there are no failures as per the expectation defined in the test plan.

Key Functionalities to Be Validated

Insight events and metrics
UAV activities and events
Error handling and recovery
Data sanity and duplicity

Expected Outcome

All test cases have been executed, test results updated, and critical defects are resolved.
Test evidences (session ID, screenshots, test output data, logs) shared with NAGRAVISION.
The SDK is properly integrated with the application and is ready for production deployment.

Test Setup

Prerequisites for executing the SDK Integration Test Plan:

A test device (e.g., STB, handheld) with the application and OpenTV Analytics SDK installed
A TV set and stable network connection
To validate playback metrics and playback events scenarios, the integrator should have a Redshift user which has SELECT access to raw_playback_metrics and raw_playback_events views.
To validate UAV activity and events, the integrator should have a Redshift user which has SELECT access to 'activity' , 'playback_metrics' view.
Reporting interval and sampling interval are set in the application configuration file.

How to Access Insight Raw Data

Install and Set up SQL Client

Download and install SQL Workbench.
Download the JDBC 4.2 driver from Amazon Redshift JDBC driver. Save the .jar file, to be used to build the driver:

Build the Driver
The driver for Amazon Redshift JDBC must be installed in SQL Workbench before you can connect to the database. The following steps describe how to install the Amazon Redshift JDBC driver:

Open SQL Workbench.
Click Manage Drivers in the lower left of the screen, then click the Create a new entry icon and perform the following steps:
1. In the Name field, enter a name.
2. Click the folder icon to the right of the library text area. Navigate to the location of the driver, click to select it, and then click Open.
  The Classname field gets auto set.
3. Click OK.

Your Redshift database is now configured:

Create a Connection Profile

In the SQL Workbench interface, click the Create a new connection profile icon.
Enter a name, select your created driver from the dropdown, enter the database username, password, and URL as jdbc:redshift://<HOST>:<PORT>/<DATABASE>?ssl=true, and enable the Autocommit option.
Provide a username and password.
Click Test to ensure that your credentials are correct. If successful, click the Save Profile List icon to store the connection details.
Click OK to connect and access your database. Now you can start executing SQL queries in the SQL editor.

SQL Queries

Validate Test Scenarios for Playback Metrics

The following SQL query should be used to validate test scenarios for playback metrics. It retrieves the generated playback metrics sorted by timestamp.

SQL

select * from <schema_name>.raw_playback_metrics where playercontext.device.id='deviceId' order by timestamp desc;

Sample result:

JSON

{
    "ipaddress": "<ip>",
    "sessionid": "3c1fd-78a2-ae84-c5ee-fe237ac0a",
    "playbacktimezone": "<timezone>",
    "timestamp": 1741767730000,
    "contentinformation": "{\"channelid\":\"<channelid>\",\"channelidsource\":\"local\",\"channelname\":\"<channelname>\",\"duration\":30,\"eventid\":\"<eventid>\",\"eventname\":\"<eventname>\",\"framespersecondnominal\":0,\"genre\":[\"\"],\"type\":\"LIVE\"}",
    "playercontext": "{\"application\":{\"name\":\"<applicationname>\",\"version\":\"<version>\"},\"device\":{\"id\":\"<deviceid>\",\"type\":\"STB\"},\"hardware\":{\"manufacturer\":\"<manufacturer>\",\"model\":\"<model>\"},\"network\":{\"connection\":\"Ethernet\"},\"os\":{\"type\":\"<type>\",\"version\":\"<version>\"},\"player\":{}}",
    "contentcompletionminutes": ,
    "contentcompletionpercent": ,
    "viewingseconds": "30",
    "contentquality": "<quality>",
    "offsetfromlive": "",
    "availablebufferlevel": ,
    "bytesdownloaded": ,
    "downloadspeed": ,
    "submissionfailures": ,
    "userinfo": "{\"accountid\":\"<accountid>\",\"userid\":\"<userid>\"}",
    "submissionerrors": ,
    "globalproperties": ,
    "collectiontimestamp": "1741768099057"
  }

Validate Test Scenarios for Playback Events

The following SQL query should be used to validate test scenarios for playback events. It retrieves the generated playback events sorted by timestamp.

SQL

select * from <schema_name>.raw_playback_events where playercontext.device.id='deviceId' order by timestamp desc;

Sample result:

JSON

{
    "ipaddress": "<ip>",
    "sessionid": "gd54c6-167f-b0f9-2461-0e37838cdc6",
    "timestamp": 1741722208000,
    "playercontext": "{\"application\":{\"name\":\"<name>\",\"version\":\"<version>\"},\"device\":{\"id\":\"<deviceid>\",\"type\":\"STB\"},\"hardware\":{\"manufacturer\":\"<manufacturer>\",\"model\":\"<model>\"},\"network\":{\"connection\":\"Unknown\"},\"os\":{\"type\":\"<type>\",\"version\":\"<version>\"}}",
    "contentinformation": "{\"bitrates\":[],\"channelid\":\"<channelid>\",,\"channelidsource\":\"local\",\"channelname\":\"<channelname>\",\"duration\":0,\"eventid\":\"<id>\",\"eventname\":\"<name>\",\"genre\":[\"\"],\"type\":\"LIVE\"}",
    "playbacktimezone": "<timezone>"
    "events": "[{\"name\":\"play\",\"timestamp\":1741722208000,\"properties\":null},{\"name\":\"timeToVideoStart\",\"timestamp\":1741722208000,\"properties\":\"{\\\"time\\\":0}\"},{\"name\":\"playing\",\"timestamp\":1741722208000,\"properties\":null},{\"name\":\"stop\",\"timestamp\":1741722208000,\"properties\":null}]",
    "userinfo": "{\"accountid\":\"<accountid>\",\"address\":{},\"userid\":\"<userid>\"}",
    "collectiontimestamp": "1741768996064"
}

Validate the Test Scenarios for UAV Activities and Events

The following SQL Query below should be used to validate the test scenarios for UAV activities and events.

SQL

select * from <schema_name>.activity where accountID = <accountID> and name = <activity_name> order by createddatetime DESC

Sample result:

JSON

{
    "activityid": "<activityid>",
    "accountid": "<accountid>",
    "activitytypeid": 1,
    "name": "playbackStart",
    "value": ,
    "createddatetime": "2024-05-16 06:33:19.248",
    "modifieddatetime": ,
    "programmeid": ,
    "channelid": ,
    "playlistid": ,
    "userid": ,
    "deviceid": "<deviceid>",
    "channel": ,
    "devicetype": "<type>",
    "tags": ,
    "position": ,
    "favouritelistid": ,
    "categories": ,
    "editorialcontentid": ,
    "editorialchannelid":,
    "eventid": "",
    "contentgroupid": ,
    "contenttype": "<live-event/live-stcu-event/start-over/ltcu/npvr-event/vod-ed>",
    "recordingid": "",
    "activitymetadataid": "<metadataid>",
    "templateid": "",
    "railid": "",
    "depth": ,
    "deeplinkproviderid": ,
    "providerresourceid": ,
    "source": ,
    "appname": ,
    "appreference": ,
    "errorcode": ,
    "errormsg": ,
    "playertypeversion": ,
    "playbacksessionid": "5cbfbf-93fc-4f17-ae0a-bb37031d179",
    "serieslink": ,
    "adsupplier": ,
    "trackingassetid": ,
    "adid": ,
    "startposition": ,
    "stopposition": ,
    "appsessionid": "61e913-724b-4238-9362-cc81fe9282",
    "sectionid": ,
    "seriesid": ,
    "contentsource": "OTT/Blend/IPTV",
    "activitydatetime": ,
    "editorialid": "<editorial_id>",
    "userprofileid": ,
    "activitytype": "Activity",
    "playouturl": ,
    "sdkversion": ,
    "appversion": ,
    "clientpageorscreen": ,
    "viewer_location": 
}

Note:

<schema_name> will be provided by NAGRAVISION.
Replace <deviceId> with the device ID of the device under test.
Replace <accountID> with the account ID of the device under test.
Replace <activity_name> with the name of UAV activity that is being validated.
Data latency:
- Playback Metrics and Playback Events: approx. 1–2 minutes
- UAV Activities and UAV Events: approx. 20–25 minutes

Environment Details

Please provide the following information when sharing your rest results with NAGRAVISION.

System & Hardware Details

Hardware Model:
Hardware Version:
Device Type:
Network Type:
DeviceID:

Software Details

SDK Version:
SDK Configuration:
Application Version:
Application Configuration:
API Endpoint:

Test Execution Details

Execution Date:
Tester Details:

Data Definition

Field	Definition	Data Type	Unit
`accountId`	The account ID of the current signed-in account	string	N/A
`activityDateTime`	Date and time in UTC that the event/action being logged occurred	Date and Time	UTC date time
`adID`	Unique advertisement reference, VAST tag	string	N/A
`adSupplier`	Name or unique ID of the advert supplier	string	N/A
`appName`	Name of the application reporting the metrics. This is used to easily identify the app in the database. Equivalent to tenant ID in the token.	string
`appReference`	Reference for the application. This could be an ID or text value – DEPRECATE	string
`appSessionId`	Unique ID to identify the current session of the application, generated by the application upon start up and persisted until next start up. The `appSessionId` is used to indicate a period of time wherein a user interacts with the app. Usually triggered by the opening of an app, a session records the length and frequency of app use to show developers, marketers, and product managers how much time users spend within an app. An app session ID should be generated/starts when a user opens an application and ends when they exit it.	string	UUID / GUID
`appVersion`	Version number for the currently deployed application	string	x.xx.xx
`available_Bitrate`	Text value to indicate the bitrates available for the content being streamed, available from the player. Content available bitrates in bits per second (bps).	string collection	bit rate per second (bps)
`avg_download_bitRate`	The average value of the bitrate stream, available from the player. The average selected bitrate in bits per second (bps) (sampled and averaged within the reporting period)	Decimal round up	bit rate per second (bps)
`bitrate_selected`	Bitrate selected for viewing (streaming) available from the player	Decimal round up	bit rate per second (bps)
`buffer_Duration`	At the the time of reporting, the total available buffer in seconds. The value is available from the player.	Int round up	seconds
`bytes_Downloaded`	Total number of bytes downloaded at the time of reporting the metrics, available from the player	Decimal round up	Mbps
`categories`	The categories attached to the content being viewed, where available.	string
`channel`	The name of the channel being watched	string
`channelId`	The ID of the channel being watched	string
`clientLoggedDateTime`	NEW – the date and time the event is logged as per the date and time set on the client device. (This may not be the same as the date and time set server-side.)	date time	UTC date time
`clientpageorscreen`	Name of the current page or screen the user is interacting with	string
`consentForTargettedAd`	If the user has consented to targeted advertising, this value should be reported as `true`.	boolean	boolean
`contentgoupid`	The content group ID if known by the client. This is optional as it is unlikely to be available.	string
`content_Type`	Indicates the type of content being consumed – see the table for expected values.	string
`content_URI`	URI of the video stream, available from the player. URI pointing to the master playlist.	string	URI
`contentQualityChanged`	Current content quality (SD, HD, 4K, 8K, Unknown)	string	one of `SD`/`HD`/`4k`/`8k`/`Unknown`
`contentSource`	One of the following: `IPTV` – TV content is delivered over an IP network (not the open Internet) `OTT` – TV content is delivered over-the-top (OTT), that is, via the Internet – DVB and more `Blend` – a mixture of IPTV and OTT	string	One of `IPTV`/`OTT`/`Blend`
`deepLinkId`	Unique value assigned to the deep link being triggered. This will be found in the metadata supplied to the client. A deep link is a reference from some content to the same content in an external, third-party system. For example the same movie present in Netflix, Disney+, or Apple TV.	string
`deeplinkProviderId`	Unique value assigned to the deep link provider. This will be found in the metadata supplied to the client.	string
`depth`	Relevant for Rails interactions, the depth of the item interacted with by the user, click, media card, or playout. The depth is the vertical position of the item in the array of Rails.	int
`deviceId`	Unique device ID as provided by the platform to the client	string
`deviceType`	Devices can be classed as either Set Top Boxes (STB) or open devices. Within these two categories, sub-categories of device types are available with different hardware and different expected use cases, resulting in differing UI functionality and requirements. These are not strict categories however, as any device could have any type of hardware.	string
`dropped_Frames`	Number of frames dropped from the video stream in the reporting period, available from the player	int
`editorialID`	For Start-Over or STCU, the required value is sourced from the `editorialId` of the event. For `live-stcu-event`, the required value is sourced from the `editorialId` of the event. For LTCU, the required value is sourced from the `editorialContentId` value.	string
`editorialchannelid`	The editorial channel ID for the current channel being watched	string
`editorialcontentid`	The editorial content ID for the current content being watched	string
`errorCode`	Unique code that relates to an error. This can be a standard code or client application-specific.	string
`errorMsg`	The relevant error message related to the error,	string
`EventId`	For live events, the ID of the specific event being watched	string
`hdepth`	For Rails, the horizontal depth of the Rail item being interacted with / reported on, starting at 0 and counted from the first item / left-most item in the Rail incrementally	int
`metadata`	Within this section, you can add any key/value pair required for reporting. This data will be stored in the metadata table within the DWH.	string	as required
`playbackProgress`	The position in seconds that the current action relates to in the current content being streamed in seconds. This should be relevant to the content being viewed in totality, i.e., if the user is watching the 20th minute of a show the value reported should be 1200 or if the user starts playout from a bookmark at the 10th minute then the value reported should be 600.	int	seconds
`playbackSessionId`	A unique playback session ID to be generated by the client application. This session is specific to the playout of content – each time the content being played is changed, either via a selection triggered by the user or due to the channel playout content changing, a new session ID should be generated and used when reporting playout session actions. The session ID (UUID) corresponding to the current session. For non-linear content such as VOD, a playback session occurs between the start of playing of a video until it stops (or aborts or crashes, not interrupted by pauses or seeks).	string	GUID / UUID
`playerTypeversion`	Version number of the player being used/integrated into the client	string
`playoutURL`	The URL of the content being streamed, the video stream URL, URI pointing to the master playlist	string	URI
`policyConsentGiven`	If the user has consented to the policy shown on screen, this value should be reported as `true`, if not `false`.	boolean	true/false
`policyRead`	If the user has indicated they have read the policy, this value should be reported as `true`, if not `false`.	boolean	true/false
`policyShown`	If the client has shown the policy on screen to the end user, this value should be reported as `true`, if not `false`.	boolean	true/false
`policyType`	The type of policy shown/consented to, e.g., Data Retention	string
`position`	The position in seconds that the current action relates to in the current content being streamed in seconds. This should be relevant to the content being viewed in totality, i.e., if the user is watching the 20th minute of a show, the value reported should be 1200. If the user starts playout from a bookmark at the 10th minute, then the value reported should be 600.	int	seconds
`programmeId`	Identifier for a TV programme as provided by the platform in the metadata response. A live TV programme. It is shown on a specific service at a certain time. See `/btv/programmes`.	string
`providerresourceid`	Any resource IS that is required for tracking	string
`railId`	The unique IS of the Rail being interacted with or reported on, as supplied in the RAILS metadata response form the head end	string
`recordingid`	The ID of the recording that the user is watching
`SDKVersion`	The video metrics SDK version installed on the client application	string	x.xx.xx
`selectedBitrateChanged`	Bitrate in bps that was selected after bitrate changed event	int	bits per second bps
`seriesId`	The series IS as supplied in the metadata from the head end	string
`source`	Broadcast source (DVB or OTT) for each channel	string
`stream_Bitrate`	Tthe current bit rate being consumed, supplied by the player	int	bits per second bps
`startposition`	Position in seconds in the current content being interacted with that the user initiated, e.g., start skip forward at the position 1200	int	seconds
`stopposition`	Position in seconds in the current content being interacted with that the user initiated, e.g., stop skip forward at the position 1500	int	seconds
`technicalId`	This unique ID is specific to the content stream provided to the client, supplied in the response to the client. The technical part of the content (Playback URI, Device Type, etc.). Usually hidden from the end-user, describes to the device if it is allowed to play content and where to get it.	string
`templateId`	Rails template ID that the client is rendering for the user to interact with, supplied in the response to the client	string
`timeToVideoStart`	Time taken from `setUrl` to render of first frame on screen	int	seconds
`total_Frames`	Total frame count, available from the player	int
`trackingAssetId`	A unique ID for tracking, as generated by the client	string

Data Mapping

Note: all payload keys are case insensitive.

Activity

Raw Activity	Payload Key
`activityid`	Auto-generated
`accountid`	`accountID`
`activitytypeid`	Auto-generated
`name`	`name`
`value`	`value`
`createddatetime`	`timestamp`
`modifieddatetime`	Not Used
`programmeid`	`programmeId`
`channelid`	Not used
`playlistid`	`Not used`
`userid`	`userId`
`deviceid`	`deviceId`
`channel`	Not used
`devicetype`	`deviceType`
`tags`	`bookmarksTags, playlistsTags, remindersTags, historiesTags`
`position`	`Position`
`favouritelistid`	`favouritesFavouriteListId`
`categories`	`favouritesCategories`
`editorialcontentid`	Not Used
`editorialchannelid`	Not Used
`eventid`	Not Used
`contentgroupid`	Not Used
`contenttype`	`contentType`
`recordingid`	`recordingId`
`templateid`	`templateId`
`railid`	`railId`
`depth`	`depth`
`deeplinkproviderid`	`deeplinkId, deeplinkProviderId`
`providerresourceid`	`providerResourceId`
`source`	`source`
`appname`	`appName`
`appreference`	`appReference`
`errorcode`	`errorCode`
`errormsg`	`errorMsg`
`playertypeversion`	`playerTypeversion`
`playbacksessionid`	`playbackSessionId`
`serieslink`	`seriesLink`
`adsupplier`	`adSupplier`
`trackingassetid`	`trackingAssetId`
`adid`	`adID`
`startposition`	`startPosition`
`stopposition`	`stopPosition`
`appsessionid`	`appSessionId`
`sectionid`	`sectionId`
`seriesid`	`seriesId`
`contentsource`	`contentSource`
`activitydatetime`	`activityDateTime`
`editorialid`	`editorialID`
`playouturl`	`PlayoutURL`
`sdkversion`	`SDKVersion`
`appversion`	`appVersion`
`clientpageorscreen`	`clientpageorscreen`
`activity_id`	`activityId`
`userprofileid`	`UserProfileId`
`activitymetadataid`	Auto-generated
`ingestion_datetime`	Auto-generated
`clientcountrycode`	`clientCountryCode`

Playback Metrics

Raw Playback metrics	Payload Key
`activityid`	Auto-generated
`selectedbitrate`	`selectedBitrate`
`createddatetime`	`timestamp`
`accountid`	`accountId`
`operatorid`	Not used
`name`	`name`
`userid`	`userId`
`deviceid`	`deviceId`
`contentid`	`editorialId`
`availablebitrates`	`availableBitrates`
`bytesdownloaded`	`bytesDownloaded`
`downloadbitrate`	`downloadBitrate`
`downloadbitrateaverage`	`downloadBitrateAverage`
`uri`	`uri`
`bufferedduration`	`bufferedDuration`
`streambitrate`	`streamBitrate`
`droppedvideoframes`	`droppedVideoFrames`
`totalvideoframes`	`totalVideoFrames`
`contenttype`	`contentType`
`appsessionid`	`appSessionId`
`playbacksessionid`	`sessionId, playbackSessionId`
`playbackprogress`	`playbackProgress`
`activitymetadataid`	Auto-generated
`contentsource`	`contentSource`
`ingestion_datetime`	Auto-generated
`metadata_activitiesappname`	`activitiesappname`
`metadata_activitiesappversion`	`activitiesappversion`
`metadata_appname`	`appname`
`metadata_appversion`	`appversion`
`metadata_deviceid`	`deviceid`
`metadata_drmtimeseconds`	`drmtimeseconds`
`metadata_licensetime`	`licensetime`
`metadata_livelatency`	`livelatency`
`metadata_maxbitrate`	`maxbitrate`
`metadata_maxresolution`	`maxresolution`
`metadata_maxsegmentduration`	`maxsegmentduration`
`metadata_minbitrate`	`minbitrate`
`metadata_position`	`position`
`metadata_programmeid`	`programmeid`
`metadata_recordingid`	`recordingid`
`metadata_selectedresolution`	`selectedresolution`
`metadata_seriesid`	`seriesid`
`metadata_technicalid`	`technicalid`
`metadata_loadlatency`	`loadlatency`
`metadata_manifesttimeseconds`	`manifesttimeseconds`