-
Notifications
You must be signed in to change notification settings - Fork 0
3.4. Standards: L4: Vendor Full Stack ‐ Video ‐ Smoke Regression Test
Ulrond edited this page Oct 18, 2024
·
1 revision
This document outlines the automated test plan for L4 (Vendor Full Stack) testing of RDK Video releases. These tests focus on verifying the stability and functionality of a new vendor release layer when integrated with the existing application (APP) and middleware (MW) layers.
Goal: To automate the execution of smoke and regression tests that are currently performed manually, improving efficiency and reducing testing time.
Framework: python_raft will be used to automate these tests, enabling engineers and Jenkins to trigger them as needed.
Test Approach:
-
Build: A full stack image is created using the previous
APP/MWlayers and the newPre-Release Vendor Layer. -
Automation:
python_raftscripts will be developed to automate the test cases described below. - Execution: Tests can be triggered by engineers or integrated into the Jenkins CI/CD pipeline.
-
Reporting:
python_raftwill provide test results and logs for analysis.
Important: These tests are designed as a pre-release quality gate for the vendor layer, ensuring no regressions are introduced before the layer is released.
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| A1 | General Sanity on APP Playchannels: Play channels for 5 minutes, changing channels via remote. | ||
| A2 | Sanity on Linear SDR Channel: Test channel 101/501. | ||
| A3 | Sanity on Linear UHD Channel: Test channel 406. | ||
| A4 | Sanity on Linear Channel Requiring PIN: Test channels 301, 302. | ||
| A5 | Sanity on VOD (SDR): Test SDR VOD content. | ||
| A6 | Sanity on VOD (UHD): Test UHD VOD content. |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| B1 | APPS - YouTube - SDR: Test YouTube with SDR content. | ||
| B2 | APPS - YouTube - HLG: Test YouTube with HLG content. | ||
| B3 | APPS - YouTube - HDR10: Test YouTube with HDR10 content. | ||
| B4 | APPS - Netflix - SDR: Test Netflix with SDR content. | ||
| B5 | APPS - Netflix - DV: Test Netflix with Dolby Vision content. | ||
| B6 | APPS - Disney+ - SDR: Test Disney+ with SDR content. | ||
| B7 | APPS - Disney+ - DV: Test Disney+ with Dolby Vision content. | ||
| B8 | APPS - Amazon Prime - SDR: Test Amazon Prime with SDR content. | ||
| B9 | APPS - Amazon Prime - HDR10: Test Amazon Prime with HDR10 content. | ||
| B10 | APPS - Amazon Prime - DV: Test Amazon Prime with Dolby Vision content. | ||
| B11 | APPS - Apple TV - DV: Test Apple TV with Dolby Vision content. | ||
| B12 | APPS - Paramount+ - SDR: Test Paramount+ with SDR content. | ||
| B13 | APPS - YouTube Kids - SDR: Test YouTube Kids with SDR content. | ||
| B14 | APPS - BBC iPlayer - SDR: Test BBC iPlayer with SDR content. | ||
| B15 | APP - Peacock: Test Peacock app. |
| Test Case | Description | Status | Remarks |
|---|---|---|---|
| C1 | YouTube to: Netflix, Amazon Prime, Disney+, Apple TV, Paramount+, YouTube Kids, Peacock. | ||
| C2 | Netflix to: YouTube, Amazon Prime, Disney+, Apple TV, Paramount+, YouTube Kids, Peacock. | ||
| C3 | Disney+ to: YouTube, Amazon Prime, Netflix, Apple TV, Paramount+, YouTube Kids, Peacock. | ||
| C4 | Apple TV to: YouTube, Amazon Prime, Netflix, Disney+, Paramount+, YouTube Kids, Peacock. | ||
| C5 | Paramount+ to: YouTube, Amazon Prime, Netflix, Disney+, Apple TV, YouTube Kids, Peacock. | ||
| C6 | YouTube Kids to: YouTube, Amazon Prime, Netflix, Disney+, Apple TV, Paramount+, Peacock. | ||
| C7 | Peacock to: YouTube, Amazon Prime, Netflix, Disney+, Apple TV, Paramount+, YouTube Kids. | ||
| C8 | Amazon Prime to: YouTube, Netflix, Disney+, Apple TV, Paramount+, YouTube Kids, Peacock. |
| Test Case | Description | Status(Pass/Fail) | Remarks |
|---|---|---|---|
| D1 | Factory Reset and Connect: Factory reset the panel and connect to 5GHz/2.4GHz SSID during setup. | ||
| D2 | Connect Ethernet: Connect Ethernet while the panel is on 5GHz/2.4GHz Wi-Fi. | ||
| D3 | Disconnect Ethernet: Disconnect Ethernet and check if it resumes the last Wi-Fi connection. | ||
| D4 | Verify IP Address: Check if the panel's IP address matches the one assigned by the router. | ||
| D5 | Play/Stream Videos: Play/stream 4K/HDR/UHD videos on 5GHz/2.4GHz network. | ||
| D6 | Soft Reboot (Wi-Fi): Soft reboot the panel while connected to 5GHz/2.4GHz and ensure reconnection. | ||
| D7 | Boot Up Time (Wi-Fi): Check boot-up time when connected to 5GHz/2.4GHz. | ||
| D8 | Hard Reboot (Wi-Fi): Hard reboot the panel while connected to 5GHz/2.4GHz and ensure reconnection. | ||
| D9 | Boot Up Time (Wi-Fi): Check boot-up time when connected to 5GHz/2.4GHz. | ||
| D10 | Reset Network: Reset the network while the panel is on 5GHz/2.4GHz. | ||
| D11 | Soft Reboot (No Internet): Soft reboot after network reset and ensure the panel boots with no internet. | ||
| D12 | Boot Up Time (No Internet): Check boot-up time with no internet. | ||
| D13 | Hard Reboot (No Internet): Hard reboot after network reset and ensure the panel boots with no internet. | ||
| D14 | Boot Up Time (No Internet): Check boot-up time with no internet. | ||
| D15 | Disable Wi-Fi: Disable Wi-Fi via settings and ensure the "no internet" screen appears. | ||
| D16 | Hard Reboot (No Internet): Hard reboot and ensure the panel boots with no internet. | ||
| D17 | Disable Wi-Fi: Disable Wi-Fi via settings and ensure the "no internet" screen appears. | ||
| D18 | Soft Reboot (No Internet): Soft reboot and ensure the panel boots with no internet. | ||
| D19 | Enable Wi-Fi: Enable Wi-Fi and ensure the panel connects to the last Wi-Fi connection. | ||
| D20 | Power Cycle Router: While connected to 5GHz/2.4GHz, power cycle the router and ensure the panel reconnects. | ||
| ... | ... | ... | ... |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| E1 | Verify Motion Events: Verify motion events are detected. | ||
| E2 | HDMI Content and Motion: Play HDMI content and verify motion events. | ||
| E3 | Linear Channel and Motion: Play linear channels and verify motion events. | ||
| E4 | Apps and Motion: Play apps like YouTube/Netflix/Disney+ and verify motion events. | ||
| E5 | Power Cycle and Motion: Power cycle the panel and check motion events. | ||
| E6 | Standby and Motion: Put the panel in standby, wake it up, and check motion events. | ||
| E7 | Deepsleep and Motion: Put the panel in deep sleep, wake it up, and check motion events. | ||
| E8 | Auto-Standby: Ensure the panel enters standby mode after 50 minutes of inactivity. | ||
| E9 | Banner Removal: Check if the banner is removed by generating | ||
| motion at the 51st minute. | |||
| E10 | Linear Channel and No Banner: Ensure no banner appears after 50 minutes of activity on linear channels. | ||
| ... | ... | ... | ... |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| F1 | Checking LED behaviour on power cycle from ON state | ||
| F2 | Checking LED behaviour on power cycle from standby mode state | ||
| F3 | Checking LED behaviour on power cycle from Deepsleep mode state | ||
| F4 | Checking LED behaviour on Soft reboot the panel from ON state | ||
| F5 | Checking LED behaviour on Soft reboot the panel from standby state | ||
| ... | ... | ... | ... |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| G1 | Wake panel via IR remote --HOME button | ||
| G2 | Wake panel via IR remote --Power button | ||
| G3 | Wake panel via IR remote --PRIME Button | ||
| G4 | Wake panel via IR remote -NETFLIX button | ||
| G5 | Navigate the EPG page, try trick modes & different buttons on remote (IR) | ||
| G6 | Pair the remote on BT | ||
| G7 | Unpair the remote & pair it again | ||
| G8 | Navigate the EPG page, try trick modes & different buttons on remote (BT) | ||
| G9 | Wake panel from standby mode via BT remote --HOME button | ||
| G10 | Wake panel from standby mode via BT remote --Power button | ||
| G11 | Wake panel from deep sleep mode via BT remote --HOME button | ||
| G12 | Wake panel from deep sleep mode via BT remote --Power button |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| H1 | Mic enable via side button - Launch channel, apps, volume etc via FFV | ||
| H2 | Mic enable via side button - Put panel in standby via FFV | ||
| H3 | Mic enable via side button - Wake panel via FFV from standby | ||
| H4 | Mic enable via side button - Put panel in deep sleep & wake it via FFV | ||
| H5 | Mic enable via side button - Soft reboot panel & check if MIC status remains persistent | ||
| H6 | Mic enable via side button - Hard reboot panel & check if MIC status remains persistent | ||
| H7 | Mic disable via side button - Launch channel, apps, volume etc via FFV | ||
| H8 | Mic disable via side button - Put panel in standby via FFV | ||
| H9 | Mic disable via side button - Wake panel via FFV from standby | ||
| H10 | Mic disable via side button - Put panel in deep sleep & wake it via FFV | ||
| H11 | Mic disable via side button - Soft reboot panel & check if MIC status remains persistent | ||
| H12 | Mic disable via side button - Hard reboot panel & check if MIC status remains persistent | ||
| ... | ... | ... | ... |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| I1 | Launch HDMI source via HDMI Source remote | ||
| I2 | Play any content on HDMI source & check AV | ||
| I3 | Change different resolution on HDMI source | ||
| I4 | When panel is in standby, wake panel via HDMI source | ||
| I5 | When panel in deep sleep, wake panel via HDMI source | ||
| I6 | Hot plug HDMI source | ||
| I7 | Switch between HDMI source, APPS |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| J1 | Connect HDMI speaker on HDMI & check sound coming from speakers | ||
| J2 | Hot plug HDMI speakers & make sure Audio switches back properly | ||
| J3 | Check any AV sync issue | ||
| J4 | Check audio for contents on HDMI source when HDMI speakers connected | ||
| J5 | Put panel in standby & wake it up, check the audio coming from HDMI speakers | ||
| J6 | Put panel in deep sleep & wake it up, check the audio coming from HDMI speakers |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| K1 | Pair BT speakers & check sound coming from speakers | ||
| K2 | Check any AV sync issue on BT speakers while playing apps, HDMI, linear channel | ||
| K3 | Check audio for contents on HDMI source when BT speakers connected | ||
| K4 | Change the output of audio via quick menu | ||
| K5 | Put panel in standby & wake it up, check the audio coming from BT speakers | ||
| K6 | Put panel in deep sleep & wake it up, check the audio coming from BT speakers |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| L1 | Checking AV of DTT channel (SD & HD channels) | ||
| L2 | Scanning DTT channels | ||
| L3 | Switch between DTT channels & HDMI source | ||
| L4 | Browsing TV guide, signal information, audio etc in DTT channels option |
| Test Case | Description | Status (Pass/Fail) | Remarks |
|---|---|---|---|
| M1 | Hard reboot (power cycle) 1 | ||
| M2 | Hard reboot (power cycle) 2 | ||
| M3 | Hard reboot (power cycle) 3 | ||
| M4 | Soft reboot (/rebootNow.sh) 1 | ||
| M5 | Soft reboot (/rebootNow.sh) 2 | ||
| M6 | Soft reboot (/rebootNow.sh) 3 |