FEWS-NL
Acceptance Test Document
FEWS-NL
Acceptance Test Document
Albrecht Weerts
Contents
1 Test Record ...1— 1 2 Introduction...2— 1
2.1 Purpose of this document...2— 1 2.2 Scope of this document...2— 1 2.3 Context of this issue...2— 1
3 Setting up the Test...3— 1
3.1 Installation of SAT-release...3— 1 3.2 Test Procedure...3— 1
4 MC2.3 System Management ...4— 1
WL | Delft Hydraulics i i
4.8 MC2.3.9 System Management - Control of currently executing tasks..4— 25 4.8.1 View currently executing tasks...4— 25 4.8.2 Kill currently executing tasks ...4— 27 4.9 MC2.3.13 System Management – Administration Interface User
1
Test Record
Test Performed By A. Weerts (WL | Delft Hydraulics)& E. Sprokkereef (RIZA Arnhem)
A. Weerts (WL | Delft Hydraulics) & L. van Hal (RIZA Arnhem)
Date of Test 4/7/2006 & 2/5/2006
Version of Software Tested FewsNL 1.70 Mei 2006 Overall pass/fail Pass
Observations & comments
WL | Delft Hydraulics 2 — 1
2
Introduction
2.1
Purpose of this document
This document defines the steps to be taken to perform system tests and to compare the results of the tests with the expected outcome for each test. It is intended to be printed and the test pass/fail information hand written on it during testing. The annotated document plus any additional materials (printouts etc) form the test record and will be filed.
2.2
Scope of this document
This document is intended to test the FEWS-NL system as it stands.
2.3
Context of this issue
This FEWS-NL Acceptance Testing Document has been produced to provide formal testing of the FEWS-NL.
The tests described in this document, cover the following areas of testing:
· Tests relating to the operation of automated forecasts by the Master Controller
3
Setting up the Test
3.1
Installation of SAT-release
Before commencing testing, SAT-Release must be installed on a suitably configured PC. The minimum requirements for the PC to be used for running FEWS-NL SAT release are:
Item Description
Operating System Windows 2000
Java Virtual Machine Sun JVM Release 1.4.2
Hard Disk Space 1 GB (per region)
Installed RAM memory 256 MB
Display Resolution 1024x768 pixels
Processor speed 1 GHz
3.2
Test Procedure
The test in this release are focussed on the Master Controller’s management and control of scheduled tasks within FEWS-NL. For the majority of users, the Master Controller is not directly visible, it performs the background processing to allow forecasts to be automatically generated and stored. The Operator Client, the functionality of which is tested elsewhere, handles the actual display and manipulation of the forecasts.
WL | Delft Hydraulics 4 — 1
4
MC2.3 System Management
These sets of tests are designed to confirm the functionality provided to manage the FEWS-NL. The majority require the use of the Administration Interface.
4.1
MC2.3.1 System Management – FEWS-NL Monitoring
These sets of tests focus on the use of the Administration Interface to monitor the operation of the FEWS-NL.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface user-id and password
FEWS-NL available and operating, with no FSS tasks initially running or scheduled to run in the next half hour
Remote MC is configured but not running.
Test workflow (use the Import workflow) present within the FEWS-NL, with a single mapping defined to FSS ‘FSS00’ (only).
Valid task properties file import_properties.xml is available on the machine on which the web browser is running (this may be obtained by downloading the properties from an existing task).
All pending forecast tasks suspended on the MCs.
No. Action Expected result Pass Fail
MC2.3.1.1 Viewing Live System Status: Log onto the Administration Interface with a valid userid and password.
Navigate to the ‘View Live System Status’ screen.
This can be found under
System Status -> Live System Status in
the navigation bar.
No. Action Expected result Pass Fail MC2.3.1.2 Live system status indicators:
Examine the live system status table. Fields will be present forthe following indicators: - Currently executing
task runs
- Currently dispatched task runs(executing or waiting to be executed)
- Current OC sessions - Log table size - MC failure status For the remote (if present) MC:
- Failover status For each FSS:
- Queue length For each MC module:
- Whether ‘up’ MC2.3.1.5 Remote MC failover:
Bring down the MCProxy windows services on the remote MC’s FSSs, and wait for 5 minutes.
The remote MC will have its status indicated (on the local MC’s Administration Interface) as ‘failed’.
MC2.3.1.6 FSS loss:
Bring down the MCProxy windows services on the local MC’s FSSs, and wait for 3 minutes.
The FSSs’ statuses will have an indication to the effect of ‘down’. The MC’s failure status will be indicated as ‘failed over’. MC2.3.1.7 FSS status:
Start sufficiently many of the MCProxy windows services for the MC not to be considered ‘failed’. (Criteria for ‘failure’ are defined, in terms of responsive FSSs, in the MC’s configuration file.)
The started FSSs will have their queue lengths indicated as zero. The ‘currently executing task runs’ and ‘currently dispatched task runs’ will indicate zero.
The MC’s failure status will be indicated as ‘OK’.
MC2.3.1.8 FSS queue length:
Schedule 4 new tasks, each to run the import workflow using task properties file
“import_properties.xml”, due to run once only.
These task may be scheduled in any way seen fit, although the simplest way may be to use the Administration Interface itself and navigate to:
Tasks -> Schedule New Task
Ensure that this workflow is mapped to the FSS “FSS00”
Wait until the new tasks become due.
As the new tasks become due, the ‘FSS00’ FSS will have its queue length indicated as ‘4’.
The ‘currently executing tasks runs’ will indicate ‘1’, and the ‘currently dispatched task runs’ will indicate ‘4’.
WL | Delft Hydraulics 4 — 3
No. Action Expected result Pass Fail
Every 30 seconds or so, as task runs complete, the ‘FSS00’ queue length and the ‘currently dispatched task runs’ indicators will each decrease by 1.
The ‘log table size’ will increase as each new task run begins executing. Once the final task run has completed, the ‘currently executing task runs’ will return to zero. MC2.3.1.9 MC Components:
Shut down the system reporter.
Log entries will indicate that the system reporter has found to have failed and been restarted. The health of the modules of the other MC components: - OC Listener - FS Listener - Synchronisation - Task Manager - System Monitor will remain indicated as ‘up’ in the system status screen.
MC2.3.1.10 Current OC sessions:
Start an Operator Client session on this MC.
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 5
4.2
MC2.3.2 System Management – FEWS-NL Start-up
These sets of tests focus on the use of the Administration Interface to start-up the FEWS-NL.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface user-id and password FEWS-NL available and operating
No. Action Expected result Pass Fail
MC2.3.2.1 Master Controller Startup:
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 7
4.3
MC2.3.3 System Management – FEWS-NL
Shut-down/Re-start
These sets of tests focus on the use of the Administration Interface to shut-down the FEWS-NL.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface user-id and password FEWS-NL available and operating
No. Action Expected result Pass Fail
MC2.3.3.1 Master Controller Re-start: Start the Master Controller
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 9
4.4
MC2.3.4 System Management – Management of System
Logs
These sets of tests focus on the use of the Administration Interface to manage the FEWS-NL System Logs.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
No. Action Expected result Pass Fail
MC2.3.4.1 Viewing System Logs: Log onto the Administration Interface with a valid userid and password.
Navigate to the ‘View Logs’ screen.
This can be found under
System Status -> View Logs in the
navigation bar.
A table appears containing details of various system messages. Options exist to:
1) Download the log to a text file.
2) Restrict the number of messages displayed (50,100,250,500) 3) Restrict the display
to certain log levels. MC2.3.4.2 Sorting System Logs:
In the View Logs screen, click on the column headings to sort the logs displayed.
Clicking on each heading sorts the log entries by that column, as follows: 1) “Code” – by log code text 2) “Entry Time” 3) “Level” – by log severity level 4) “Type” – by log type
No. Action Expected result Pass Fail MC2.3.4.3 Display additional Information:
Locate a log entry text that has additional information.
(Such entries arecoloured blue and underlined.)
Clicking on such an entry causes the
additional information to be displayed.
MC2.3.4.4 Filter System logs by severity: Filter the displayed log entries by severity level. Select each available severity level in turn and examine the log entries displayed.
The following levels will be available, in increasing order of severity: “Debug”, “Info”, “Warning”, “Error”, “Fatal Error”. Selecting any severity level causes the display to include only entries with that level or greater.
MC2.3.4.5 Filter System logs by number: Adjust the number of log entries displayed.
This is achieved using the Entries per
page drop down list.
The following settings will be available: 50, 100, 250, 500.
On selecting 50 entries per page, 50 log entries will be displayed. On selecting the other settings in turn, the number of log entries displayed will increase in each case.
MC2.3.4.6 Download System log to file: Use the ‘download logs’ facility to save the logs to a text file in an appropriate location.
Open this file in a text editor.
The file will contain, in tab-separated format, the same log entries as displayed in the ‘View Logs’ screen.
MC2.3.4.7 Purge logs by date:
Navigate to the ‘Manage Logs’ screen.
This can be found under
System Status -> View Logs in the
navigation bar.
Purge all log entries made before midnight yesterday.
Return to the ‘View Logs’ screen, and sort the entries in ascending date/time order.
The earliest remaining log entry will be dated yesterday or today.
MC2.3.4.8 Purge logs by severity:
Return to the ‘Manage Logs’ screen. Purge all log entries with a log level of ‘debug’.
Return to the ‘View Logs’ screen, and sort the entries in ascending
WL | Delft Hydraulics 4 — 1 1
Please make any addition comments on the test here
4.5
MC2.3.5 System Management - Configuration of
scheduled tasks
These sets of tests focus on the use of the Administration Interface to view and configure scheduled tasks.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
All pending scheduled tasks suspended.
4.5.1 View Scheduled task information
No. Action Expected result Pass Fail
MC2.3.5.1 Viewing Scheduled Tasks: Log onto the Administration Interface with a valid userid and password.
Navigate to the ‘View Scheduled Tasks’ screen.
This can be found under
Forecast Tasks -> Scheduled Tasks in
the navigation bar.
A table appears listing information on scheduled tasks. An option will be available to
“Schedule a new task”.
MC2.3.5.2 Sorting Scheduled task information:
In the Scheduled Tasks screen, click on the column headings to sort the Scheduled Task display.
Clicking on each heading sorts the tasks by that column, as follows: 1) “Task ID” – by task ID 2) “Workflow ID” 3) “Priority”
4) “Next Due Time” – by time next due 5) “Repeat Time” 6) “Task Status”
WL | Delft Hydraulics 4 — 1 3
No. Action Expected result Pass Fail
level)
Clicking on the same heading a second time in succession sorts the log entries is the opposite direction.
MC2.3.5.3 View Scheduled task details: Choose a scheduled task in the ‘View Scheduled Tasks’ screen, and view its details.
This can be achieved by follow the task’s “Details” link.
The following
information is displayed: (The values will be dependent upon the task selected.) 1) Task ID: 2) Task Status 3) Workflow ID 4) Can Run on Failover 5) Task Priority 6) Task First Due
Time
7) Task Repeat Time A facility will be present to download the Task Properties file. MC2.3.5.4 Download Scheduled Task
properties file:
In the ‘Task Details’ screen, use the ‘download task properties’ facility to save the Task Properties XML file to an appropriate location.
Open this Task Properties file in a text editor.
The file will be in the similar format as given in the table below.
Format of an example Task Properties file (the exact file may appear different): <?xml version="1.0" encoding="UTF-8" ?>
- <taskProperties xmlns="http://www.wldelft.nl/fews"> <description>Actually import everything</description> <workflowId>ImportExternal</workflowId> - <taskSelection> - <scheduledTask> - <schedulingPeriod> <startDate>2004-03-12T17:00:00.000Z</startDate> <endDate>2004-03-22T17:00:00.000Z</endDate> </schedulingPeriod>
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 1 5
4.5.2 Add a scheduled task
No. Action Expected result Pass Fail
MC2.3.5.5 Enter new task details: Log in to the Administration Interface and navigate to the ‘View Scheduled Tasks’ screen.
Invoke the facility to schedule a new task.
Use the “Schedule New Task” link.
Obtain a task properties file for the import external workflow by
downloading it from an existing task. Enter the following task details: 1) Workflow ID: import external 2) Priority: Normal
3) First due time: today, within the next hour
4) Repeat time: 30 minutes 5) Task properties: upload the
properties file downloaded previously.
Submit this task, and view its details.
On submitting the task, it will appear as an entry in the ‘View Scheduled Tasks’ table.
On viewing its details, the details will be as entered.
MC2.3.5.6 Upload a faulty or non task properties file:
Attempt to schedule a new task as above, but uploading an invalid task properties file.
Such an invalid properties file may be created by removing the final “>” character from that used in test 2.3.5.5 in a text editor.
An error will be reported, indicating that the task properties file is invalid.
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 1 7
4.6
MC2.3.6 System Management - Control of Scheduled
Tasks
These sets of tests focus on the use of the Administration Interface to control (cancel, suspend and resume) scheduled tasks.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
All pending tasks are set to suspended.
4.6.1 Suspend a scheduled task
No. Action Expected result Pass Fail
MC2.3.6.1 Availability of ‘suspend task’ facility:
Log in to the Administration Interface and navigate to the ‘View Scheduled Tasks’ screen.
Schedule a repeating task with a ‘first due time’ in the recent past and a ‘repeat time’ of 1 minute. Use the workflow import external and task properties file downloaded from an existing task.
Locate this new task in the ‘View Scheduled Tasks’ screen.
Make a note of the task’s assigned Task ID:
The new task’s entry in the ‘View Scheduled Tasks’ display will show its status as ‘Pending’, and its ‘next due time’ as (approximately) the current time.
A facility will be available to suspend this task.
MC2.3.6.2 Suspend a scheduled task: Suspend the repeating task just scheduled.
WL | Delft Hydraulics 4 — 1 9
No. Action Expected result Pass Fail
displayed. On waiting for 3 minutes, no instances of this task will be dispatched in this time.
This can be checked using the task entry’s
“Task Runs” link: no new task runs will appear with a dispatched date in this time.
MC2.3.6.3 Logging a task’s suspension:
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 2 1
4.6.2 Resume a scheduled task
No. Action Expected result Pass Fail
MC2.3.6.4 Availability of ‘resume task’ facility:
Locate task entered in MC2.3.6.1 in the ‘View Scheduled Tasks’ screen.
A facility will be available to resume the task.
MC2.3.6.5 Resume a suspended task: From the ‘View Scheduled Tasks’ screen, resume the task just suspended.
The task’s entry in the ‘View Scheduled Tasks’ screen will have its status change to ‘Pending’, and its ‘next due time’ will appear as (approximately) the current time. Within the next 2 minutes, an instance of the task will be dispatched.
This can be checked using the task entry’s
“Task Runs” link: a new task run will appear with a dispatched date in this time.
MC2.3.6.6 Logging a task’s resumption: Navigate to the ‘View Logs’ screen.
4.6.3 Cancel a scheduled task
No. Action Expected result Pass Fail
MC2.3.6.7 Availability of ‘cancel task’ facility:
Locate task entered in MC2.3.6.1 in the ‘View Scheduled Tasks’ screen.
A facility will be available to cancel this task.
MC2.3.6.8 Cancel a scheduled task: Cancel the repeating task just located.
The task’s entry in the ‘View Scheduled Tasks’ screen will have its status change to ‘Finished’, and no ‘next due time’ will be displayed. No facility will be available to resume the task.
On waiting for 3 minutes, no instances of this task will be dispatched in this time.
This can be checked using the task entry’s
“Task Runs” link: no new task runs will appear with a dispatched date in this time.
MC2.3.6.9 Logging a task’s cancellation: Navigate to the ‘View Logs’ screen.
WL | Delft Hydraulics 4 — 2 3
Please make any addition comments on the test here
4.7
MC2.3.7 System Management - Configuration of
Workflows
These sets of tests focus on the use of the Administration Interface to view and configure Workflows.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
4.7.1 View Workflow Information
No. Action Expected result Pass Fail
MC2.3.7.1 View workflow information: Log onto the Administration Interface and navigate to the ‘View Workflows’ screen.
This can be found under
Workflows and FSSs -> Workflows in
the navigation bar.
WL | Delft Hydraulics 4 — 2 5
4.8
MC2.3.9 System Management - Control of currently
executing tasks
These sets of tests focus on the operation of the Master Controller to view and control tasks that are currently executing.
Please indicate whether the test has passed/failed by writing a comment in the pass/fail box. Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
All pending scheduled tasks set to suspended.
4.8.1 View currently executing tasks
No. Action Expected result Pass Fail
MC2.3.9.1 View currently executing tasks: Schedule three once-off tasks to run the import external workflow, each due now.
Navigate to the “Running Tasks View”.
This can be found under
Forecast Tasks -> Running Tasks in the
navigation bar.
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 2 7
4.8.2 Kill currently executing tasks
No. Action Expected result Pass Fail
MC2.3.9.2 Kill facility for currently executing task:
Schedule two once-off tasks to run import external, each due now. Navigate to the “Running Tasks View”.
This can be found under
Forecast Tasks -> Running Tasks in the
navigation bar.
An entry will appear in the ‘Running Tasks’ table for each of the two tasks just scheduled. An option will be available to “Kill” each running task.
MC2.3.9.3 Kill a currently executing task: Within 15 seconds of having scheduled the tasks, kill one of the task runs just dispatched and refresh the task runs view.
The entry will be removed from the ‘Running Tasks’ table.
MC2.3.9.4 Killed task terminated: Wait for another 15 seconds. Navigate to the “Scheduled Tasks” view and then to the ‘Task Runs’ view for the task whose run was just killed.
The ‘Running Tasks’ table for the task in question will contain a single task run entry. It will have a status of ‘terminated’.
MC2.3.9.5 Non-killed task not terminated: Navigate to the “Tasks View” and thence to the ‘Task Runs’ view for the task scheduled above whose run was not killed.
Please make any addition comments on the test here
WL | Delft Hydraulics 4 — 2 9
4.9
MC2.3.13 System Management – Administration
Interface User Management
These sets of tests focus on the management of user access to the Administration Interface. Please indicate whether the test has passed/failed by writing a comment in the pass/fail box.
Reference:
Test Record (please include name and signature)
Date of test: _____________________________________
RIZA representative: _____________________________________
Delft/Tessella representative _____________________________________
Initial Condition Administration Interface available.
Valid Administration Interface userid and password FEWS-NL available and operating
No. Action Expected result Pass Fail
MC2.3.13.1 Display of User Accounts: Go to the “User Administration” screen.
A table should be displayed indicating the user accounts that have been defined for the system.
MC2.3.13.2 Add a new user:
Select the “Add User” link. Create a new user account. Ensure that the “Password” and “Confirm Password” entries are identical.
Confirm that an entry in the System Log indicates that a user has been created.
The new user account is created.
MC2.3.13.3 Modify a user account:
In the “User Administration” screen, select the account just created and click “Modify”.
Change the username and password. Confirm that an entry in the System Log indicates that a user has been modified.
Confirm that the username is updated. Confirm that the password is updated (you will need to logout and then log back into the system).
MC2.3.13.4 Delete a user account: Create a new account.
Select the user account previously created.
Select “Delete”.
Confirm that an entry in the System Log indicates that a user has been deleted.
No. Action Expected result Pass Fail MC2.3.13.5 Attempt to delete the current
userid:
Create a new account.
Logout and logon with this new account.
Attempt to delete this account.
A warning message is displayed and you will not be able to delete the account.
MC2.3.13.6 Test Deleted
MC2.3.13.7 Attempt to add a user with an invalid (<3 characters) user ID Attempt to create a user account with the user ID “ab”.
Confirm that the system refuses to create this user.
WL | Delft Hydraulics 4 — 3 1
Please make any addition comments on the test here
