7    Artificial Neural Networks

7.1    Introduction

Artificial Neural Networks (ANNs) are able to approximate any function to any degree of accuracy given enough internal nodes (Sandhu and Finch, 1996).  ANNs run significantly faster than the functions they approximate with only a small loss of accuracy.  CALSIM has the capability of using ANNs (as well as the G-Model) to approximate Delta flow-salinity relationships.  The ANNs used are created from the Stuttgart Neural Network Simulator (SNNS).  More recently, ANNs have been used to generate salinity values at Martinez for planning and forecasting studies.

During the last year, numerous approaches were taken to improve the accuracy of the ANNs.  These approaches included:

  • The use of different node network structures;
  • Perturbing the values of the inputs to reduce correlation and increase the amount of available data; and
  • Refining the scripts that generate the ANNs to improve their stability.

7.2     Different Node Structures

The current network structure of the ANN consists of 72 input nodes.  Each of these nodes connect to a layer of eight internal hidden nodes that connect to a second layer of two internal hidden nodes and end with a single output node.

The 72 values entered into the input nodes come from 4 input sources, each of which have 18 input values.  The four input sources are the Delta Cross Channel position, amount of combined exports and diversions, Sacramento River inflow and San Joaquin River inflow. Sacramento inflow includes Sacramento River flow, Yolo Bypass flow, and the combined flow from the Mokelumne and Cosumnes Rivers (also known as the East Side Streams or ESS). The combined exports and diversions includes: the State Water Project (SWP), Central Valley Project (CVP), and North Bay Aqueduct exports, Contra Costa Water District diversions, and the net channel depletions.  These net channel depletions are calculated using the Delta Island Consumptive Use (DICU) model.  The 18 input values for each input source come from the 8 daily values followed by 10 weekly averages.  These values cover the previous 77 days worth of values and the current daily value.  These 78 days of values create a memory that allow the ANN to account for the influence of the previous state of the system and calculate the state of the system.

The two layers of internal hidden nodes allow for interactions between the input nodes.  By allowing the input nodes to interact, the normal interactions that would occur from the inputs of the system can be approximated.  The weights that get assigned during training reflect the amount of influence that different inputs have on each other.  The output node yields the final result that comes from the application of all of the weightings on the inputs.

One network that was experimented with increased the number of internal nodes in the first internal hidden layer to 16 and the second internal hidden layer to 4.  This configuration increased the time to train, but did not significantly increase the accuracy of the results.  The introduction of additional internal nodes allows for more flexibility of the network and as such allows for the ANN to approximate more spikes.  At a certain level of increased nodes, the network will lose integrity and become unresponsive due to not having enough data to properly condition the nodes.  By increasing the amount of data, the number of internal nodes that can be supported is increased.

Another network experiment involved splitting the first hidden layer into five hidden nodes.  Two of the nodes contained linkages to only the northern inputs: the Delta Cross Channel (DCC) position and Sacramento River flow.  Two more of the nodes contained linkages to only the southern inputs: the combined exports and diversions and San Joaquin River flow.  The last node contained linkages to all four sources.  This was tested because the sources within the northern and southern areas are more likely to have an impact on each other prior to interacting with the other area.  This configuration increased the level of complexity of the ANN structure without significantly increasing the accuracy of the results.

7.3    Perturbation of Inputs

In an effort to increase model sensitivity to the effects of the individual sources and increase the amount of available data to train the networks, the inputs to the ANN were perturbed.  Traditionally, while training the ANN, the effect of the DCC position is isolated by using its normal operation and then an inverted open/closed schedule.  This is done by running DSM2 for the normal operation and then running it again with the inverted operation and then using the inputs and outputs for both DSM2 runs for training the network.  The desired effect of this perturbation is to isolate the effects of the DCC from the Sacramento River and remove their correlation to each other.

The network is normally trained on 5840 daily values retrieved from a standard 16-year DSM2-QUAL planning study from October 1, 1975 through September 30, 1991.  The run is started at October 1, 1974, but the first year's worth of data (except for 77 days used for memory building) is discarded to account for the warm-up period that DSM2-QUAL requires.  Although QUAL’s warm-up period is only 3-4 months, the 1974 simulation data is not included in the training data set in order to keep the number of data points for each month uniform. Each perturbation increases the number of available training values by 5840.

In the same manner as with the DCC position, the remaining three input values were perturbed by 20 percent. Increasing and decreasing the values of each individual input while keeping the remaining inputs constant and using the normal DCC operation yielded an additional six 16-year DSM2 runs worth of values.  The values for these six runs were added to the normal two done for the DCC for a total of eight runs worth of data.  In addition to creating additional data for training, this data better reflected the influences of the individual inputs.  The increase or decrease of the value for each input removed some of the correlation otherwise present in the training data when the inputs were not treated independently.

The additional data allowed for an increase in the number of internal nodes, from 8 to 32, without collapsing the integrity of the network.  The increased number of nodes allowed the network to have more flexibility and increased the number of peaks that could be approximated.

7.4    Example of Using the ANN to Forecast Martinez Salinity

DSM2-QUAL requires that water quality conditions be specified at the downstream boundary located at Martinez (RSAC054).  ANNs have been used by the Section over the past few years to estimate Martinez EC for DSM2 planning studies.  This year a new ANN was developed to estimate Martinez EC for DSM2 forecasting applications.

Since this tool was designed for forecasting rather than planning applications, the training inputs were limited to data that would be readily available for forecasts.  The ANN was trained on the following daily average historical data measured from May 19, 1983 through December 10, 1999: Martinez EC, Sacramento River flow at Freeport, San Joaquin River flow at Vernalis, and combined SWP and CVP exports. Smaller Delta inflows such as East Side Stream and Yolo Bypass are assumed to be small constant values in DSM2 forecasting applications; therefore, they were not included in ANN training.  Delta Cross Channel operation was not included in ANN training as it has no measurable impact on Martinez salinity.

The training period was selected based on data availability.  Missing values were filled in using a simple linear fit. The first three-quarters of the historical data were used for calibration and the last quarter was used for validation.

The trained ANN was first used in several DSM2-QUAL simulations of historical Delta conditions during November and December 1999.  In these post-cast simulations, historical SWP and CVP exports were altered in order to study the impacts of project operations on Delta water quality. (Five different export scenarios were evaluated.)  Export modifications have a direct influence on net Delta outflow and influence salinity at Martinez – thus requiring estimates from the ANN.  The trained ANN was applied in several DSM2-QUAL forecast applications conducted in January and February 2000.

The use of the ANN to forecast EC at Martinez has been replaced by a new boundary salinity forecasting process. This process is described in Chapter 11.

7.5    Conglomeration of Scripts

The scripts used to generate an Artificial Neural Network were written in a combination of Python, Awk and C-shell scripts.  Recently all of the scripts were moved to a single scripting language (Python) and incorporated as a part of the new VPlotter program.  Through this change, the generation of ANNs is now a more stable and repeatable process that is easier to perform.

7.6    References

Sandhu, Nicky and R. Finch. (1996). "Application of Artificial Neural Networks to the Sacramento-San Joaquin Delta." Estuarine and Coastal Modeling, Proceedings of the 4th International Conference.


Author: Tawnly Pranger
Back to Delta Modeling Section 2000 Annual Report Table of Contents

Last revised: 2000-10-23

About Us Hydrology & Operations Section Delta Modeling Section Computer Assistance Section
Home Page Available Studies Computer Models Reports Home Page Available Studies Computer Models Reports

Disclaimer, Comments or Questions
Webmaster email to htdelmod@water.ca.gov