Contents

1  Introduction

There are few common problems that one will be faced during process of establishing of experimental catchment in urban drainage. Choosing the right site for catchment is the essential, and it must take into consideration following tips:

1. Well defined catchment boundaries. In order to have a representative set of data, we must be sure that every rain drop is taken into account and that there is neither loss nor water that is drained towards some other catchment.
2. Land use of catchment is constant over time, or at least its temporal distribution is known.
3. Rain intensity and sewage outflow can be measured easily. Also, it is advisable to measure meteorological data (wind speed, air temperature, sunshine duration and intensity, air humidity etc.) together with soil humidity and evaporation. The extensive set of data will be needed by the next generation of software models.
4. The direct link between experimental site and laboratory will result in a safety of the measuring equipment together with much easier management of the data acquisition for the whole catchment.

In this chapter, only last two items will be discussed. It will be assumed that experimental catchment is known, measuring quantities are chosen and measuring sites are established. The question is now what is the most convenient way of data gathering, storing and retrieving for later processing. Thus, once again, it is assumed that:

• measuring quantity is converted to voltage (or current) by means of suitable transducer (it is a real art to chose an adequate measuring technique, optimal measurement range and other characteristics in order to obtain representative data),
• calibration function for the transducer used is known, ie. that analytical relationship between measured quantity and output signal is found (one must not forget the hydraulic background of measuring site, whether all considerations are fulfilled in order to permit the usage of the calibration curves) ,
• dynamic behaviour of measuring ensemble is analysed and described by dynamic transfer function (specially having in mind dynamic characteristics of the tipping bucket rain gauge and stilling basins for critical depth flumes).

• the possible ways of data acquisition, mainly concerning hardware, different characteristics that user must consider together with strong and week points of some schemes:

  1. strip chart recorders, rather old and clumsy way of data recording on paper, but still in use,
  2. analog data recorders, with magnetic tapes,
  3. digital data loggers, as state of the art technique,
• software tools, compilers and commercial packages for data acquisition, preprocessing and data base creation.

2  Data Acquisition Hardware

2.1  Strip Chart Recorders

It is an rather old system, widely used in past and relatively inexpensive. Special mechanism is used to move paper, with constant velocity, while another mechanism moves pen proportionally to measured quantity. The result is the line (or several lines for multichanel instruments) that plots the changes of the measured quantity through the time either in polar or rectangular coordinate system. If instead of pen, very tiny light sheet is used together with optical paper (normally, UV light is used) the response time of instrument can be much faster (those instruments are normally used for transient measurements). There is also possibility to use thermal paper together with special pens with heating tips.

General characteristics of strip chart recorders are:

• analog signal should be current, mostly 0-20 mA for self supplying sensors and 4-20 mA for current driven sensors,
• number of input channels varies between 1 up to 6,
• each input channel can have its separate ground, independent of the ground of recorder itself, meaning that there is no risk of ground loops,
• system is not sensitive to electromagnetic induced noise and can stand high energy spikes (this is valid only for mechanical recorders, UV and thermal are more sensitive),
• paper advance system is built around linear or stepper motor, or mechanical clock mechanism that can be completely mains voltage free,
• speed of paper is mechanically controlled by means of gear box,
• accuracy of overall system is few percents, and is directly proportional to price of the instrument.

Usage of strip chart recorders, of any kind, is connected with several common problems:

• general maintenance problem: cleaning of moving parts, regular pen inspection, paper jam due to different kinds of papers, out of paper condition, aging of paper (specially for UV and thermal recorders), aging of pens, problem with air humidity,
• fixed time base: to obtain good time resolution, very high paper speeds of 5 mm/min or similar must be used, (leading to 7.2 m/day); problem can be easily solved by introduction of the switch that will switch off the recorder during periods that are not of interest, but with the price of loosing the time synchronization (to keep a track of real time, second recorder can be used with much lower paper speed and with continuous operation, see figure 1),
• data processing, digitization and paper handling is not as easy task as it may look, specially for large amount of data; process of digitization can be very expensive, it can introduce new source of errors and special algorithms for error detection is then needed.

2.2  Analog Data Recorders

Storing of data using analog tape data recorders (see figure 2) is adequate for large number of channels, when high bandpass frequency is needed for relatively long periods, ie. measurements of velocity fluctuations, with 500Hz bandpass and for several minutes. Data storing medium is magnetic tape, either standard audio or video cassettes. To store analog voltage on tape, two basic principles are used to convert DC (direct current) signals to AC (alternate current): either by modulating high frequency carrier signal (the frequency of carrier must be in the order of magnitude higher than the maximum frequency of the input signal) or by converting the input signal to the trail of pulses whose width is proportional to the input voltage (pulse width modulation) and converting that pulse width information to time code. Data retrieval is in fact restoration of original analog voltages and its measurements by voltmeter or oscilloscope.

Analog data recorder are very popular in laboratory measurement rigs for investigation of transient phenomena and for vibration problems. For urban experimental catchments they are not widely used, mostly because of their unsuitability for field environmental conditions. Also, relatively high price cannot be neglected.

2.3  Digital Data Loggers

Some measuring quantities (discharge for example) are analog by their nature, meaning that the transition from one level to another is done through the infinite number of steps. Converting that analog signal to its digital representation (number), estimating the value of analog signal in one instant of time, with the predefined and finite precision (finite number of decimal places, or finite number of steps that analog signal can be equaled with), means that one can feed that number to digital computer, process it and store it for later use. Benefits from digital approach are:

• data degradation is kept at minimum, after converting analog signal to digital form, noise, additional losses and other influences are kept away from data,
• virtually unlimited number of input channels, sampled simultaneously or with different sampling rates for each channel, analog or digital channels, counter inputs etc,
• unlimited data preprocessing capabilities, after converting analog voltage to digital form, any kind of calculation can be performed (but there is no way to restore lost information during analog to digital conversion),
• complex data acquisition logic can be used, leading to some kind of intelligent sensors, with possibility to reduce amount of stored data drastically,
• networking and remote operation, connection of several loggers, control of logger operation through the Public Switched Telephone system,
• storage capacity can be high, especially if data reduction techniques are used (data compressions, run time length compression etc.),
• for on-line analysis, acquired data can be directly fed to numerical simulation models, thus very important for real time control purposes.

Schematic diagram of the digital data logger is given on the figure 3. On the next figure (fig. 4), simplified scheme of the commercially available data logger is given, named DT500, from Data Electronics (Australia) Pty. Ltd. just as an example. In order to make things easier, data logger can be separated into four main parts:

1. analog part of logger, with input filters, multiplexer and programmable gain amplifier,
2. analog to digital converter,
3. digital part of logger, with microprocessor that controls the operation of whole logger, I/O devices and storage devices,
4. power supply unit.

2.3.1  Analog Filter

Analog input filter is very important part of digital data logger. Unfortunately it is often neglected from both sides, from manufacturers of measuring equipment and data loggers. End user must know the details of the equipment used, the site locations and measuring conditions, as well as input characteristics of used data logger. Analog filters and accompanying circuits can be quite expensive, so detailed analysis must be performed.

• Surge protector (fig. 5 - A) - in field measurements, large voltage spikes can be induced in wires, due to atmospheric inductions. Input impedance of data logger is very often in the order of megaohms, so input potentials of hundred volts can be easily obtained. To prevent the burnout of the input circuits, suitable diode or varistor protection is needed.
• Low pass filter - it must be calculated according to frequency characteristic of the measured signal and the sampling speed of the data logger. Sampling frequency must be higher then maximal possible frequency that can occur in input signal, to avoid aliasing problems (fig. 5 - B). Once, analog signal have been converted to digital, there is no way to distinguish the true signal from false one. Also, according to Niquist theorem, for analysing purposes sampling frequency must be at least two times greater than the frequency that is of interest.
• Differential input - to reduce the level of noise, true differential input must be used. Since unwanted signal is in phase on both inputs, it will be attenuated by the Common Mode Rejection Ratio (CMRR) (fig. 5 - C). To reduce further level of noise, shielded input wires should be used, with shield connected to common ground point. For data loggers that are not equipped with differential inputs, twisting of input lines can significantly reduce input noise.
• Optical isolator (fig. 5 - D)- if common measurement ground is source of noise, due to ground potential difference, optically isolated inputs must be used. Ground difference can be induced by long input wires, when several measuring equipments are connected and each equipment have its own power supply unit.
• Long input lines - when input lines exceeds certain limits, analog modulator/demodulator can be used. Modulator will convert DC voltage to modulated carrier signal of specific frequency (fig. 5 - E). One line (2 wires or radio link) can be used to send more than one analog quantity if different carrier frequencies are used.

2.3.2  Analog Multiplexer

It has been said earlier that number of input channels for digital data loggers can be very high (more than 256). Generally, there are three approaches of analog data measurement using computer:

1. to use one analog to digital (A/D) converter (very expensive part of the logger) and input multiplexor (MUX) that will scan through the input channels; cheapest solution, but the problem is that measurements on two successive analog channels is performed at different time intervals (points f₁ and f₂¹ on the figure 6),
2. to use as many A/D converters as analog input channels to obtain true simultaneous measurement, all input channels are sampled at the same time (points f₁ and f₂ on the figure 6); expensive solution but it is used for special high speed applications where number of input channels is small (for example in frame grabber systems where real time conversion of video signal is performed, three A/D converters are used, one for red signal component, another for green and third for blue one).
3. the same as the first approach, but with analog memory (Sample and Hold) for each channel; analog voltages on all input channels are stored in analog memory (each channel must have its own memory, a kind of capacitor with very high impedance buffer) at same instant of time and than input multiplexer scans through analog memory (points f₁ and f₂² on the figure 6) of all channels.

For the first and third method, the analog multiplexer will switch through the list of the predefined channels. Important parameters of multiplexer are:

• the time that elapses between two switchings (Dt on figure 6) is sum of time that multiplexer needs to switch from one channel to another and time that analog to digital converter needs to convert analog voltage on the first channel to its digital binary representation; if analog input of logger does not have sample and hold circuit, Dt must be taken into account for fast acquisition rates,
• settling time, ie. time that analog part of multiplexer needs to settle down after each switching; settling time is incorporated in Dt, and is one order of magnitude smaller than conversion time,
• crosstalk between channels; this is a percentage of voltage that can be copied from one channel to another, expressed in decibels.

2.3.3  Input Programmable Gain Amplifier

To gain the maximum accuracy of the logger, input signal should be within the optimal range of analog to digital converter. Using Programmable Gain Amplifier (PGA), input signal can be amplified, each channel with its own gain. If control software allows, the autoranging facility can be used, where logger will change the gain factor to maintain the highest accuracy. This possibility is very interesting in low speed applications (sampling rates from [ 1/10] seconds and slower), where data logger has enough time to find, by trial and error the optimal measuring range. Some important parameters of PGA are:

• gain setting, usually in decade steps (×1, ×10, ×100,×1000),
• input impedance of amplifier (1 to 10MW),
• settling time (few nanoseconds, and it is generally larger for higher gain settings),
• Common Mode Rejection Ratio (CMRR) for differential PGA, ie. the amount of attenuation of hume and noise, that is in phase on PGA differential input (about 60dB for low-end PGA and more than 80dB for high-end amplifiers),
• offset drift shows how much output voltage of PGA changes because of aging, thermal influences etc, when input voltage is constant; this drift is important for extremely low voltage inputs,
• noise figure, or percentage of noise introduced by the PGA itself,
• frequency characteristics of PGA for different gain factors.

2.3.4  Analog to Digital Converter

Analog to Digital (A/D) converter is the heart of data logger. It performs the convertion of continuous analog signal to discrete samples (readings), and then provides binary number presentation for that step (quantification). On figure 7 a continuously varying input voltage is shown, presenting an analog input voltage, together with a grid of possible discrete steps that will be used during process of quantification. Obviously, A/D converter can not distinguish between all voltage levels that fall down between two grid steps. Three following parameters are the most important for A/D converters, their precision, speed and price.

Precision of analog to digital data conversion depends on number of used bits for voltage discretization (as shown on figure 7 each bit corresponds to one grid spacing). For 8 bit A/D convertor, input voltage will have one of 256 possible values between minimum and maximum input voltage; ie. for 10 V input span (±5 V), the result of digitization will be within 10/2⁸=0.04V=40mV (this is a voltage level of one grid spacing). For 12 bit conversion there is 4096 possible values 10/2¹2=0.0024V=2.4mV, and for 16 bit conversion 65536 10/2¹⁶=0.00015V=150mV. Apparently, the larger number of used bits for A/D conversion, the higher accuracy of conversion is achieved.

All conversion methods used in A/D converters can be separated into the three groups. Each method has its own field of usage, so it is hard to speak about the ``best one'':

• flash converters - the fastest converters and with lowest accuracy; input voltage is fed to an array of 2ⁿ of comparators, and the binary output is just the result of comparison, all job is done in just one cycle; number of used bits is from 6 to 8,
• successive approximation converters - moderate speed and accuracy; input voltage is compared with one comparator with another voltage, generated by the computer through the digital to analog (D/A) converter; number of used bits id from 8 to 16,
• dual slope/ramp conversion - slow speed and the highest accuracy; high quality capacitor is charged by the input voltage, and then discharged by the constant current drain, using separate counters with constant time base; it measures the time needed by capacitor to empty down to 0 volts; to avoid some bias errors, same procedure is done once again but with reversed polarity; number of bits can be very high, up to 20, but generally linearity errors are much higher than conversion errors (for 18 bit converter, conversion error is 0.0004% and linearity error can be 0.04%, ie. 100 times larger).

Speed of A/D converter is vital for some applications. Generally, higher speed of conversion means the higher price and the lower resolution. Those converters are used for video digitization purposes. Successive approximation converters are ideal for general purpose transient measurements, audio measurements etc. For the present application, in the field of urban drainage measurements, the cheap, slow and accurate method of dual slope conversion is the best solution. One must take into account is that the number of accurate bits is not always the same as the number of used bits. For low and medium quality devices, linearity error introduced by the discharging capacitor can be much higher than conversion error.

2.3.5  Microprocessor and Operating System

' Program for DT500 ver. 1.00
' Data acquisition on Miljakovac catchment
' of rain intensity (1 rain gauge) and sewage outflow (2 level meters)
RESET                   ' First reset data logger
P39=0                   ' Set some parameters
P31=1
P22=44
D=03/10/1991            ' Set date and time
T=08:16:00
/u /n /X /e /q          ' Set some control bits, format of data etc.
' Next program line is the main job definition, 
' First is S (single step) job that will record date and time
' whenever is activated with X command
' second is A job repeated every 30 seconds -- 
' read the state of the rain counter and reset it to zero (ie. store
' the number of pulses in the 30 second interval), and read current
' input on two channels, representing levels in a discharge measuring
' critical depth flume. Immediately after storing two job definitions, 
' halt job A (HA command). Job A will be started through the ALARM1
S D T  RA30S 1C(R) 1#I 2#I  HA
' System of alarms control the start and stop of data acquisition. 
' ALARM1 is monitoring if there is any impulse on first counter
' (1C>0), and if there is any, give a time stamp of the beginning 
' of the rain, start job A, halt the operation of ALARM1 and start the
' operation of ALARM2. ALARM2 is then used to monitor the end of rain
' condition - if there is no impulses on the rain counter for more than
' one hour, give a time stamp of the end of the event, halt A job, switch
' off the ALARM2 and start again ALARM1.
ALARM1(1C>0)"[X GA OFF1 ON2]"
ALARM2(1C<1/1H)"[X HA OFF2 ON1]"
OFF2                    ' Turn off the second alarm
ON1                     ' Turn on the first alarm
LOGON                   ' Start logging of measured data to memory
X                       ' Perform once single step job, record a time of
                        ' start of logger operation.

Microprocessor (mP) and Operating System (OS) are the main part of the whole data logger. Through the OS, the mP controls the operation and defines the characteristics of the system. The power of mP is not of vital importance, very good loggers can be found even nowdays built around the old 8 bit Z80 chip. Real Time Clock (RTC) must be the integral part of data logger, with compulsory separate battery backup facility. Control software, Operating System (OS) can be completely open, allowing easy programing of the logger with user defined scanning intervals, select different gain factors for various channels, use whatever kind of interpolating polynom for conversion from voltages to user SI system, or it can be closed, disabling any possibility to change the preprogramed jobs. Multitasking OS is welcome, because it simplifies programing of complex data acquisition jobs, different schedules for predefined channel lists etc, together with possibility to perform real time data analysis, storage and control.

For data loggers on experimental catchments, open system is preferred, because it leaves a degree of freedom to suit the control program to specific needs. Programing language can be either internal language (such as shown on the figure 8 as a sample of control program used on Miljakovac catchment on DT500 data logger from Data Electronics Pty. Ltd or normal basic interpreter. Data security is of vital importance - it is advisable to have a data logger with watch dog circuit, a piece of electronics that is used to monitor the operation of mP (if mP is idle for certain time, watch dog will reset it and store time and date for latter analysis) with possibility to give a warning signal and to start the preprogramed reset procedure - vital for field loggers.

Power Supply Unit (PSU) must be considered during establishment of data acquisition system. If all sensors and accompanying equipment is powered from mains power, data logger must have its separate battery backup. Possible intervals of battery operation must be calculated taking into account the worst possible case. Some data loggers have built in sleep mode, ie. during long intervals of disabled acquisition, much of the logger is turned off, only mP and awake circuits are on (lowering the power consumption from hundreds of miliamps to few microamps). In the field of urban drainage, control software can be written in such a way, that rain gauge signal is used as a trigger signal for awaking of data logger. Data logger will be placed back to sleep mode, if there is no signal from rain gauge certain predefined period of time (depending on the area of the catchment).

Analysing the basic characteristics of the mP and OS for field data loggers, it is clearly seen that commonly used personal computers (PC) equipped with data acquisition boards are not well suited for long term operations. PC configuration is perfect for laboratory measurement rigs, where data acquisition is performed in controlled environmental conditions. Absence of watch dog circuit, high power ratings and low level of resistance on the electro magnetically induced (EMI) noise are characteristics of general purpose PC that makes it unacceptable for long term field use. Special versions of field PC are available, even on EURO card format, without hard disk etc, but the price of those computers are much higher than small data loggers. The role of PC is in reading and analysing of data collected by loggers, and in that job small notebook PCs and portables are adequate.

2.3.6  Data Storage

Digitized analog input voltage is optionally preprocessed (filtered out, scaled, combined by other channel readings) and then stored for latter retrieval and analysis. There are several ways of data storage that can be used in digital data loggers:

• on magnetic media - tapes, floppy and hard disks; it will use high power consumption, it is reliable in holding data but it should have good environmental conditions, very large amount of data can be stored,
• on erasable programmable read only memory (EPROM) - there are two categories of EPROMs, UV (that can be erased by strong ultraviolet light) and E (electrically erasable memory); low power consumption is one of the major characteristics of EPROM and possibility not to lose data if disconnected from power supply for long period of time, problem is that EEPROM is very expensive for massive memory, on the other side UVEPROM is cheap but slow for programming,
• on random access memory (RAM) - static or dynamic; relatively cheap solution, specially with dynamic memory, moderate power consumption, allow high speed and high volume of data storage, static memory can be backuped by small battery for years.

There are four parameters that are important for choice of data storage for logger:

• the way in which data logger is powered, battery power will result in low power consumption solutions (UVEPROM or static RAM) while loggers powered from main supply can use magnetic media or dynamic RAM chips,
• environmental conditions; large humidity and vibrations will discard magnetic media, electro - magnetically induced noise (EMI) is dangerous for RAM chips etc,
• storage capacity is the most important factor; user must calculate the number of maximum possible data as number of channels × sampling rate × period between two data retrieval,
• and on the end price per storage unity must not be neglected.

2.3.7  Data Logger Interface Circuits

For data retrieval, control and programming of digital data logger, a number of interfaces circuits is available:

Display

- for visual inspection of logger, graphical presentation of results etc. Catod ray tube (CRT) is not suitable for permanent use especially in crude environments, liquid crystal display (LCD) is much preferred. For most purposes just few light emitting diodes (LED) is sufficient for brief inspection of data logger's state.

Keyboard

- most field data loggers doesn't have it, because the operation of logger can be controlled by other interfaces, and it is not a good habit to play with its operation while it is on the measuring site. For field instruments mechanical switch keyboard is not adequate, membrane switches are much better.

Modem

- allows remote inspection of measuring rig through telephone line. Most modern loggers can be reprogrammed and can unload stored data through modem. For large amount of data, unloading through the modem is not an adequate means, because of limited speed. Password protection is needed for public telephone system together with protection not to block out a logger if telephone lines are broken during some critical operations. The latest loggers can use normal voice operation telephone lines, and report its status with synthetic voice.

Serial and Parallel input/output (I/O)

- for connection of logger and control computer, several loggers in local network, logger and printer for hardcopy of unloaded data etc. Parallel connections are much faster then serial, but are limited to shorter distances. Local logger network is very interesting option, where distributed and supervised operation of several data loggers is possible (one logger is master others are slave loggers, or all loggers are independent and accessible through one connection etc.) keeping analog paths as short as possible.

3  Data Acquisition Software Tools for PC

Software is integral part of data acquisition system. User can distinguish several levels of software usage through the process of data acquisition and data storage in urban drainage experimental catchments:

• for control and/or programming of measuring equipment; if digital data loggers are used, control program must be written using internal commands from OS (like one shown on figure 8),
• for data retrieval from measuring equipment (digital, analog tape or strip chart) through the process of reading data or digitizing,
• for data preprocessing (conversion from internal codes to SI system using interpolating functions or lookup tables, filtering of data, reduction of data etc.),
• for data processing and data base management (tests on validity and quality of input data through numerical and statistical filters, mathematical models or simple visual inspection, data base creation and storage, link to other national or international data bases through conversion programs and world-wide computer links etc.),
• for interfacing with different hydrologic and hydraulic models (interface programs to prepare input data files needed by specific model).

All noted steps in data acquisition and processing can be done by:

• writing own programs with different compilers - BASIC (very powerful, with special data acquisition versions, new compilers are much alike to FORTRAN with old BASIC flexibility), FORTRAN (limited I/O capabilities, popular language for data analysis, it can be used for reading data from digital loggers or from digitizers, but not as easy as it can be done by other languages), C (unlimited capabilities, possibility for low level programming, all PC resources are accessible), ASYST (powerful, with high and low level routines, can be used to access data loggers directly),
• commercial packages - LABTECH (data acquisition and control, limited processing capabilities, excellent for data acquisition using PC in laboratory environments), DADiSP WORKSHEET (scientific spreadsheet for post acquisition signal analysis, time and frequency plots etc.), MATHLAB (powerful data analysis interpreter), LOTUS, QUATROpro etc. (spreadsheets for data storage and interpretation, powerful in graphics presentation, can not be easily used to interface with data loggers or digitizers),
• data base management - dBase, FoxBase, MAGIC, SQL based systems (for data storage and interfacing with existing factual data base), link between numerical and graphical files is possible.

4  Conclusion

The aim of this chapter was to introduce the possible data acquisition techniques that are available today, with strong emphasize on computer controlled data loggers. Technical details of all parts of logger are explained with simplified schemes, having in mind that the average reader of this chapter will be a civil engineering and thus keeping away boring things as much as possible.

In order to make system work correctly, modern engineer must know all good and bad points of chosen hardware. Since there is no ideal data acquisition system, and that we must live with real world problems, the best solution is to learn all relevant hints about measuring sites and quantities on the experimental catchment. It is necessary to make an error analysis and to purchase the equipment (sensors and amplifiers) that will satisfy our needs (range of sensors, accuracy, temperature sensitivity, response time etc.). The same analysis must cover the used data logger together with interface circuits between sensors and logger. User must consider the basic accuracy of analog to digital converter, the amount of memory as a function of used data acquisition protocol, environmental conditions, source of power etc. The choice of software for data collection, data base formation and analysis, depends on personal taste. Whether it will be a commercial package or written by a user, it must provide the full data security, together with back-up facility. It is amazing how easy is to delete the results of few years work with just a few commands.

Good choice of equipment will result with the reliable data acquisition system, with possibility to transfer easily data to the applied data bases and numerical models. The worst thing that might happen to one is to have a bucket of useless data in a computer, trusting that everything is OK. Using modern techniques, computers and data loggers, can give an impression that used system is error-free. Calibration of numerical model based on false measurements will lead to enormous errors, and can produce really false conclusions in urban drainage.