Implementation of Monte Carlo Simulation in Evaluation of The Uncertainty of Rainfall Measurement

Many factors trigger the uncertainty of rainfall measurement. Several factors can be related to the instruments, weather conditions, and acquisition methods. The degree of uncertainty could be obtained through the calibration process. In principle, rain gauges are calibrated based on the standard process ruled by ISO/IEC 17025 using the law of propagation of uncertainty (LPU). However, LPU requires complex and complicated mathematical calculations. An alternative approach is needed to evaluate measurement uncertainty besides the LPU method. This research used the Monte Carlo method to determine the uncertainty during the rainfall measurement. This method involves repeated random simulations by providing probability distribution on the input and output of rainfall measurement. The results showed that the Monte Carlo method can accurately determine the uncertainty of rainfall measurement. In addition, the uncertainty analysis also showed that instrument inaccuracy is the most significant factor that causes the uncertainty of rainfall measurement.


INTRODUCTION
Metrology is the science that encompasses all theoretical and practical concepts of measurement. It includes how to do the measurements, equipment, uncertainty evaluation, and measurement standards development (JCGM 200, 2012). When applied, it can produce acceptable accuracy and metrological reliability in the measurement process. Evaluating and analysing the metrological concepts is crucial in any field where measurement results are used to make decisions. For instance, when providing weather information services such as floods in a particular area, the intensity of rainfall in that region is a crucial component that affects flood information. An instrument (rain gauge) must be calibrated to measure the rainfall intensity to provide traceability and metrological reliability measurements. Therefore, the concept of metrology and measurement reliability is crucial in establishing the uncertainty of the rain gauge measurement. The rain gauge must be established, verified, and validated to ensure the quality and reliability of weather information. However, one of the challenges in evaluating and estimating measurement uncertainty is the requirement for mathematical and statistical competencies. Despite this challenge, international standards like ISO/IEC 17025 have established requirements for calibrating the measurement process.
ISO/IEC GUM (Guide to Expressing Uncertainty in Measurement) was first published in 1993 by seven international organizations: BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, and OIML. This guide guides the estimation of uncertainty in measurement. The uncertainty estimation, as presented by GUM (JCGM 100: 2008), is based on the Law of Propagation of Uncertainty (LPU) and the Central Limit Theorem (CLT). Most calibration and testing laboratories have successfully applied this methodology to evaluate measurement uncertainty for various measurement processes worldwide for several years. GUM uses the LPU approach to take estimates (values) of input quantities and their associated standard uncertainties to obtain estimates of the output quantity and its associated standard uncertainty.
The measurement model is used to calculate (1) the value of the output quantity and (2) the sensitivity coefficients, which are the first partial derivatives of the output quantity concerning each input quantity (evaluated at the estimate of the input quantity) (Montgomery, at al, 2018). The second part of the calculation involves partial derivatives calculated through analytical differentiation, which is often impractical and requires skills that laboratory staff and researchers may not have (Adriaan, at al, 2021). The second approach GUM uses is the CLT approach, which states that the combination of several original distributions, whatever their distribution form, is assumed to form a new distribution that approximates a normal distribution. However, applying the GUM method is not free from criticism, as assumptions that determine the validity of the LPU and CLT approaches used by GUM must be met to produce estimates of measurement uncertainty that are close to their true values. The assumptions that must be met include a linear measurement model, a symmetrical distribution, and whether there is correlation between the input quantities (Paulo Roberto Guimarães Couto et al., 2013). If these assumptions are not met, it could potentially result in biased uncertainty estimates. This is the limitation of the GUM method, where constraints must be met first. Due to these limitations, in 2008, ISO/IEC issued Supplement 1 GUM (JCGM 101: 2008), providing general guidance on distribution propagation using the Monte Carlo Method (MCS) to evaluate measurement uncertainty.
Although Supplement 1 GUM has been published for a long time, the adoption of the MCS by calibration laboratories in Indonesia still needs to be improved. One advantage of the Monte Carlo method is that sensitivity coefficients are not required. Only the measurement model, which can be an algorithm and probability distribution specifications for the input quantities are required. These probability distributions (normal, rectangular, etc.) are usually already specified in the uncertainty budget when using the LPU-GUM. Another advantage of the MCS is that it can numerically evaluate partial derivatives, so they do not need to be derived analytically as in the LPU-GUM method. Additionally, the combination of several original distributions, whatever their distribution form, is not necessarily assumed to approximate a normal distribution. The basic principle of the Monte Carlo method is to generate random numbers based on a specific probability distribution (PDF) (Ian Farrance, at al, 2014) to solve mathematical problems whose analytical solutions are unknown or difficult to obtain. Currently, many programming languages can generate these random numbers, including the Python programming language. Therefore, MCS can be used as an alternative method for determining measurement uncertainty and verifying the LPU-GUM method's validity or replacing the LPU-GUM method in cases where the LPU-GUM method is perceived to have failed (Walpole, et al, 2011).

METHOD A conventional approach: Guide to the expression of uncertainty in measurement (GUM)/ Law of propagation of uncertainties (LPU)
The GUM, provides guidance on estimating the uncertainty in measurements that requires a deep understanding of the measured quantity and its measurement process. The output (y) is a function of inputs (x) (JCGM 100: 2008), (Equation (1)), = f( 1, 2, …….. n) (1) The uncertainty is calculated by combining uncertainties from sources of input quantities evaluated based on their distribution types. The output Y depends on input quantities (X1, X2, ..., Xn) and has a functional relationship with them. Therefore, the final uncertainty of the measurement result depends on the inputs. Here, the relative uncertainty of each input is calculated. This treatment of the model, where the measured quantity is expressed as a function of input variables 1, . . . , (Equation (1)), leads to the general expression for uncertainty propagation (JCGM 100: 2008), (Equation (2)).
The result given by Equation (2) corresponds to the interval that contains only one standard deviation (or about 68.2% of the measurements). To have better confidence in the result, the GUM approach widens this interval by assuming a t-Student distribution on the measurand. The effective degrees of freedom eff for the t-distribution can be estimated using the Welch-Satterthwaite formula (JCGM 100: 2008), (Equation (3)).
where vxi is the degrees of freedom for input quantity i. The expanded uncertainty is then evaluated by multiplying the combined standard uncertainty by a coverage factor that expands it to the coverage interval limited by the t-distribution with the chosen confidence level (JCGM 100: 2008), (Equation (4)).
= (4) When combining multiple source distributions, whatever their shape, the resulting new distribution is assumed to be close to a normal distribution. Figure 1 shows various steps to calculate uncertainty of measurement using LPU method  An alternative approach : Monte Carlo simulation The Monte Carlo simulation method can be used as an alternative approach to evaluate measurement uncertainty. This approach uses statistical techniques to experimentally validate some theoretical results for uncertainty evaluation ( Harshvardhan , at al, 2021).
By providing an appropriate probability distribution function on input quantities and using a model to provide the final output distribution, the output distribution is no longer assumed to be normally distributed as in the GUM method. In this method, random values are generated using algorithms and follow a predetermined distribution. For all inputs, numerical values are generated based on their respective distribution types. Furthermore, these values are generated based on the functional relationship provided so that one numerical output value is produced. The entire process is repeated N times to obtain a set of simulation results (Rubinstein, 2016). This can be referred to as the procedure used for PDF propagation. Figure  2 shows various steps to calculate uncertainty of measurement using MCS method

Calibration Process of Rain Gauge
The rain gauge is calibrated according to the calibration procedures (WMO No.8 2018). A measuring instrument is used to measure the diameter of the rain gauge funnel which is then used to determine its area. The rain gauge measuring cylinder is also used to determine the volume of rainfall. In general, the measurement uncertainty is calculated by dividing each contributing uncertainty value by the corresponding factor. All sources of uncertainty and probability distributions are shown in Tables 1 and 2. With different relative uncertainty contributions that are achieved from all above mentioned sources of uncertainty (Tables 1  and 2) and the uncertainty of the rain gauge measurement, the measurement uncertainty of the rain gauge is computed as follows.

RESULTS AND DISCUSSION
An evaluation of measurement uncertainty of a rain gauge has been conducted using LPU-GUM and Monte Carlo methods for 2 different cases: using a certified dial caliper and using an uncertified ruler, with measurement data sources and related PDFs as shown in table 1 and 2.

Evaluation of uncertainty of measurement using LPU-GUM method
The results of the uncertainty evaluation of the rain gauge measurements for both cases using the LPU-GUM approach can be seen in Table 3 and 4.  Table 4. Case 2 -The results obtained for the rainfall model using the GUM uncertainty approach and measuring the diameter of the rain gauge funnel using an uncertified ruler, with a 95% coverage probability.

Evaluation of uncertainty of measurement using MCS method
The MCS method has also been applied to evaluate the uncertainty of measurement of a rain gauge with input sources and related PDFs as shown in tables 1 and 2. Random numbers were generated using Python programming language for N=10 6 iterations for each input source according to the specified PDFs. The estimation of measurement uncertainty using the MCS method for the rain gauge for both cases is presented in tables 5 and 6. 20.57 mm The combined final distribution for both cases has also been obtained as shown in figures 3 and 4. Figure 3a. Case 1 -The histogram represents the PDF generated for rainfall measurement using Monte Carlo simulation (using a calibrated dial caliper).
| 514 Figure 3b. Case 1 -The low endpoint and high endpoint graphs for a coverage factor of 95% represent the PDF generated for rainfall measurement using Monte Carlo simulation (using a calibrated dial caliper).

Figure 4a
. Case 2 -The histogram represents the PDF generated for rainfall measurement using Monte Carlo simulation (using an uncertified ruler).

Figure 4b
. Case 2 -The low endpoint and high endpoint graphs for a coverage factor of 95% represent the PDF generated for rain gauge measurement using Monte Carlo simulation (using an uncertified ruler).
Case 1. Estimation of measurement uncertainty of a rain gauge using a calibrated dial caliper. In case 1, the measurement uncertainty obtained from both methods had good agreement with similar values. This is because the model used to calculate the measurand (Equation 5) has insignificant nonlinearity. However, when the model presents elements with strong nonlinearity, the approach made by truncating the first term in the Taylor series (Equation 2) used in the GUM approach may not be sufficient to accurately estimate the uncertainty of the output quantity (Paulo, at al, 2013).
The LPU-GUM method uses the central limit theorem (CLT) approach to generate the output probability distribution where the validity of the central limit theorem states that the convolution of a large number of distributions will produce a normal distribution (Paulo, at al, 2013). Thus, it is assumed that the output probability distribution approximates a normal distribution and can be represented by a t-distribution. On the other hand, the output probability distribution from the MCS method is also in the form of a normal distribution (Figure 3), which is due to the dominant normal distribution of input sources, resulting in agreement in the output probability distribution between the LPU-GUM method and the MCS method.
Case 2. Estimating the measurement uncertainty of a rain gauge using an uncalibrated ruler.
In case 2, the uncertainty of measurement obtained from both methods showed a significant difference, where the uncertainty of measurement from the LPU-GUM method was three times larger than that from the MCS method. This was due to the use of an uncalibrated ruler with a 5 mm division, resulting in limited information on the measurement uncertainty available only from the ruler division. It was assumed that readings could be made with a maximum accuracy of half of the ruler division, or 2.5 mm, which could be considered as an interval of ±2.5 mm as the measurement limit. However, there was no probability information available within this interval, so the only PDF that could be assumed was a uniform distribution, where there was an equal probability for values throughout the interval.
In the LPU-GUM method, regardless of the type of input distribution, the output distribution was assumed to be normally distributed due to the use of the central limit theorem (CLT) approach, as explained previously. In this case, the resulting distribution showed asymmetrical or non-normally distributed characteristics because the dominant input distribution was a uniform distribution (ruler division), thus invalidating the central limit theorem approach. This was proven by the MCS method, which produced a probability distribution of the output that tended to approach a uniform distribution (Figure 4).

Advantages and disadvantages
In the MCS method, calculations related to partial derivatives, sensitivity coefficients, and degrees of freedom used in the LPU-GUM method have been eliminated. Therefore, in this MCS method, the mathematical calculations are significantly reduced. Thus, its use to determine measurement uncertainty does not require complex mathematical ability.
In its application, the MCS method requires a very large number of iterations, more than 200,000 iterations (JGCM 101 : 2008). If this is done without the help of reliable computing tools such as Python, R and others, it will consume a lot of time. Therefore, the MCS method requires reliable computing capabilities. This is the disadvantage of the MCS method because not everyone has computational skills.
The LPU-GUM method uses a central limit theorem (CLT) approach, which assumes that the output probability distribution is normally distributed. Thus, the LPU-GUM method is considered less reliable for cases where the dominant input distribution is not normally distributed.

CONCLUSION
In this paper, the estimation of measurement uncertainty for rain gauges using two different approaches, namely LPU-GUM and MCS methods, has been discussed. The uncertainty of measurement of the rain gauge for case 1 and case 2 has been obtained for both methods. From the data obtained in case 1, there is a good agreement of results for both methods, and the final results of the measurement uncertainty from both methods are comparable and consistent. However, the MCS method appears to be a better method with a flexible approach. From the data obtained in case 2, it can be concluded that there is a disagreement in the results for both methods. Furthermore, the final distribution shape of both methods was found to have a significant difference. However, the MCS method seems to be better because it does not assume the final distribution shape to be normally distributed; instead, the final distribution shape results from the combination of several input distributions. The MCS method has been proven more suitable, reliable, and convenient than the LPU-GUM method. It requires fewer mathematical calculations but better computational skills than conventional approaches.

RECOMMENDATION
Based on the results of this study, the following recommendations are suggested. Firstly, it is recommended that the calculation of rain gauge uncertainty is necessary to ensure accurate data. Accurate rainfall measurement is important in various fields, such as agriculture, water resource management, and climate studies. Therefore, it is crucial to estimate the measurement uncertainty of rain gauges to ensure the accuracy of rainfall data. Secondly, an appropriate and efficient measurement uncertainty estimation is required. The results of this study have shown that the MCS method is more suitable and reliable than the LPU-GUM method. The MCS method provides a more flexible approach and requires fewer mathematical calculations. However, it requires better computational skills compared to other conventional approaches. Therefore, it is recommended to use the MCS method for estimating measurement uncertainty for the rain gauge. Lastly, further research is needed on the effectiveness of the MCS method for estimating the measurement uncertainty on other measuring instruments. Although the MCS method is more suitable for estimating measurement uncertainty for rain gauges, its effectiveness on other measuring instruments must be investigated. Further research can enhance understanding of the MCS method's performance and applicability in different fields.