Evaluation of Type A Uncertainty in a Network Analyzer From 300 kHz to 8.5 GHz

A R T I C L E I N F O A B S T R A C T Article history: Received: 01 January, 2020 Accepted: 10 May, 2020 Online: 21 May, 2020 Network Analyzer is equipment widely used for the execution of radio frequency application scattering parameters. Throughout absolute reading the scattering parameter measured. An absolute reading does not include error, drift, offset, linearity, resolution, coefficient of sensitivity and several other variables that will contribute to the measured measurement dispersion. Type A evaluations of measurement uncertainty clarified according to ISO / IEC Guide 98-1. Type A uncertainty assumption is always done best to characterize an input quantity given in repeated indication values. The assumption was calculated from arithmetic mean, variance of probability distribution and standard deviation respected to the frequency from 300 kHz to 8.5 GHz in a Network Analyzer measurements. Furthermore, ISO/IEC 17025 is the standard for accredited testing and calibration laboratory to calculate the uncertainty of measurement to be declared in the scope of accreditation. Type A uncertainty calculation is one of the mandatory requirements for an accredited testing and calibration laboratory. The Type A uncertainty will be combined with Type B uncertainty to calculate the expanded uncertainty in measurement.


Introduction
National Instruments manufacture network analyzer. NI PXIe-5632 Vector Network Analyzer [1] is one of the popular and commonly used models in Radio Frequency (RF) manufacturing. The PXIe-5632 is a Vector Network Analyzer (VNA) able to perform measurements of the scattering parameters which are reflection coefficient and transmission insertion loss. The reflection coefficient measures the voltage reflected from the device under test (DUT). It is known as port 1 VNA reflection coefficient (S11) and port 2 VNA reflection (S22) in a full two ports VNA. The reflected voltage will measure as the mismatch [2] of the DUT. Ttransmission line from the VNA measured from port 2 to port 1 known as forwards transmission (S21) and reverse transmission port 1 to port 2 (S12) [3]. The transmission voltage will measure as the insertion loss.
The reflected voltage calculated from the prototype or product will help researchers and designers ensure that their design reaches impedance of 50-ohm. It is because RF devices usually use 50-ohm impedance to apply the principle of maximum power transfer. High reflection voltage will create a high return loss. The return loss will determine by ratio. The best loss of return is 1:1 ratio meaning there is no reflected voltage towards the generator. If metric 4:1 of the return loss is measured as 25% of the reflected power towards the generator [4]. The measured reflected voltage knows as absolute reading. The absolute reading does not contain the errors known as systematic and random errors. ISO / IEC Guide 98-1 [5] clarified the importance of estimated uncertainty would determine the accuracy and performance of the entire measuring system. The measurement uncertainty is calculated based on Type A and Type B contributors.
Type A uncertainty contributor is an evaluation of standard uncertainty based on any valid statistical calculation. The statistical calculation mainly estimated by using the sample of the mean and standard deviation of mean formula. The sample of the mean is based on the sample of repeated measurements under the same condition. The standard deviation of mean is the estimation of Type A uncertainty. The standard deviation of mean will help researchers to perform repeatability [6] and reproducibility [7] statistical analysis. The objective is to set up a stable and accurate test system for manufacturing production. It is mainly to increase ASTESJ ISSN: 2415-6698 precision and accuracy [8]. Throughout the statistical analysis, the researcher can enhance the system from low accuracy and low precision to a better performance set up. As the results of this paper, it will guide to create a good system with calculating the lower numbers of Type A uncertainty. This paper is an extension of the work originally presented in the 2016 7 th International Conference on Mechanical, Industrial, and Manufacturing Technologies [9] , Journal of Engineering Science and Technology Vol. 14, No3 (2019) 1587-1600 [10] and Journal Advances in Science, Technology and Engineering System Journal Vol 5, Issue 3 [11].

Selection of Network Analyzer
Until a manufacturer begins production, engineers are expected to conduct a lot of research until making the decision to invest in any of the production standards. Table 1 contrasts the variations between the network analyzer PXIe-5632, E5080B and ZNB8. Because of the cost-effectiveness and scale, the key reasons PXIe-5632 was chosen. A piece of cost-effective machinery will shorten investment returns. Capital investment risk [12] is low, and profitability is optimised. It is to ensure that the returns on the production product are greater than capital equipment expenditure and very necessary to persuade creditors that a sense of estimating the returns is adequate to cover the debt. Comparison of Network Analyzer between 3 different manufacturers and models was shown in Table 1. Every square foot is costing money on the production floor. An 18 slot PXI-Express chassis with 4U (7 Inches) rack size capable of fitting 6 PXIe-5632 Network Analyzer units. Compared to six E5080B and ZNB8 units, they are 36U (63 Inches) and 48U (84 Inches) respectively. As for power consumption, PXIe-5632 Network Analyzer's 6 units consume the same power rate in a single PXIe-1095 chassis with internal clock and trigger synchronization accuracy of timing system working well [13]. E8050B and ZNB8 are, however, going to consume 6 times the rated capacity. The 6 times increase in the rated power would generate a lot of heat on the production floor. Indirectly, the facility team had to raise the flow of air conditioner to sustain at 23 degree Celsius in the production cycle.
As a result, the operating cost increase in production by the volume of E5080B or ZNB8 running.   Figure 2 shown the Keysight manufactured E5080B 4-ports VNA. It is a highly integrated, complete characterization for radio frequency and microwave impedance measurement solution. The embedded software installed in the E5080B enables complete device characterization for passive device and components for researcher [14]. This VNA package with an extra wide bandwidth of up to 152 dB dynamic range feature improves the RF component and system testing throughput. It is approximate 10 times faster than PXIe-5632 but in terms of price, it is expensive in capital equipment investment. Figure 3 shown the ZNB8 network analyzer with the mechanical calibration kits used in an electrical laboratory to perform characteristic impedance calibration. ZNB8's key benefit is this VNA capable of extending up to 48 ports by using the matrix switches [15]. The switch matrix benefit is that it allows 48 channels to function simultaneously in output. This feature helps production reduce the waiting and testing period substantially, but the investment and maintenance plan cost will also be high. As a result, the operating cost increase in production by the volume of E5080B or ZNB8 running in a manufacturing production floor compare to PXIe-5632 is not merely cost-effective in terms of capital investment and manufacturing space savings as well.

Recommended Equipment for PXIe-5632 Calibration
From Table 1, it was found that the PXIe-5632 VNA is low in capital investment and physical small in size are the two main advantages compare to E5080B and ZNB8. The low investment in capital equipment will shorten the return of investment. The PXIe-5632 hardware small in size helps production significantly reduce the space for their test system. Table 2 shown the recommended equipment for PXIe-5632 during calibration. To operate the PXIe-5632 a PXI chassis with model PXIe-1095 was required. The PXIe-1095 available with 18 slots, where it can slot and work simultaneously in 6 units of PXIe-5632 VNA. The PXIe-1095 chassis equipped with a PXIe-8880 controller. Only 1 controller needed to operate all the 18 slots in a chassis. An automatic VNA calibration kit used to perform full 2 ports calibration. Figure 1 provided a complete calibration setup for the PXIe-5632 characteristic impedance calibration.
After full 2 ports calibration completed shown in Figure 4, the device under test (DUT) measurement is ready to be measured. In this analysis, a mini-circuit fixed attenuator VAT-3 used to measure the Type A analysis. It is a 3 dB insertion loss fixed attenuator. The fixed attenuator forwards transmission (S21) and reverse transmission port 1 to port 2 (S12) analysis shown in Figure 5. Type A measurement would acquire a small number of repeated measurements of N number to populate a precision Type A measurement. The minimum of repeated measurement is 2. It is to calculate a small number of standard deviations. The smaller number of standard deviation calculated, the better performance of the network analyzer. In this case study, it is applying 5 repeated measurement under the same condition including equipment, personal and environmental condition. The objective is to minimize the error and standard deviation.

Estimation of Type A Uncertainty Calculation
Evaluation of measurement uncertainty classified into two categories Type A and Type B. Type A evaluation evaluates the uncertainty resulted from the statistical study. The statistical analysis may be an undergrowth series of repeated measurements of the same process. The series of repeated measurements will fill in a standard deviation of mean to calculate the Type A uncertainty.

Arithmetic Mean in Type A Uncertainty Analysis
The estimation of Type A evaluation of uncertainty can be applied when a set of measurement was recorded under the same condition. Minimum repeated measurement is 2. Example a quantity of Y input with N statistically independent (N > 1) observed as (where j = 1, 2, 3, …, n). The estimation of quantity of Y is ̅ , arithmetic mean wrote as Equation 1 [16].
The system measured in the VNA under test may differ each time it under repeated condition. Even though it is the same set of equipment and the technician performs the measurement in production, it is difficult to determine the true value [17] from 300 kHz to 8.5 GHz frequency swept. This may be caused by various factors such as environmental condition, input line frequency stability, random error effect, impedance mismatch and others. Estimates the mean will help harmonize the cause of effect in the PXIe-5632 VNA towards the measurement. The higher sample size of repeated measurement will be resulting the measurement in lower noise.

The variance in Type A Uncertainty Analysis
The arithmetic means of ̅ is evaluated in Equation 1 will applied in Equation 2 to estimate the variance of probability distribution define as 2 ( ) [16] The size of the arithmetic mean samples determined by PXIe-5632 VNA which corresponds to Equation 2. In this study, it is very important that to keep the probability of variance distribution in a small number. The smaller number of the distribution of variance reflects the capability of PXIe-5632 VNA measured in good repeatability. Although the PXIe-5632 measured in good repeatability condition, it does not determine the system is accurate. The PXIe-5632 VNA framework must be validated against another external reference standard [18], which is specified in discussion.

Type A Uncertainty Analysis
After estimation of arithmetic mean and variance had been calculated, the experimental standard deviation for Type A uncertainty was calculated in Equation 3 [19].
The standard uncertainty ( ̅) associated with the initial set of measurement repeated ̅ is the experimental standard deviation of mean as shown in Equation 4.
The key purpose of repetitive analysis is to disperse the data into a predictive waveform or pattern. To fill a Type A precision calculation, a high-stability system will acquire several repeated measurements of N (example 5). Indirectly, it calculates a small number of standard deviations. Calculating the smaller number of standard deviations resulting the PXIe-5632 measured precisely. If the standard deviation analysis did not calculate in small number, the higher sample size of repeated measurement is required.

PXIe-5632 Type A Uncertainty Calculation
After 5 repetitive measurements were taken by PXIe-5632, a statistical analysis to analyse the data. Figure 3 indicates the unit under test swept at 50 MHz calculated S21 and S12 with a total number of 201 points to approximate the nominal 3 dB error population as guide. The maximum and the minimum were measured at respectively 3.06 dB and 2.94 dB. The error was calculated as 0.02% from 3 dB nominal reference point. A linear graph was plot, and the formula was determined as y=0.5012x+0.0147. R square was calculated at 0.3564. R square shown in Figure 3, calculate 35.64% variance distributed along the linear line at y=0.5012x+0.0147 as shown in Figure 6. The error calculated 0.02% from 3 dB with R square 35.64% conclude that the measurement variability of the dependent at 3 dB, where approximately at 71 points. The 71 points are distributed from 3.0075 dB (maximum) until 2.9983 dB (minimum). The remaining 130 points are calculated as 64.36% of the variability is still uncounted from total number of 201 points. However, the maximum and minimum error calculated at ±0.06 dB and it was measure at 3.06 dB and 2.94 dB respectively. The error of ± 0.06 dB was found insignificant respect to the product specification limit at ±0.15 dB. The population lies within the 3 dB attenuator specification.
A nominal 3 dB fixed attenuator repeatability distribution was plotted as shown in Figure 4. The population distribution close to normal distribution trend with total number of 199 points lies within the 95% confidence interval. There are only 2 points was measures outside the 95% confidence interval as shown in figure  4.  The results from Figure 6 and Figure 7 were found satisfaction, precision and accurate. It can conclude the system repeatability is good and stable.

PXIe-5632 Experimental Transmission S21 Analysis
Measurement of transmission S21 from port 2 of PXIe-5632 to port 1. It is also known as forward transmission measurement. The configuration shown as Figure 2. The concept is PXIe-5632 VNA power generated from port 2 transmitted to the DUT and received the power at PXIe-5632 VNA port 1. The power is transmitted over the DUT and received at port 1 also known as insertion loss. Equation 1 required minimum 2 repeated measurement to calculate the arithmetic mean. In this experiment. a total of 5 repeated measurements swept from 300 kHz to 8.5 GHz. Frequency swept at 8.5 GHz is the maximum capability for PXIe-5632 VNA. The 5 repeated measurement was proceeded with the same set of equipment used and perform by same technician in 5 consequences day in calibration laboratory. The repeated measurement results obtained in Table 1. From Table 3, minimum and maximum insertion loss S21 was measured at -2.935 dB and 3.562 dB at frequency 300 kHz and 8.5 GHz respectively. The frequency swept at 7 GHz , 8 GHz and 8.5 GHz are beyond manufacturer specification. It is expected when the frequency increased will results higher error and drift.  Figure 8 was taken from the manufacturer's manual Mini-Circuit VAT-3 Fixed Attenuator product specification. The manufacturer warranted specification only up to 6 GHz. This experiment was a case study of Type A uncertainty up to 8.5 GHz. The frequency swept from 6 GHz to 8.5 GHz does not adhere to the manufacturer specification. Although the frequency swept beyond the manufacturer warranted specification, this study only required a precise in repeated measurement. However, the repeated measurement study will helps to plot the statistical analysis and the VAT-3 fixed attenuator characteristic. The statistical analysis will calculate the error dispersion from nominal 3 dB attenuation. This error also contribute by drift [20] proportional to the PXIe-5632 VNA frequency increased. The precise repeated measurement will helps to reduce the Type A uncertainty in calculation.  Figure 9 shown the transmission S21 with repeated number of 5 measurement. From 300 kHz found the lowest error calculated as 2.47%. The maximum error was calculated as 18.44% from nominal 3 db reference point. The VAT-3 fixed attenuator warranted specification only up to 6 GHz cut off at -3.45 dB, but this device remain stable up to 8.5 GHz with maximum error less than 20 % from the nominal points. From figure 9, it can conclude that the attenuation loss and error propotional with frequency increased from nominal point at 3 dB.

PXIe-5632 Experimental Transmission S12 Analysis
Transmission S12 measurement is measurement from PXIe-5632 port 1 to port 2. It is also known as reverse transmission. The set up was shown as Figure 5. The methodology applied from transmission S21 to perform the transmission S12 analysis. The only different the PXIe-5632 VNA measured the transmission S12 opposite direction from transmission S21. The power will be transmitted from Port 1 and received in Port 2 at PXIe-5632. Table     Figure 10 shown the transmission S12 with repeated number of 5 measurement. From 300 kHz found the lowest error calculated as 1.64%. The maximum error was calculated as 18.72% from nominal 3 db reference point.

Mini-Circuit VAT-3 Transmission S21 and S12 Measurement
Error and Limit Table 5 indicates repeated measurement error on transmission S21 and S12. Transmission S12 error was found from Table 5 to calculate a minimum of 0.217% and a maximum error of 18.926%. The Mini-Circuit VAT-3 was swept beyond its manufacturer specification, so it's important to measure the frequency error maximum. The function of the S21 and S12 limits is used to monitor the repeatability of Type A analysis and identify any deviations or variations in performance by the measurement system as shown in Figure 11. If the S21 or S12 error is measured beyond the control limits, it is required to carry out validation and verification again from chapter 4.1. The main objective to perform validation and verification again is to detect the characteristic change from the equipment used or the device under test. The standard used characteristic changed may affect the accuracy in repeated measurement. The inaccurancy of the measured may cause by mishandling of the equipment or the equipment degraded over the time being used in production. The same causes happen to the device under test. The new limit is very important to maintain the stability and repeatability measurement and to ensure the system always running in optimum condition.  Figure 11 shown the Transmission S21 and S12 characteristic are close to each other. The device under test, Mini-Curcuit VAT-3 fixed attenuator is a known good attenuator and high stability device eventhrought the calibration was proceed approximate 20% beyond its warranted frequency specification.

The variance of S21 and S12
The variance of S21 and S12 is important to distinguish between the population from the data measured across the frequency. The variance of S21 and S12 were calculated from Equation 2. The variance is calculated from the 5 sample of measurements in S21 and S12. It was found that the observation

Measurement Error and Limit
S21 Error S12 Error S21 Limits S12 Limits does not vary from the group of mean swept from frequency 300 kHz until 8.5 GHz.
The calculated variance concludes that the 5-sample size of mean provide a low number of variances. This experiment does not require more sample of measurement. If the variance calculates a larger number, it represents that the individual frequency results was vary greatly from the group of mean. It could be contributing by any factor such as personal, skills, equipment, environmental condition and other factors.  The results from Table 6 is ready to calculate the Type A uncertainty.

Type A Uncertainty Calculation
The Type A uncertainty calculated by applying the Equation 3. Standard deviation is calculating for each frequency points. It is several variations between each data point relative to the mean. The calculated standard deviation in Table 7 will be the calculated Type A uncertainty as per Equation 4.   Table 7, the minimum standard deviation for S21 measure at 0.003314 at 300 kHz meanwhile S12 measure at 0.002196 at 1 GHz. The minimum Type A uncertainty represent the best repeatability measured in the system. The maximum standard deviation for S21 and S12 were calculated as 0.021 and 0.019 respectively at 6 GHz. The worst case [21] of standard deviation demonstrate the maximum deviation at 6 GHz. From the worst case Type A uncertainty, it can observe that occurs at the maximum frequency point at warranted specification. As the conclusion from Table 7, the minimum and maximum standard deviation did not proportional towards the frequency increased. The minimum and maximum standard deviation could scatter between the frequency range from 300 kHz until 8.5 GHz. Table 8 and Figure 12 shown the Type A uncertainty evaluation reported into the device under test measurement report. The Type A uncertainty reported respect to its frequency point. The reading of transmission S21 and S12 were the magnitude of absolute measurement from PXIe-5632 VNA and associate with Type A uncertainty. Each of the individual Type A uncertainty will be computed with Type B uncertainty to calculate the total expended uncertainty expended uncertainty which the total expended uncertainty should reported in 2 decimal points [11]. Repeated measurement concept is involved multiple measurement of each frequency. Under the repeated measurements condition, the device under test will exposed the best and worst conditions from the calibration report. It is a statistical measurement calculate the variation in the response to the variation variable that is cause by random and systematic errors. The random and systematic effect will be analysed when performing the Type A and Type B uncertainty.

Validate Type A Uncertainty by Using Correlation S21 and S12 Method
The correlation between S21 and S12 is important to ensure the performance of the Network Analyzer generator and receiver measured the power precise and accurately. The correlation of S21 and S12 calculated as shown in Table 9. The mean of S21 and S12 were calculated from Table 3 and 4 respectively by applying the equation 1 to verify the correlation of the transmission measured by the PXIe-5632 VNA. In ideal case, both transmission S21 and S12 should be the same magnitude in term of forward and reverse insertion loss. It is because the same device under test measured at the same condition. However, this theory does not apply in the experimental. There are gaps identified in the calibration.  Figure 13 shown the correlation of S21 and S12 from 300 kHz until 8.5 GHz. From the graph, it was found that the delta of mean is random. It does not proportional to frequency. The maximum delta of mean is 0.0192 dB at frequency 1 GHz and follow by 7 GHz and 6 GHz. Meanwhile, the minimum delta of mean measured at 8 GHz only differentiate with 0.0008 dB. Figure 13 : Correlation S21 and S12 Overall Figure 9 conclude that strong correlation between S21 and S12 trend crossing each other from 300 kHz until 8.5 GHz. The PXIe-5632 VNA port 1 and 2 are two different receivers to measure the signal received. The strong correlation concludes the PXIe-5632 VNA port 1 and port 2 measured insertion loss very close to each other by two different sensors applied in the system.

Verification of PXIe-5632 VNA Results with Other Original Equipment Manufacturer
Performing the Type A uncertainty analysis only with the PXIe-5632 VNA would not sufficient to determine the accuracy of repeated measurement. With the study had been done is required to perform the gaps analysis with another external system. It is reducing the risk to make mistake in the calibration and to ensure the PXIe-5632 VNA also demonstrate the strong correlation with another system. If the comparison between the PXIe-5632 and external system results are independent, a further detail analysis is required including validate the calibration procedure or hardware being used in this study. Figure 14 shown the same device under test measured by Rohde & Schwarz ZNB20 VNA from 100 kHz until 8.5 GHz. It was the screenshot from the ZNB20 VNA display. It shown the device under test trend across the frequency swept with markers function turn on. The reading pointed by markers will compare with the PXIe-5632 VNA. From the ZNB20 VNA was found the device under test characteristic does not linear from 1 GHz until 8.5 GHz. It is because the ZNB20 VNA perform the measurement without averaging function turn on. Without the averaging function, the device under test is measured under the "worst case" scenario. If the averaging function turn on, it is expected the device under test measured in a linear characteristic. Figure 14: ZNB20 Transmission S21 and S12 Absolute Measurement Figure 14 observe that the ZNB20 VNA measured the device under test similar to PXIe-5632 VNA. Table 10 and 11 calculate the delta between two independent system in term of precision and accuracy. It is a very useful information to research and development team improve the next generation of VNA design.   Table 10, the maximum delta between the two system occurs at 1 GHz. It is differentiated by -0.0917 dB or 2.98% above the PXIe-5632 VNA measurement. Overall Table 10 can conclude that the S21 does not measure any significant delta from 300 kHz until 8.5 GHz.  Table 11, the maximum delta between the two system occurs at 1 GHz. It is differentiated by 0.2314 dB or -8.98% below the PXIe-5632 VNA measurement. At 4, 6 and 8 GHz were found large deviation measured beyond 3% delta. Overall Table 11 can conclude that the S12 measure multiple significant changes from 300 kHz until 8.5 GHz. Especially at 1 GHz was identify maximum delta in both S21 and S12.

Summary of Discussion
In the discussion, there are 2 main issues were found. First issue identifies as the frequency swept at 1 GHz and second issue is transmission S12 comparison between PXIe-5632 and ZNB20.
Frequency swept at 1 GHz did not found significant issue when the measurement of S21 and S12 correlation between PXIe-5632 itself. The changes in insertion loss was low measured at 0.0192 dB. However, the same frequency point resulting the biggest delta compare to another system. This is the first improvement required corrective action in next research.
The second issue was found in transmission S21 comparison between PXIe-5632 and ZNB20. There are 4 out of 10 measurement points are beyond 3% delta. Although the 3% changes are not significant to both systems. If the ± 3% set as the benchmark or pass rate. Overall, only 6 out of 10 measurement points were measured below 3% changes. It is only score 60%. Nevertheless, corrective action required in next research as well.

Conclusion
In this study, an evaluation of Type A measurement uncertainty for NI PXIe-5632 Vector Network Analyzer from 300 kHz to 8.5 GHz is calculated. This frequency bandwidth is widely used in telecommunications particularly in the calibration system for the 5 G sub-6 GHz range that required to perform calibration on the telecommunication devices. This paper also carries out several the comparison technique applied in the PXIe-5632 VNA, valued to specific calibration techniques. A complete mathematical review shown the effects in each step involve in the Type A uncertainty analysis. This paper could help improve the scattering of parameters for commercial laboratories to extend the calibration potential in the new 5G telecommunication sector. The calibration analysis by using a 3 dB fixed attenuator also helps the commercial calibration laboratory explore existing measuring capacities and increase the degree of competence in the laboratory. In the discussion, it is explained the limitations are identify in existing PXIe-5632 VNA system. A part of the limitation of this paper would contribute to next improvement research area. As future of this work, the first development is to extend the frequency range beyond 8.5 GHz. The second development is extending to PXIe-5632 VNA Scattering Parameters port 1 (S11) and port 2 (S22). The calculation of the reflections S11 and S22 involved only port 1 or port 2. In addition, a measure of the test uncertainty ratio may also be added in future to satisfy the requirements of ANSI / NCSL Z540. A test uncertainty ratio also can be calculated to meet the ANSI/NCSL Z540 requirement.