Skip to main content



The determination of the refractivity of air is of great importance for a number of measurement tasks. It is of particular concern when length is precisely determined in the free atmosphere using interferometry. In order to maintain the highest accuracy in interferometric length measurement it is essential to determine a value for the refractivity of the ambient air.

The refractivity of air may be determined by calculation in an equation or by direct measurement. The calculated refractivity may be inaccurate due to errors in any of the atmospheric measurements required for the equation or changes in the composition of the air.

The extent and causes of refractivity differences have been assessed using a refractometer instrumentation package in industrial environments where interferometers are used to calibrate a wide variety of engineering artefacts. In addition, measurement sensors used in industrial environments to determine air refractivity have been compared and an error analysis undertaken.

A commerical interference air refractometer with modifications was connected via a control interface system to a host computer. The control interface consists of a control system which allowed selected solenoid valves in the refractometer manifold to be operated by computer, a Piranic vacuum pressure measurement system, a bidirectional fringe counter and fractioning unit and a central power console incorporating spike suppression filters. The host computer was an IBM personal computer (PC) compatible system with an integral plasma screen which incorporated plug in interface boards to allow data aquisition and control of the refractometer. All the developed software incorporated screen prompts for user friendly operation. The accuracy of the refractometer was verified as having an uncertainty less than 6.2 E-8.

The atmospheric monitoring unit has been developed and incorporates air temperature, atmospheric pressure, relative humidity and carbon dioxide measuring sensors. The unit communicates with the control computer via an IEEE interface using a defined protocol. The complete unit had to be transportable and therefore the measuring devices has to be robust reliable and adequately stable. The unit is run by the system program UNWELT. The software enables sensor measurements, the collection of measurement and status data and the control of the air sample pump. The measurement data and the status are shown on the inbuilt screen. An error window and a communication window are available for control purposes. The calculated refractivity values from a modified Edlen equation are found to be within 4.5 E-8.
When measuring distance using laser interferometers it is necessary to take into account the refractive index of the air through which the beam passes. This is in turn a function of the composition of the air and thus errors may become significant when measurements are made in polluted environments if corrections are not made. To this end compensation devices are commercially availble but the limit of their effectiveness is unclear in comparison to the measurement accuracy of 1 part in 1E7 now sought by a number of key industries (e.g. electronics).

Work by PTB under typical industrial conditions demonstrated that refractive indices determined with commercial compensation devices could differ by up to 1 part in 1E6 from those measured with the PTB air refractometer. It is the aim of the project to determine reasons for these differences.


A portable measurement package composed of a refractometer and instrumentation for the measurement of air temperature, pressure, relative humidity and CO2 content has been built and calibrated. It enables to determine the refractive index of air directly or by calculation using the environmental measurements and correction equations, both methods having a relative uncertainty of less than 1E-7. The package has been circulated to 15 locations of 6 industrial companies. At each place, samples of air have been taken for subsequent chemical analysis. It has been shown that the index directly measured by the package and the calculated one differ by only about 1E-8 if a modified Edlen equation is used and corrections are applied for the levels of CO2 which exceeded that assumed by Edlen. Commercial compensation devices used by industry gave a measuring accuracy of >5E-7 whilst in-house compensators achieved about 1E-7. This is considered to be due to the greater stability of the sensors in the latter instruments and the employment of a more exact determination of humidity. If humidity is not measured but assumed to be 50% errors of the order 2E-7 can be expected. The best accuracies were achieved by refractometers which, however, need also regular calibration. Generally, the only significant air contaminant measured was CO2, the uncertainty due to hydrocarbon contamination being only of the order of 3E-9.
The results are published in EUR report 13517.

Funding Scheme

CSC - Cost-sharing contracts


Bundesallee 100

Participants (1)

National Physical Laboratory (NPL)
United Kingdom
Queen's Road
TW11 0LW Teddington