The Accuracy of Predicted Acoustical Parameters in Ancient Open-Air Theatres: A Case Study in Syracusae

Featured Application

The work aims to give more insights into the relation between the sensitivity of the simulated objective parameters and the software input parameters for open-air ancient theatres. It is meant to raise awareness on the use of predictive acoustic software for unconventional outdoor environments in order to validate the possibility of re-using them as performance spaces.

Abstract

Nowadays, ancient open-air theatres are often re-adapted as performance spaces for the additional historical value they can offer to the spectators’ experience. Therefore, there has been an increasing interest in the modelling and simulation of the acoustics of such spaces. These open-air performance facilities pose several methodological challenges to researchers and practitioners when it comes to precisely measure and predict acoustical parameters. Therefore this work investigates the accuracy of predicted acoustical parameters, that is, the Reverberation Time (T₂₀), Clarity (C₈₀) and Sound Strength (G), taking the ancient Syracusae open-air theatre in Italy as a case study. These parameters were derived from both measured and simulated Impulse Responses (IR). The accuracy of the acoustic parameters predicted with two different types of acoustic software, due to the input variability of the absorption and scattering coefficients, was assessed. All simulated and measured parameters were in good agreement, within the range of one “just noticeable difference” (JND), for the tested coefficient combinations.

1. Introduction

The recent interest in the design of ancient theatres and in their acoustical characteristics has drawn attention to the lack of methodologies in metrology for historical acoustics [1]. The ISO 3382-1 standard [2] was used in the European ERATO project [3] to evaluate the acoustical apparatus of ancient theatres through room acoustic parameters, such as the Early Decay Time (EDT), Reverberation Time (RT), Clarity (C₈₀), and Sound Strength (G). However, ISO 3382-1 basically refers to indoor environments and temporal decay parameters seem to be less suitable for open-air conditions [4,5,6,7,8]. Farnetani et al. [4] reported that EDT is not a robust predictor of the acoustic quality of open-air theatres. The lack of robustness in EDT is due to a marked and intrinsic variability of this parameter, according to the source position, which defines the delay and incidence direction of the first reflections to the receivers. The same study asserted that RT behaviour in an open-air theatre is clearly different from that dealt with in the classical reverberation theory, which refers to a reference room volume. However, this parameter showed a limited variability. Chourmouziadou et al. [5] also suggested the use of RT when comparative studies are performed. However, it should be utilised with caution since it is usually used to evaluate enclosed spaces. Mo et al. [6] conducted a listening test with monaural and binaural auralisations of an open-air space. They stated that the perceived reverberance in an unroofed space is not only affected by the temporal characteristics during the decay process, but also by the spatial characteristics, due to the distribution of the reflections. The results showed that the conventional RT described in ISO 3382-1, which only deals with the sound energy decay rate, is not suitable for evaluating the reverberance of an unroofed space. Thus, more insight is needed into the adoption of an indoor acoustic measurements standard for the investigation of the acoustic conditions of open-air theatres. These sites represent particular environments that have their own specific sound field, which is rather different from the ideal diffuse field.

Besides the doubts about the applicability of the aforementioned indoor standard to outdoor case studies, other specific problems could arise when conducting measurements in ancient theatres. In fact, archaeological field measurements are also clearly influenced by the current conditions of the architecture of the theatres. Most ancient theatres have undergone damage of anthropologic and atmospheric nature. It was attested in Farnetani et al. [4] that the measured values of RT, G, and C₈₀ in ancient theatres are affected to a great extent by the state of conservation of the theatres themselves, with particular reference to the completeness of the architectural elements. Therefore, it is currently difficult to design acoustical correction guidelines for their contemporary reuse as performance spaces. Moreover, particular attention should be paid to the outdoor environmental conditions, such as temperature (t), relative humidity (RH), and air velocity, which could affect the variability of the measurement results, in the same way as for indoor measurements [9,10].

The topic of acoustical characterisation has already been examined in detail for indoor spaces, through statistical analysis, in order to investigate the reproducibility of measurements, the accuracy of the parameter calculation, the influence of source-receiver position displacement, and the measurement chains of different systems [9,11,12].

An alternative to the experimental acoustical characterisation is the virtual reconstruction of the theatre, using room acoustics simulation software. Since they were introduced, geometrical acoustic (GA) software applications have been used as the standard room acoustics models [13]. In order to enable a better acoustic design of existing buildings, the simulations first need to replicate the real acoustical conditions of the examined environment through three important steps: (1) appropriate geometry modelling; (2) material properties; and (3) simulation settings. This procedure, namely, the calibration of the model, is even more complicated for open-air theatres as the acoustic scattering and diffraction phenomena are more relevant than in closed theatres [14]. An appropriate calculation method and a geometrically detailed model are of fundamental importance to achieve accurate predicted results [15].

The reliability of simulations is an on-going matter of discussion and interest, as testified by the Round Robin comparisons of room acoustic modelling tools [16,17,18], and the more recent overview on the uncertainties of input data in simulations [13]. In the latter overview, it was reported that the specific uncertainties that characterise the absorption coefficient (α_w) and scattering coefficient (s) of materials [19,20] could affect the estimation accuracy of room acoustic parameters in the end. Such parameters are derived from simulated Impulse Responses (IR) or from energy reflectograms, depending on which analysis algorithm of the room acoustics software is being used. In situ and scale measurements [4] have revealed that the IRs of ancient theatres are composed of the direct sound and of two major reflections, which come from the orchestra floor and the scaenae frons (the ancient stage building), respectively, when these parts of the theatres still exist. Therefore, in the case of open-air theatres, the IR should be modelled with a limited number of specular reflections and a high number of scattered reflections, because of the irregularities in the steps of the cavea [21]. This configuration is difficult to handle using geometrical acoustic-based software (GA), such as Odeon (Version) and CATT-Acoustic [22,23]. Yet, most of researchers still rely on such tools also for open-air theatres in everyday practice; thus, special attention should be given to properly controlling the boundary conditions. In fact, open-air theatres represent a special case, which creates a challenge for these prediction algorithms. The absence of a roof, and therefore of a reverberant field, urges to have a high reliability in the prediction of the early reflections. Moreover, the concave shape of these theatres is responsible for the creation of “shadow zones” of the mirroring surfaces in great lateral areas of the cavea [14]. This affects the deterministic method of the Image Source, which is used by the GA software to build the early part of the IRs.

The aim of this work is to assess the performance of predictive software in calculating a set of acoustic parameters for ancient theatres, a particular type of open-air spaces, taking the case study of the ancient theatre of Syracusae (SR). The objective is to give more insight about the relation of the sensitivity of the simulated results to the input parameters. It is mainly referred to raise awareness on the use of this kind of software for outdoor unconventional environments. The theatre is located in Sicily, an island in the South of Italy, a region where ancient Greek culture had historically a lot of influence. The simulation accuracy of two kinds of software, Odeon and CATT-Acoustic, is considered. This theatre was selected because it was relatively easy to model due to the lack of contemporary additional elements. In this manner, the virtual model of SR could be considered as a valid archetype model. The paper is organised as follows:

• Section 2 (Case Study) includes a brief description of the state of conservation of the theatre chosen for this research.
• Section 3 (In Situ Measurements) includes a description of the acoustical measurement campaign carried out in the investigated theatre.
• Section 4 (Uncertainty Expression of the Acoustic Prediction Models) comprises the assessment of the uncertainty contribution related to the absorption (α_w) and scattering (s) input data assigned to the materials, predicted with Odeon (v. 13.02) (Odeon A/S, Lyngby, Denmark), and with CATT-Acoustic (v. 9) (CATT, Gothenburg, Sweden) software.
• Section 5 (Discussion) is focused on analysing the differences between measured (in situ) and predicted (through software) acoustic parameters, and it includes a discussion on the overall limitations of the study.

2. Case Study

The theatre of Syracusae (SR) was chosen as case study for a measurement campaign carried out by the Department of Energy at the Politecnico di Torino, from the 5th to 7th September 2015. SR (Figure 1) has Greek origins, dating back to the 5th century BC, but it was later modified by the Romans. Apart from a few ruins, nothing is visible of the original scaenae frons, but the surviving part of the rock-cut cavea has a diameter of 105 m.

Several studies that refer to the acoustics of SR have been retrieved from literature. These studies refer to measurements on a scale model of the ancient theatre and its contemporary use [4,24], to acoustic and lighting simulations [25], and to in situ acoustical characterisations with temporary scenery [26]. Measurements had only been carried out in empty conditions at one point of the orchestra area, as a pilot study in which different techniques were used [27].

This ancient open-air theatre is intensively used during cyclic summer season festivals in its current (deteriorated) condition, and acoustic measurements are made also for conservation purposes. Therefore, this study concerns the “historical acoustics” research field, which is the study of the auditory and acoustic environment of historic sites and monuments [1], with a valorisation purpose. The empty condition has been chosen for obvious practical reasons, as with the public present it is very difficult to carry out reliable measurements due to high background noise levels and unsteady boundary conditions [28]. Moreover, in order to simulate correctly the presence of public or the placement of an acoustic shell for renovation purposes, the reliability of simulated data must be verified, starting from the calibration of the acoustic model.

3. In Situ Measurement Methods

Standard measurements have been performed in unoccupied conditions, with omnidirectional sound sources and receivers, as stated in the ISO 3381-1 [2]. Different considerations on ancient theatre measurements, defined during the European ERATO Project [3], were taken into account. The measurement results for SR have been used in Section 4 for the calibration of the simulation model and as references for the acoustic parameters predicted through computer simulations. The source and receiver positions for the theatre are shown in Figure 1b.

Receivers were positioned on three radial axes of the cavea in the theatre, 1.2 m above the floor resembling the height of the ear of a sitting person. An omnidirectional microphone (Shoeps CMC 5-U, Durlach, Germany) was used to record the IRs. Ten receiver positions were considered. There was only a single microphone, meaning that all position measurements were carried out sequentially.

Measurements were repeated two or three times for each source position for most of the receivers, in order to evaluate the repeatability of the results. Two source positions were investigated: S1 was shifted horizontally by 1 m from the centre of the orchestra, in order to avoid any acoustical focus [29]; S2 was located behind S1, closer to the ancient scaenae frons position. The S1–S2 distance was equal to 7.6 m. Firecrackers were used as impulsive sources to measure the IRs (“Raudo Manna New Ma1b” Napoli, Italy and “Perfetto C00015 Raudo New”, Napoli, Italy). The IRs were measured directly by recording the impulse produced by the firecracker blast. Firecrackers were used in S1 and S2, in order to overcome the problem of the low Signal-to-Noise Ratio (SNR): they maximise the SNR and this constitutes a significant advantage in outdoor measurements, but on the other hand, caution should be used as they are also more likely to be influenced by random effects (e.g., atmospheric conditions and random directivity). According to San Martin et al. [30], in the case of firecrackers the generated impulse is nearly omnidirectional. Its directivity index is, on average, around 1 dB for the octave bands between 125 Hz and 16 kHz. In addition, both its time curve and spectral power are highly repetitive, resulting in levels above 115 dB (reference 1 pW) within the aforementioned range.

The Background Noise Level (BNL) was measured as an equivalent continuous A-weighted sound pressure level (L_Aeq) over a period of 10 min, before the measurement sessions. The measured BNL was 45 dB (A), in unoccupied conditions.

The sound source was positioned at a height of 1.5 m from the floor, and a custom-made tripod was used to hold the firecrackers in a fixed position. Aurora (version 4.4, Parma, Italy) was used as acquisition software.

The air temperature and relative humidity were monitored during the whole measurement campaign, using a thermometer/hygrometer, Testo 608-H1 (Croydon South, VIC, Australia). The wind speed was measured by means of an anemometer, Testo 450-V1 (Croydon South, VIC, Australia). The environmental parameters acquired during the measurements campaign were t = 33 °C, RH = 65%, wind speed = 0.30 m/s. These did not change significantly during the measurement campaign.

In order to characterise the acoustical conditions of a performance space, the ISO 3382-1 standard lists a series of parameters that can be obtained from the IRs measured at each receiver position. Although open-air theatres cannot be considered typical performance spaces, like closed theatres or concert-halls, the ISO 3382-1 standard was used as the reference for the acoustical characterisation. In particular, the following room acoustical parameters were measured, as these are considered the most relevant parameters for the acoustical characterisation of open-air theatres [4]:

• Reverberation time, RT, (s): duration required for the space-averaged sound energy density in an enclosure to decrease by 60 dB after the source emission has stopped. The integrated impulse response method was applied to obtain the RT from the IR [2]. RT can be evaluated on a smaller dynamic range than 60 dB and extrapolated to a decay time of 60 dB. It is then labelled accordingly. The RT in SR was derived from decay values of 5 dB to 25 dB below the initial level, and it was therefore labelled T₂₀.
• Clarity, C₈₀, (dB): the balance between early- and late-arriving energy. This was calculated for an 80 ms early time limit, as the results were intended to relate to music conditions, using equation:

  $${\mathrm{C}}_{80}=10 \log \frac{{\int }_0^{80}{\mathrm{p}}^2\left(\mathrm{t}\right)\mathrm{dt}}{{\int }_{80}^{\infty }{\mathrm{p}}^2\left(\mathrm{t}\right)\mathrm{dt}}$$

  (1)

  where p(t) is the instantaneous sound pressure of the impulse response measured at the measurement point.
• Sound Strength, G, (dB): the logarithmic ratio of the measured sound energy (i.e., the squared and integrated sound pressure) to the sound energy that would arise in a free field at a distance of 10 m from a calibrated omnidirectional sound source, as expressed in the following equations:

  $$\mathrm{G}=10 \log \frac{{\int }_0^{\infty }{\mathrm{p}}^2\left(\mathrm{t}\right)\mathrm{dt}}{{\int }_0^{\infty }{\mathrm{p}}_{10}^2\left(\mathrm{t}\right)\mathrm{dt}}={\mathrm{L}}_{\mathrm{pE}}-{\mathrm{L}}_{\mathrm{pE},10}$$

  (2)

  in which

  $${\mathrm{L}}_{\mathrm{pE}}=10 \log \left[\frac{1}{{\mathrm{T}}_0}\underset{0}{\overset{\infty }{\int }}\frac{{\mathrm{p}}^2\left(\mathrm{t}\right)\mathrm{dt}}{{\mathrm{p}}_0^2}\right]$$

  (3)

  and

  $${\mathrm{L}}_{\mathrm{pE}, 10}=10 \log \left[\frac{1}{{\mathrm{T}}_0}\underset{0}{\overset{\infty }{\int }}\frac{{\mathrm{p}}_{10}^2\left(\mathrm{t}\right)\mathrm{dt}}{{\mathrm{p}}_0^2}\right]$$

  (4)

  where:
  • p(t) is the instantaneous sound pressure of the impulse response measured at the measurement point;
  • p₁₀(t) is the instantaneous sound pressure of the impulse response measured at a distance of 10 m in a free field;
  • L_pE (dB) is the sound exposure level of p(t);
  • L_pE,10 (dB) is the sound exposure level of p₁₀(t);
  • p₀ is the reference sound pressure of 20 μPa;
  • T₀ is the reference time interval of 1 s.

In the above equations, t = 0 corresponds to the start of the direct sound, i.e., which corresponds to the arrival of the direct sound at the receiver, and ∞ should correspond to a time that is greater than or equal to the point at which the decay curve has decreased by 30 dB [2].

G requires a calibration procedure for the sound power of the source. Different procedures have been described previously [2]. L_pE,10 can be calculated from the sound pressure p_d(t) measured at a source-to-receiver distance d (≥3 m) according to the following equation:

$${\mathrm{L}}_{\mathrm{pE},10}={\mathrm{L}}_{\mathrm{pE},\mathrm{d}}+20\log \left(\frac{\mathrm{d}}{10}\right)$$

(5)

where:

L_pE,d (dB) is the sound exposure level of p_d(t), obtained from (3) (using p_d instead of p₁₀).

The Aurora plugin was used for the calculation of G with the firecrackers [31]. According to this procedure, the anechoic segment (direct sound) of each IR was used for calibration, providing the distance between the source and the receiver that allows for the estimation of L_pE,10, and it is recommended to keep a length of the IR of at least 1 s and to silence the signal just after the end of the direct sound. In this way, the smearing out in time caused by the octave filtering does not push the energy outside the time window, even at a low frequency, and the correct value of the signal level can be computed. A calibration file was obtained in situ from each analysed IR and was used to calculate the G value for that measurement path, with the knowledge of the exact source-to-receiver distance.

The resulting dataset is composed of the octave-band values from 125 Hz to 8 kHz of the acoustic parameters calculated by the Aurora software (v. 4.4) [31] from the measured IRs.

Measurements Results

The measurement results at receiver positions R1–R10 are reported in

Table 1. Mean values and standard deviations of the measurements for the T₂₀, C₈₀ and G acoustical parameters for the firecrackers, in positions S1 and S2. The data refer to the averages of the 500 Hz and 1 kHz octave bands and to the repetitions for the same receiver position. The rows spatial means are also reported. The standard deviations of the spatial means outside the JND range (for a definition see Section 4) are shown in bold. INR is also reported to help assess the quality of the measurements.

Acoustical Parameters
Row	Receiver	No. of Repetitions		Distance from Source (m)		T20 (s) (St. Dev.)		C80 (dB) (St. Dev.)		G (dB) (St. Dev.)		INR (dB)
		S1	S2	S1	S2	S1	S2	S1	S2	S1	S2	S1	S2
First row	R3	2	2	13.8	18.7	0.58 (0.09)	0.80 (0.02)	20.8 (3.0)	13.3 (0.6)	−0.3 (0.3)	−3.0 (0.1)	58	60
	R6	3	2	14.6	21.2	0.70 (0.04)	0.84 (0.01)	16.9 (2.3)	15.2 (0.6)	−0.9 (0.4)	−5.2 (0.1)	51	57
	R9	2	2	15.5	23.0	0.31 (0.05)	0.60 (0.14)	22.1 (0.5)	16.9 (0.8)	−2.4 (0.1)	−9.7 (0.0)	52	55
Sp. mean				14.6	20.9	0.53 (0.20)	0.75 (0.13)	19.9 (2.7)	15.1 (1.8)	−1.2 (1.1)	−6.0 (3.4)	-	-
Second row	R2	2	2	23.3	27.5	0.81 (0.07)	0.83 (0.01)	16.5 (0.7)	11.0 (0.8)	−4.8 (0.2)	−6.4 (0.1)	57	55
	R5	2	2	24.0	30.2	0.66 (0.19)	0.85 (0.03)	19.2 (3.8)	13.3 (0.2)	−5.4 (1.0)	−7.2 (0.1)	50	52
	R8	2	2	24.9	32.3	0.71 (0.16)	0.91 (0.11)	15.9 (2.8)	13.0 (1.6)	−6.0 (0.0)	−8.5 (0.4)	50	54
Spatial mean				24.1	30.0	0.73 (0.08)	0.87 (0.04)	17.2 (1.8)	12.4 (1.3)	−5.4 (0.6)	−7.4 (1.1)	-	-
Third row	R1	2	2	31.7	35.6	0.98 (0.05)	0.96 (0.01)	15.4 (2.0)	11.2 (0.1)	−7.8 (0.5)	−8.3 (0.1)	52	53
	R4	3	2	32.4	38.4	0.94 (0.15)	0.98 (0.03)	16.9 (1.8)	12.7 (1.1)	−8.0 (0.7)	−9.6 (0.3)	51	51
	R7	2	2	33.3	40.6	1.04 (0.09)	1.05 (0.01)	15.7 (0.2)	13.9 (0.0)	−9.3 (0.2)	−10.5 (0.0)	51	50
Spatial mean				32.5	38.2	0.99 (0.05)	1.00 (0.05)	16.0 (0.8)	12.6 (1.3)	−8.4 (0.8)	−9.4 (1.1)	-	-
	R10	2	2	39.2	45.6	1.31 (0.03)	1.91 (0.05)	14.3 (0.2)	12.3 (0.1)	−10.2 (0.3)	−11.2 (0.4)	52	49

, expressed as T₂₀, C₈₀, and G acoustical parameters obtained with firecrackers at source positions S1 and S2. All the values are the averages of two or three repetitions at each receiver position and of the central 500 Hz and 1 kHz octave band frequencies, as indicated in ISO 3382-1 [2]. In accordance with the ISO 3382-1, spatial averages for each row were also reported in Table 1. It was assumed that each row can be considered as a homogeneous area, as in open-air theatres the direct sound and the distance from the source play a predominant role in the acoustic response. The Impulse Response-to-Noise Ratio, INR, (dB) is also reported as a parameter for judging the validity of the measurement, in order to establish the reliability of the outdoor acoustical measurements [31]. According to ISO 3382-1, the source level should be at least 35 dB above the background noise level in the corresponding frequency band for the case of T₂₀. All the measurements considered in this study had INR values well above 35 dB and up to 60 dB for the octave bands from 250 Hz to 8 kHz. It is important to underline that in the case of the T₂₀ values, the larger standard deviation is due to the presence of only one strong reflection from the orchestra after the direct sound (as shown in Figure 2), which determines an irregular course of the decay curve and a greater variability in the slope of the decay curve.

4. Uncertainty of the Geometrical Acoustic Prediction Models

In the acoustic domain, it is important to recall that the parameters have the aim of evaluating the perception of the acoustic signal, namely the average capability of a “conventional” listener to notice sound variations. An important factor that correlates the subjective field to objective measures has been defined as the Just Noticeable Difference (JND), that is, the smallest perceivable change in a given acoustical parameter, which is specified for information in Annex A of ISO 3382-1 [2] for central frequencies (500 Hz and 1 kHz), but which is also acceptable for lower and higher frequencies [32,33,34,35]. This issue will be further discussed, when analysing the accuracy of the acoustic prediction models.

The uncertainty contribution of the input data, propagated to the results obtained from two different types of room acoustic software, Odeon version 13.02 and CATT-Acoustic version 9, was assessed and compared with the measurements values.

Odeon version 13.02 [22] is based on a hybrid calculation method. Early reflections are calculated through a mixture of the Image Source Method and the Ray-Tracing Method (RTM), by means of a stochastic scattering process that uses secondary sources. Late reflections are calculated by means of a special RTM, where the secondary sources radiate energy locally from the surfaces and are assigned a frequency-dependant directionality, namely the reflection-based scattering coefficient. The secondary sources may have a Lambert, Lambert oblique, or Uniform directivity: this directivity depends on the properties of the reflections as well as on the calculation settings.

CATT-Acoustic version 9 [23] is made up of two modules: CATT-A is the main programme, and it handles the modelling, surface properties, and directivity libraries, and TUCT (The Universal Cone Tracer), which is the main prediction and auralisation programme. TUCT can use three alternative cone-tracing algorithms: the first algorithm is based on stochastic diffuse rays, while the second and third algorithms are based on the split-up of the actual diffuse rays. The difference between these modules is that the latter handles two orders of diffuse split-up reflections in a deterministic way, thus resulting in lower random run-to-run variations.

Both CATT-Acoustic and Odeon base their scattering algorithms on two main implementations, which are described in detail in a previous paper [36]. These two methods are the Hybrid Reflectance Model (HRM) and vector mixing (VM). The HRM method complies with the definition of the scattering coefficient based on ISO 17497-1 [20] which defines it in a quantitative way as the fraction of the non-symmetrically reflected energy. In the HRM method, a random number between 0 and 1 is used to determine whether the reflection is specular or scattered. This number is compared with the surface scattering coefficient (s) assigned to the surface. In case it exceeds the value of s, the scattered energy is assumed to be distributed according to Lambert’s Law, i.e., the intensity of the reflected ray is independent on the angle of incidence but proportional to the cosine of the angle of reflection. This is the basic concept implemented in CATT-Acoustic [23] and in Odeon for the uniform and Lambert directivity scattering [22]. On the contrary, the VM is based on the linear interpolation of the specular and diffuse reflection [37]. In this way the direction of a reflection vector is calculated by adding the specular vector scaled by a factor (1-s) to a scattered vector following a certain direction that has been scaled by a factor s. This is the basic concept implemented in Odeon [22,38,39], named “vector-based scattering”, where the scattered vector follows a random direction, generated according to the Lambert distribution named oblique Lambert directivity.

4.1. General Procedure for the Implementation of the Models

In order to compare the two software packages and to obtain the best match with the measurement results, it was necessary to perform simulations with the same geometric model and source/receiver positions as in the measurements. To the best of the authors’ knowledge, this preliminary benchmark procedure has never been performed before on ancient open-air theatres, although many studies on indoor environments have been conducted [13,14,15]. Both types of software used for the simulation, that is, Odeon and CATT-Acoustic, have been validated in Round Robin tests. One of the main findings of these tests was that precise knowledge of the characteristics of the surface material is an important prerequisite for a reliable room simulation. Thus, a more detailed analysis on absorption and scattering coefficient changes was proposed.

A preliminary benchmark test study was carried out on SR, whose model had previously been used in different investigations, e.g., simulations concerning its ancient conditions, during the European ERATO project, [3] and in investigations on its contemporary use [25]. Figure 3a shows the 3D model configuration of SR.

The procedure applied for the comparison of the simulation tools was focused on solving the following issues:

• Geometrical model: MATLAB software, version R2015b, was used to create a parametric open-air theatre script. Two 3D cavea model script outputs were created, one suitable for Odeon (dxf file) and the other for CATT-Acoustic (.geo file). In order to reduce the simulation time, the theatre geometry was simplified and designed as symmetric. A few geometrical simplifications have been performed in both Odeon and CATT-A models. The number of surfaces was 1357 in CATT-Acoustic and 1362 in Odeon. As recommended previously [14], the steps were modelled. The higher number of surfaces in Odeon corresponds to an additional boundary box with totally absorbing walls and top which is required in Odeon to simulate open-air conditions [22]. CATT-A algorithms are implemented in order to detect lost rays, i.e., rays that escape from the geometrical model. In open cases, such as an open-air ancient theatre, rays disappear whenever they do not hit any surface during the calculation time. This principle is similar to the one used in Odeon, where the escaping rays disappear since they are totally absorbed by the boundary box. The circular geometry was modelled with 20 segments, as recommended by Charmouziadou [40], who showed that a number between 12 and 24 segments is optimal with respect to the influence on the objective acoustic parameters.
• Source-receivers: The source-receiver path was defined as in the measurement set-up, considering the theatre as unoccupied. For an easier comparison, only the source in position S1 was considered, as shown in Figure 3a.
• Surface material properties: The main surface considered in the model was the cavea, the stone and steps of which are not well-conserved. In both types of software, 20 material alternatives were assigned to the cavea stone, that is, from the most reflective one, with α_w = 0.05 and s = 0.25, to the most porous one, with α_w = 0.2 and s = 0.85, and all the intermediate combinations of α_w were tested in steps of 0.05, while s was tested in steps of 0.15, as explained in Figure 4b [14]. Other elements were then added: the remains of the ancient entrances to the orchestra area (aditi maximi), which was considered as an aperture (α_w = 0.9; s = 0), and the floor, which includes the ruins of the scaenae frons (α_w = 0.8; s = 0.8) and the better conserved orchestra area (α_w = 0.1; s = 0.2). Odeon and CATT-Acoustic software allow for frequency dependent absorption coefficients. The same absorption coefficients have been used for both software. The Odeon software allows giving as input value for the scattering coefficient the value as an average between 500 and 1000 Hz, and considers a frequency dependent scattering by using default interpolation curves as shown in the Manual. These curves have been used in CATT-Acoustic, i.e., a frequency dependent scattering coefficient, by inserting each value for each octave-band. The values given in Figure 4b refer to the mean values at 500 and 1000 Hz.
• General settings: The following settings were considered for all the simulations: a 100 dB source sound power level, 1500 ms as the impulse response length, and 4 million rays. The Transition Order (TO) in Odeon was limited to 1, which better resembles the impulse response characteristics in the real condition with only one specular reflection from the stage floor. The third calculation algorithm in CATT-Acoustic, described above, was chosen as it is the most suitable for the simulation of open-air spaces. Scattering and diffraction settings were defined as in

  Table 2. Scattering and diffraction set-up in Odeon and CATT-Acoustic.

  Phenomenon	Model	Odeon	CATT-Acoustic
  Scattering	Lambert	Disabled	Late part of the IR (not manageable by the user)
  	Oblique Lambert	Disabled	Not managed by the software
  	Uniform	Enabled, for early and late part of the IR	Early part of the IR (not manageable by the user)
  Edge + surface diffraction		Enabled (i.e., Reflection-Based Scattering)	Enabled (in CATT-A, i.e., Diffuse reflection)
  Diffraction after 1st order		Not managed by the software	Disabled (in TUCT)

  in order to allow a more coherent comparison between the two software. The diffraction phenomenon occurs when a sound wave hits edges, i.e., intersections between surfaces, or when the surface dimensions are limited. Both these events are taken into account by the software Odeon and CATT-A when the Reflection-Based Scattering and Diffuse reflection method are enabled. In Odeon, the Lambert and the Oblique Lambert functions for scattering were disabled, as suggested in a previous paper [39]. The uniform scattering distribution was considered more suitable for the cavea which is made of steps that can be reassembled as periodic triangular section [39,40,41], as shown in Figure 3a. In CATT-Acoustic, the diffraction after 1st order option was deactivated, even though it is usually suggested for ancient theatres [42], in order to take into account the current large amount of damage to the cavea steps in SR. In this way, it was possible to avoid the typical “chirp” echo due to diffraction phenomenon which has been attested to come from the regular stone steps in ancient theatres in empty conditions [43]. This phenomenon was not encountered during in situ measurements or recordings in SR. However, the first order diffraction has been taken into account since it occurs in coherence to the scattering phenomena. Moreover, based on the literature [44], it was found that higher orders and combinations of edge diffraction components were not usually as significant as first-order diffraction components when the receiver was visible to the source. The environmental data considered in both of the prediction tools were those obtained during the in situ measurements (t = 33 °C, RH = 65%).
• Data analysis: the analysis algorithm has been taken into account. Odeon conducts an energy based analysis, while the CATT-Acoustic software conducts both energy and pressure-based analyses. The variation of different types of analysis algorithms can lead to different results, as pointed out in Katz [45]. Thus, in order to avoid further uncertainty in the results, the simulated IRs have been exported and analysed by means of Aurora, version 4.4, in the same way as the measurements.

As reported in Vorländer [13], the level of detail in the model, besides the curved surfaces, is considered a systematic source of uncertainty. Besides the number of rays employed in the simulation, the absorption and scattering coefficients are defined as random sources of uncertainty. Both kinds of software use a ray-tracing method to build the late part of the IR. Since this method is based on stochastic calculation, which depends on the input general set-up data, it could affect the uncertainty of the resulting parameters when a run-to-run analysis is considered. All the aforementioned random sources of uncertainty were subjected to analysis, considering both the Odeon and CATT-Acoustic software, which, for the sake of an easier presentation of the results, will hereafter be referred to as O and C, respectively; the results are shown in the following sub-sections.

4.2. Run-to-Run Variation

The run-to-run variations of the applied algorithms are due to the stochastic implementation of the ray-tracing algorithm in the GA software. In order to test this effect, ten repeated simulations were performed with the GA model of SR, using both kinds of software. An analysis based on the assessment of the Normalized Error [46] was performed on the T₂₀, C₈₀, and G results, considering a confidence level of 95%. The results for each receiver position and octave-band frequency were all within the upper and lower limits of the respective limit range. This confirms the results obtained in analogous analyses conducted on an enclosed space [47].

4.3. Number of Rays

GA software usually distinguishes between deterministic and stochastic ray-tracing, depending on which algorithm is applied: The first algorithm is used to detect the image sources, while the second is used to estimate the reverberant tail. It is possible to select separately the number of early and late rays in O. Early rays are used in the deterministic ray-tracing, while late rays determine the ray density in the late part of the IR. The number of rays/cones in C only refers to the stochastic ray-tracing; that is the construction of the late rays. It becomes important to investigate the variation in results due to stochastic ray-tracing, which is a random source of uncertainty in GA.

Stochastic ray-tracing was here investigated by comparing simulations with different numbers of rays (4000–40,000–400,000–4 million). A Normalized Error analysis revealed that the results for each receiver position and octave-band frequency were all within the upper and lower limits of the respective limit range. This investigation was performed in order to verify the stochastic fluctuation, which may result as numerical errors in the results due to the low number of rays. This has been extensively studied and validated in systematic experiments [48]. The number of rays is strictly related to the systematic uncertainty in the final results of the parameters, and independently on the used method of the ray tracing, the fluctuations can be reduced by increasing the number of rays or by averaging repeated simulations. The choice of the number of rays becomes important in cases where large environments with uneven distribution of the absorption are considered. Therefore, a compromise should be reached between a very large number of rays and a smaller one since it may affect significantly the computation time. In fact, the reverberant field in a simulated open-air theatre is spatially uneven. The absorbing area is concentrated on the ceiling of the boundary box (in the case of O), while the theatre itself is mostly reflective. Thus, despite the prolongation of the computation time, a number of rays above 1 million would be preferable for the correct estimation of the reverberation tails at different receiver positions [22]. It is assumed that at least one ray is received at the longest source-to-receiver distance, which in this case is about 40 m (R10). The receiving area is considered as a spherical receiver with a radius r_d of about 0.06 m, thus the area of the visibility cone per ray A(ray) was 0.01 m². Considering that the total surface covered by the emitted rays is a sphere of radius 40 m, whose surface A(sph) is equal to 20,096 m², it is possible to calculate the minimum required number of rays N_min(rays) by means of Equation (6), which was also indicated in Vorländer [13]:

$${\mathrm{N}}_{\min }\left(\mathrm{rays}\right)=\frac{\mathrm{A}\left(\mathrm{sph}\right)}{\mathrm{A}\left(\mathrm{ray}\right)}= \frac{4{\left(\mathrm{ct}\right)}^2}{{\mathrm{r}}_{\mathrm{d}}^2}$$

(6)

where c and t are the speed of sound in air and the max arrival time counted from source excitation, respectively.

N_min(rays) is equal to 2 million rays. Thus, 4 million rays are necessary to ensure that at least two rays (instead of one) arrive at the receiver at a distance of 40 m from the source.

4.4. Absorption and Scattering Coefficients

The predictive software considers α_w and s as input variables that have to be assigned to the surfaces of the model. Thus, it is important to evaluate the uncertainty (U) of the calculated values, due to the uncertainty of the absorption (Uα_w) and scattering (Us) variables. These uncertainties were estimated to be higher than 0.05 and 0.15, respectively, as was found on the basis of the user’s experience in Vorländer [13] and Shtrepi et al. [49]. This case study considered only a few materials: stones and grass in particular. This allowed variations due to different α_w and s combinations regarding the cavea stone, which is the main surface considered in the model, to be investigated. To this aim, as shown in Figure 4b, twenty alternative materials were considered in both kinds of software, with α_w equal to 0.05, 0.10, 0.15, and 0.20, and with s equal to 0.25, 0.40, 0.55, 0.70, and 0.85. These values considered the possibility of having different degrees of damage on the steps of the cavea. In the case of the scattering coefficients of 0.85 [41], a perfectly preserved periodic triangular section with an angle of 45° has been considered, whereas in the case of scattering coefficient of 0.25, a heavily damaged cavea was represented.

As suggested previously [50], the sensitivity coefficients were calculated in order to evaluate the uncertainty propagation. This evaluation was conducted considering the average simulation results of the 500 Hz and 1 kHz octave bands [2]. The variability of each simulated receiver was calculated, and no systematic effects were detected. Thus, the sensitivity coefficients were calculated considering the normalized values, with respect to the relevant average value. An appropriate mathematical model, based on linear regression, was defined so as to relate the simulated values of each acoustical parameter to the absorption and scattering coefficients [50,51]. The expanded uncertainty was obtained as 2σ, where σ is the standard deviation of the model [50].

The expanded uncertainties for the O and C simulation software (U_O and U_C) are shown in

Table 3. Just Noticeable Difference (JND) of the T₂₀, C₈₀, and G acoustical parameters, the expanded uncertainty due to the variability of the input values of α_w and s for the simulation software O and C (U_O and U_C). Values higher than the JNDs are reported in bold.

Acoustical Parameter	JND	UO	UC
T20 (s)	5% ≈ 0.03	0.01	0.05
C80 (dB)	1	0.50	1.20
G (dB)	1	0.01	0.30

. The uncertainty, due to the input variability of α_w and s, is lower than the JND for all the parameters, except for T₂₀ and C₈₀ when the C software is used. The lower uncertainty values are due to the software algorithm, which is less sensitive to variations in α_w and s.

5. Discussion

This work aimed at providing an overview of the many methodological challenges that should be faced when dealing with the acoustics of open-air ancient theatres, both in the case of measured (i.e., for the acoustical characterisation of the current state) and predicted (i.e., for the simulation of a no longer/not yet existing state) room acoustics parameters. Measurement and simulation are strictly interconnected, also considering that the former is often required to validate the latter; the rationale for addressing both these aspects within the framework of this paper is that this is particularly true for open-air ancient theatres. Indeed, measurements of such unroofed spaces have been shown to be problematic with the application of current standards. Achieving reliable acoustical measurements is important in order to provide calibration data for the simulation software. In the context of cultural heritage research, and specifically for archaeological or historical acoustics, simulation becomes crucial because of the need to investigate (in most of cases) physical conditions which no longer exist (acoustics of the past), due to, among other aspects, the deterioration of the architectural elements. For these reasons, while measurements and simulations are concerned with different uncertainty issues, it was decided to compare the measured and calculated parameters (Section 5.1), as well as discussing the overall limitations of the considered protocols (Section 5.2).

5.1. Comparison of the Measured and Simulated Results

The aim of acoustical simulations is to obtain predictions that would closely match measured data. A well-calibrated model should minimise the perceivable differences between simulation and measurements for any considered acoustic parameter.

The subsequent considerations were also based on the α_w and s values of the cavea surface and its variations. The differences between the measured and simulated results are shown in Figure 4, which reports the acoustical behaviour during the calibration of both kinds of software, considering the variations due to the 20 alternative combinations (5 scattering coefficients × 4 absorption coefficients), for all the receivers, and the average between the 500 Hz and 1 kHz octave-bands. The isolevel curves shown in Figure 4 have been obtained by a two-dimensional data interpolation using the MATLAB function “interp2” with the “spline” method active. This method was chosen in order to have smooth first and second derivatives throughout the curves. Figure 4a,b, which pertain to O and C, respectively, refer to parameter T₂₀, while Figure 4c,d refer to C₈₀ and Figure 4e,f to G. The light yellow colour in the graphs shows the α_w and s combinations for which the simulated values were closest to the measured ones. These isolevel curves were based on SAD, i.e., the Sum of the Absolute Differences between the simulated values, s_n, and the measured ones, m_n, for each receiver position, expressed as follows by Equation (7) [52]:

$$\mathrm{SAD}= {{\displaystyle \sum }}_1^{\mathrm{n}}\left|{\mathrm{s}}_{\mathrm{n}}-{\mathrm{m}}_{\mathrm{n}}\right|$$

(7)

The results show that, depending on which parameter is considered, the best agreement between the simulated and measured values could not be obtained for the same combination of α_w and s. From the isolevel curves layout it is observable that, apart from T₂₀, Odeon software is more sensitive to variations of α_w than of s, while the opposite occurs for CATT-Acoustic. For T₂₀, lower differences between the simulated and measured values are detectable for both high and low absorption and scattering values in the case of Odeon software, while mainly for high scattering values over the whole range of absorption values in the case of CATT-Acoustic. For C₈₀, a good matching between measured and simulated values occurs with high absorption values over the whole range of scattering coefficients in the case of Odeon software, while it occurs with low scattering coefficients over the whole range of absorption coefficients in the case of CATT-Acoustic. For G, the best matching between measured and simulated values occurs with low absorption values over the whole range of scattering coefficients in the case of Odeon software, while it occurs with a medium scattering coefficient over the whole range of absorption coefficients in the case of CATT-Acoustic. Only in the case of G do both kinds of software show an agreement that is obtained in a range around the values of α_w = 0.10 and s = 0.55. Thus, this combination was considered for the calibration of the model.

Table 4. Mean values of the measurements and simulations for the T₂₀, C₈₀, and G acoustical parameters for the source in position S1. The data refer to the averages of the 500 Hz and 1 kHz octave bands and to the repetitions for the same receiver position. Spatial means refer to receivers on the same row.

Acoustical Parameters
Row	Receiver	T20 (s)			C80 (dB)			G (dB)
		Measured	Pred. (Odeon)	Pred. (Catt)	Measured	Pred. (Odeon)	Pred. (Catt)	Measured	Pred. (Odeon)	Pred. (Catt)
First row	R3	0.58	0.66	0.68	20.8	17.0	15.1	−0.3	−1.7	−1.0
	R6	0.70	0.66	0.62	16.9	17.0	15.2	−0.9	−2.3	−2.6
	R9	0.31	0.70	0.64	22.1	15.4	15.3	−2.4	−1.8	−1.4
spatial mean		0.53	0.67	0.65	19.9	16.5	15.2	−1.2	−1.9	−1.7
Second row	R2	0.81	0.92	0.97	16.5	14.6	14.5	−4.8	−6.7	−6.4
	R5	0.66	0.93	0.92	19.2	15.3	14.6	−5.4	−7.1	−6.7
	R8	0.71	0.90	0.77	15.9	14.5	15.4	−6.0	−7.0	−7.0
spatial mean		0.73	0.92	0.89	17.2	14.8	14.8	−5.4	−6.9	−6.7
Third row	R1	0.98	1.10	1.00	15.4	14.0	13.0	−7.8	−9.0	−9.0
	R4	0.94	1.03	1.05	16.9	13.9	12.5	−8.0	−9.8	−9.6
	R7	1.04	0.99	0.90	15.7	13.8	14.2	−9.3	−9.8	−9.7
spatial mean		0.99	1.04	0.98	16.0	13.9	13.2	−8.4	−9.5	−9.4
	R10	1.31	1.11	0.89	14.3	14.5	15.2	−10.2	−11.4	−10.6

shows all the simulation results of the calibrated model of SR, expressed as T₂₀, C₈₀, and the G acoustical parameters, considering both O and C. All the values are averaged over the central 500 Hz and 1 kHz octave-band frequencies and spatial values have been added for each row. In this way, the results can be compared directly with those of the corresponding measurements. A good agreement has been shown between the results obtained with the two different types of software, as can be also seen from the graph in Figure 5, where the average G for each row is represented along the average distance from the source, in the cases of measurements and simulations with Odeon and CATT-Acoustic.

In particular, the average values for each row obtained from the two software are always within or at the limit of the JND for each parameter, except C₈₀ in the first row. The differences between the simulated and measured results, in terms of average values per each row, are within two to seven times the JND for T₂₀, without any systematic behaviour related to the row. In the case of C₈₀, the differences from simulated and measured values are higher for the first row, with average simulated values that are three and five times the JND with Odeon and Catt-Acoustic, respectively, within 2 JND for the second and third rows, and within the JND for the last row, for both the software. For G, the average simulated values for each row are always within or quite close the JND compared to measured values for both the software, with a slightly worse behaviour for Odeon. Figure 5 shows as both the software correctly simulated the reduction of G with the distance from the source, with slopes in dB per distance doubling (dB/dd) that are 6.6 dB/dd and 6.3 dB/dd, for Odeon and Catt-Acoustic, respectively, compared to 6.3 dB/dd for the measurements.

5.2. Limitations of the Study

Given the complexity of the task, there are, of course, a number of limitations in the methodological approach implemented in the present study. Most of such shortcomings are related, as previously mentioned, to the actual applicability of the ISO 3382-1, intended for roofed performance spaces, to open-air environments.

Certainly, Section 7 of the ISO 3382-1 deals with the “Measurement uncertainty” and specifies that for practical evaluation of the measurement uncertainty of reverberation time using the integrated impulse response method, it can be considered as being of the same order of magnitude as that using an average of n = 10 measurements in each position with the interrupted noise method. No additional averaging is necessary to increase the statistical measurement accuracy for each position. However, considering the variability due to the atmospheric conditions, more than one repetition is needed. On the other side, anyone who has performed measurements in ancient open-air theatres knows that a large number of repetitions is rarely feasible, for a number of practical reasons due to the stability of the boundary conditions; thus, the scope of this study was to assess the protocols’ reliability with fewer measurements.

Table 5. ISO 3382-1 recommendations and their applications in the measurement campaign (X).

ISO 3382-1 Section	Recommendation	Implemented	Notes
4. Measurement conditions	Temperature and Relative Humidity: these quantities should be measured with an accuracy of ±1 °C and 5%, respectively.	X
	Equipment: omnidirectional sources and receivers. Maximum deviations of directivity for an omnidirectional source are indicated.	X	The deviation of directivity of the used sound source respected the maximum values indicated by the standards [30,54,55].
	Number of source positions: minimum 2, located where the natural sound source would take position. Height of sources: 1.5 m.	X
	Number of microphone positions: Microphone positions should be at positions representative of positions where listeners would normally be located. For reverberation time measurements, it is important that the measurement positions sample the entire space; for the room acoustic parameters, they should also be selected to provide information on possible systematic variations with position in the room. Height of the receivers: 1.2 m.	X
5. Measurement procedures	Integrated Impulse Response method: any source is allowed provided that its spectrum is broad enough to cover from 125 Hz to 4 kHz. The peak sound pressure level has to ensure a decay curve starting at least 35 dB above the BNL.	X	In some receiving positions, the 125 Hz frequency band did not guarantee the required 35 dB over the BNL, with the firecrackers.
	Time averaging: it is necessary to verify that the averaging process does not alter the measured impulse responses.
6. Decay curves	Regression analysis: a least-squares fit line shall be computed for the decay curve. If the curves are wavy or bent, this may indicate a mixture of modes with different reverberation times and thus the result may be unreliable.		The open-air condition is characterised by a cliff-decay curve [54] linked to a few strong reflections, but this case is not considered by the standard.

summarises the most salient aspects and recommendations provided in the different sections of the ISO 3382-1, confirms on whether such requirements were met and reports briefly on each circumstance (“notes” column).

Moreover, another limitation of the work derives from the use of GA software. The differences between simulations and measurements are mainly related to the approximations of GA with respect to the real wave effects, which result to be important for an open-air environment where the number of surfaces is limited and the generation of a diffuse field becomes critical. The GA principals are valid above the Schroeder frequency, which is not easy to estimate for an ancient theatre. The limits of GA are related to large rooms, low absorption coefficients, and broadband signals [48]. Furthermore, they neglect phase. As shown in different Round Robin tests [16,17], the GA based software differ between each other even when the same input data of absorption coefficients are given to the surfaces. Therefore, the major drawback, for the state of the art modelling software, is that the different simulation tools require different input data [53]. In practice, the absorption and scattering coefficient values are calibrated, i.e., varied within the range of their measurement uncertainty, in order to match the simulation results to the measured values. This may result in different values of these coefficients for the different software.

6. Concluding Remarks

This work deals with the accuracy of acoustical measurements and prediction models related to the ancient open-air theatre of Syracusae. Measurements based on ISO 3382-1 were conducted in unoccupied conditions. Firecrackers were used, because of the relatively high background noise level. The acoustical parameters described in the ISO 3382-1 standard, that is, Reverberation Time (T₂₀), Clarity (C₈₀), and Sound Strength (G), were obtained from the IRs measured at each receiver position. The uncertainty contributions due to the input values of sound absorption and scattering coefficients, α_w and s, have also been calculated with two simulation tools, that is, Odeon, version 13.02, and CATT-Acoustic, version 9. The models have been calibrated on the basis of the best match between the simulated and measured parameter values. Other sources of uncertainties, that is, the run-to-run variations and number of rays, have also been analysed and the obtained results have all been found to be under or at commonly accepted limit values of the Just Noticeable Differences (JNDs). The variability of the results is related to the algorithms used to approximate the acoustic phenomenon of the absorption and scattering. This kind of software are based on geometric acoustic principles, which rely on a statistical approach used to include diffuse sound scattering and predict the reverberant tail of an impulse response [22,23].

The following main results have been found from the uncertainty analysis that was conducted on the simulations of the Syracusae theatre:

• The uncertainty, due to the input variability of α_w and s, is lower than the JND for T₂₀ and C₈₀, when the Odeon software is considered, and for G when both types of software are considered;
• Apart from T₂₀, Odeon software is more sensitive to variation of sound absorption than of sound scattering, while the opposite occurs for CATT-Acoustics;
• Comparable behaviour of the simulated values of G has been shown for both types of software; G has been found to be the most suitable parameter for the calibration of the open-air theatre model;
• A good agreement with the measured values has been found, at the limit of the JNDs, in the calibrated model for all the parameters, in spite of the limitation of the GA software that has emerged in this case study, for both types of software.

Future studies will be conducted on a larger number of case studies, considering the influence of the architectural state of conservation, completeness, and dimensions on the acoustic field. Moreover, more suitable parameters for the acoustical characterisation of the open-air theatres than those described in ISO 3382-1 standard are the subject of continuous research [49].