Next Article in Journal
Arbuscular Mycorrhizal Fungus Rhizophagus irregularis Influences Artemisia annua Plant Parameters and Artemisinin Content under Different Soil Types and Cultivation Methods
Next Article in Special Issue
Low-Fat and High-Quality Fermented Sausages
Previous Article in Journal
Amplicon-Based High-Throughput Sequencing Method Capable of Species-Level Identification of Coagulase-Negative Staphylococci in Diverse Communities
Previous Article in Special Issue
Selection and Evaluation of Staphylococcus xylosus as a Biocontrol Agent against Toxigenic Moulds in a Dry-Cured Ham Model System
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Microorganisms
Volume 8
Issue 6
10.3390/microorganisms8060898
microorganisms-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Listeria monocytogenes Survey in Cubed Cooked Ham Packaged in Modified Atmosphere and Bioprotective Effect of Selected Lactic Acid Bacteria

by
Lucilla Iacumin
,
Giorgia Cappellari
,
Andrea Colautti
and
Giuseppe Comi
*
Dipartimento di Scienze AgroAlimentari, Ambientali e Animali, Università degli Studi di Udine, Via Sondrio 2/a, 33100 Udine, Italy
*
Author to whom correspondence should be addressed.
Microorganisms 2020, 8(6), 898; https://doi.org/10.3390/microorganisms8060898
Submission received: 1 May 2020 / Revised: 9 June 2020 / Accepted: 11 June 2020 / Published: 15 June 2020
(This article belongs to the Special Issue Bioprotection in Meat and Meat Products)
Browse Figures
Versions Notes

Abstract

:
The aim of this work was to study the presence of Listeria monocytogenes, as well as the potential activity of two bioprotective cultures (Lyocarni BOX-74 and Lyocarni BOX-57), versus a mix of three L. monocytogenes strains that were intentionally inoculated in cooked cubed ham, packaged in Modified Atmosphere Packaging and stored at different temperatures. The bioprotective cultures limit L. monocytogenes growth in cubed cooked ham stored either at 4 °C for 60 days and at 4 °C for 20 days and at 8 °C for 40 days. The inhibition at 8 °C is particularly useful for industrial cooked meat products, considering there are often thermal abuse conditions (8 °C) in the supermarkets. Both the starters can eliminate L. monocytogenes risk and maintain the products safe, despite the thermal abuse conditions. In addition, both culture starters grew without producing perceptible sensory variations in the samples, as demonstrated by the panel of the untrained tasters. The bioprotective LAB produced neither off-odours and off-flavours, nor white/viscous patinas, slime, discoloration or browning. Therefore, according to the obtained data, and despite the fact that cooked cubed ham did not show pH ≤ 4.4 or aw ≤ 0.92, or pH ≤ 5.0 and aw ≤ 0.94, as cited in the EC Regulation 2073/2005. It can be scientifically stated that cubes of cooked ham with the addition of bioprotective starters cultures do not constitute a favourable substrate for L. monocytogenes growth. Consequently, these products can easily fall into category 1.3 (ready-to-eat foods that are not favourable to L. monocytogenes growth, other than those for infants and for special medical purposes), in which a maximum concentration of L. monocytogenes of 100 CFU g−1 is allowed.

1. Introduction

Listeria monocytogenes can cause fatal disease (30–40%) in foetuses, infants, pregnant women, elderly subjects and immunocompromised individuals with cancer, kidney disease, heart disease or AIDS; subject to organ transplants; and/or treated with immunosuppressants [1,2,3]. The incidence of the disease caused by L. monocytogenes, which is named listeriosis, is decreasing and varies every year. In the US, where control is more robust, an annual incidence of 0.7 cases/100,000 inhabitants, with a mortality rate greater than 40%, is estimated [1]. Listeriosis is also widespread in Europe, and although there are slight variations, an incidence of 0.48 cases/100,000 inhabitants is estimated, with a mortality rate of approximately 13.7% [4,5].
L. monocytogenes is a microorganism of environmental origin and is isolated from many foods, such as milk and dairy products, fresh and processed meats, fresh and processed (smoked) fish products, vegetables and fruit [4,5,6,7,8,9,10]. By means of biofilm production, L. monocytogenes can also contaminate inert surfaces, such as stainless steel, polyethylene and rubber [11,12,13,14,15]. In particular, damp and cold surfaces can provide favourable conditions for the survival, and even the growth of L. monocytogenes [14,15]. In fact, L. monocytogenes grows without any problem in raw or processed food, because does not have great nutrition or oxygen demands and grows well at 4 °C. In fact, it is well-known that, despite its initial level of contamination, L. monocytogenes can quickly reach high concentration levels in some refrigerated ready-to-eat (RTE) products [1,2,16]. The growth of L. monocytogenes can be particularly dynamic in foods stored at extreme temperatures, which often occur in both the food trade and domestic refrigerators [17,18]. Among RTE products, cooked ham can be contaminated with L. monocytogenes [19,20,21], although cooking and subsequent pasteurisation after final packaging eliminate its presence [19,20,21]. However, L. monocytogenes can recontaminate cooked ham during slicing and dicing to obtain cubes used in culinary recipes. To eliminate this recontamination, post-processing technologies are often performed in cleanrooms, where the environment is thoroughly sanitised and the air is filtered, which consequently greatly reduce the spread of L. monocytogenes through atmospheric dust. Moreover, slicer equipment is cleaned and disinfected, while workers are trained and wear additional sterile coveralls over their work clothes. Therefore, these measures allow for reduction in contamination. Despite the above-mentioned preventive systems, L. monocytogenes is often isolated either in pre-sliced or in cooked cubed ham, and the cause of the recontamination is facilitated by the contact surfaces of the machinery or from the air [19,20,21]. Nonetheless, recontamination cannot occur from the cooked ham bars used for slicing or dicing, because they are cooked in moulds, packaged and pasteurised. All Italian factories use the following protocol: brined pork meat is stuffed into plastic bags, placed in mould-like bars and cooked at up to 70–73 °C (the cooking process lasts for 7 h; 1 h per kg product); then, after cooling at 4 °C, the product is stored for 4–5 days at 4 ± 2 °C, pasteurised at 85 °C for 15 min, cooled, subjected to dicing, and packed in Modified Atmosphere Packaging (MAP) (100 g product each). Therefore, contamination can only take place in dicing machines or slicers. Moreover, this equipment has often been associated with L. monocytogenes contamination of retail RTE meat products [20,21,22,23,24,25,26,27]. L. monocytogenes was isolated by swabs from 13% of slicer blades used in different butchers [28], and from 10% of delicatessen knives and supermarket slicers [10,22,29,30,31,32]. Contamination occurs from the slicer to the product, and vice versa [22,31,33], thereby causing cross-contamination. In this regard, thorough cleaning and disinfection protocols can prevent the formation of L. monocytogenes biofilms, which usually grow over time, through continuous and prolonged cross-contamination. It is known that the presence of this pathogen on both products and contact surfaces [34], the composition of the food [31,35,36,37], the cutting force and the speed of the blade [38], and other factors, such as temperature [22], surface topography [31] and contact time [30,35,39], can contribute to the recontamination of cooked charcuterie products. However, the level of this accidental contamination is usually low, at less than 10 CFU g−1 or cm−2 [18,22,31,33]. Despite this low initial concentration, sliced or diced cooked ham can represent a risk for consumers, because L. monocytogenes can reach high concentrations during their shelf lives at 4 °C of 30 and 60 days, respectively. In particular, in the case of thermal abuse (6–8 °C) during storage at the supermarket, L. monocytogenes may grow and reach dangerous concentrations for the consumer, considering that slices of cooked ham are eaten as is, and that cubes are not always cooked before consumption. Recently, to avoid the presence or growth of L. monocytogenes in meat products, in addition to the application of good production hygiene standards, bioprotective cultures or natural preservatives, such as plant or animal (chitosan and propolis) and microbial extracts or metabolites (lysozyme, nisin and other bacteriocins), have been used [40,41,42,43,44]. Since the use of plant and animal extracts can often change the smell, taste and texture of food [45], there has been an increase in the direct use of bacteriocins with antimicrobial effects or bioprotective cultures, because of their antagonistic effect and production of organic acids or bacteriocins against L. monocytogenes [46,47]. These bacteriocin inhibition effects are likely caused by different actions, such as competition for nutrients, as well as organic acid and bacteriocin production, in addition to the “hurdle” parameters. Strains of Lactobacillus sakei, L. casei, L. brevis, L. curvatus, L. plantarum and Carnobacterium spp. isolated from meat products frequently produce bacteriocins or bacteriocin-like compounds [48,49,50,51,52,53]. In particular, these strains have good antilisterial effects, and are therefore used as bioprotective cultures in European meat products [48,49,50,51,52,53].
Currently, evidence on the use of bioprotective cultures against the presence and growth of L. monocytogenes in cooked cubed ham remains scarce. Therefore, the aim of this work was to evaluate the use of two selected commercial bioprotective starter cultures, to eliminate or prevent the development of L. monocytogenes intentionally added to cooked cubed ham stored for 60 days (end shelf life) at 4 °C, and under thermal abuse conditions (8 °C).

2. Materials and Methods

2.1. L. monocytogenes Monitoring in Cooked Cubed Ham Packages

One hundred and eighty (180) samples of cooked cubed ham, placed in Modified Atmosphere Packaging (MAP) of 100 g of product produced by 3 Italian factories (60 each), were collected from local supermarkets and analysed in an epidemiological study to evaluate the frequency of the presence of L. monocytogenes.
Samples belonged to 6 different lots of production. Each lot included 10 samples, 5 of which were analysed immediately after purchase, whereas the remaining 5 were stored at 4 ± 2 °C and analysed at the end of their shelf life (60 days). L. monocytogenes was determined using the ISO 11290-1 method [54]. The positive samples were also evaluated by the ISO 11290-2 method [54].

2.2. Cooked Cubed Ham Package Preparation for a Bioprotective Study

Cooked ham was produced by one of the main Italian factories. The production steps included the use of various trimmings of the pork leg (75%) and saline (water, natural spices and their extracts, dextrose, sodium ascorbate, carrageenan, sodium chloride and sodium nitrite-E250, the concentrations of which remained confidential). The brined pork meat was stuffed into plastic bags, which represented the final package, and was then cooked up to 73 °C (the cooking process lasted for 7 h; 1 h per kg product). Then, after cooling at 4 °C, the product was stored for 4 days at 4 ± 2 °C, pasteurised at 85 °C for 15 min and, after cooling, unpacked and subjected to dicing, to prepare the samples for the trials.

2.3. Preparation of the Inoculum of Listeria monocytogenes and Bioprotective Starter Cultures

The inoculum consisted of 3 strains of different Listeria monocytogenes strains: L. monocytogenes NCTC 10887 (serotype 1/2b), L. monocytogenes 9Di4A (serotype 4b) isolated from the meat matrix and L. monocytogenes 11Di4A, which was isolated from human samples, and is responsible for invasive listeriosis. Four millilitres of an overnight culture in Brain Heart Infusion broth (Oxoid, Italy) of each strain as subjected to centrifugation at 13.400 rpm for 10 min at room temperature, after which, the pellets were resuspended in saline-peptone water (Peptone, 1 g, Oxoid, Italy; NaCl, 30 g, Sigma-Aldrich, Italy; distilled H2O, 1000 mL; Aw 0.97). Then, the suspensions were standardised at an optical density at 600 nm [OD600] of 0.1, and their microbial counts were evaluated. In particular, serial dilutions were carried out in sterile saline-peptone water, and 0.1 mL of each dilution was spread in Petri dishes containing Palcam Agar Base (Oxoid, Italy). Incubation was performed at 37 °C for 48 h, and the colonies were counted. Each suspension contained approximately 7 log CFU mL−1. At this step, 1 mL of each standardised suspension was combined, and inoculation was performed by spraying a final concentration of 2 log CFU g−1 on the cooked cubed ham packages.
Two different bioprotective cultures consisting of freeze-dried lactic acid bacteria were used: Lyocarni BOX-74 and Lyocarni BOX-57 (Sacco srl, Via Alessandro Manzoni 29/A, 22071 Cadorago, CO, Italy). Lyocarni BOX-74 contained Carnobacterium divergens, Carnobacterium maltaromaticum and Lactobacillus sakei, whereas Lyocarni BOX-57 contained the Carnobacterium divergens, Carnobacterium maltaromaticum and Lactobacillus sakei bacteriocin producers). Both of the cultures were chosen because of their fast growth at 4 °C, and because they are selected and commercially sold, due to their ability to prevent spoilage and pathogenic microorganisms in meat products. Before usage, the effective concentration was checked. Ten (10) grams of freeze-dried cultures were diluted 1:10 in sterile peptone water (NaCl 2%; Aw 0.98) and homogenised. After setting up the decimal dilution in peptone water, counts were performed in deMan Rogosa Sharpe medium (MRS, Oxoid, Italy) using a double layer method. Plates were incubated at 37 °C for 48–72 h, and the colonies were counted. The concentration of both bioprotective starters cultures was 11 log CFU g−1. After dilution, each starter was inoculated by spraying the cooked cubed ham with a final concentration of 5 log CFU g−1.

2.4. Inoculation Design

In parallel, a total of 6 trials were set up as described below:
(A)
Control trial (not inoculated, NI);
(B)
Samples inoculated with the Sacco BOX-74 bioprotective starter culture and L. monocytogenes (BOX74 + LM);
(C)
Samples inoculated with only the Sacco BOX-74 bioprotective starter culture (BOX74);
(D)
Samples inoculated with only L. monocytogenes (LM);
(E)
Samples inoculated with the Sacco BOX-57 bioprotective starter culture and L. monocytogenes (BOX57 + LM);
(F)
Samples inoculated with only the Sacco BOX-57 bioprotective starter culture (BOX57).
For each trial, 33 samples, each of which consisted of 100 g of cooked cubed ham (prepared as described in Section 2.1), were prepared and inoculated following the scheme described above.
Then, the cooked cubed ham (100 g each) was packaged using trays with top (PET/PE/EVOH/PE) and bottom films (PVC/EVOH/PE) in Modified Atmosphere Packaging (MAP), consisting of N2 (55%) and CO2 (45%). The inoculated samples were left for 2 h at room temperature to favour adhesion of the microorganism to the cubes. Then, two different storage temperatures were tested: 4 °C for the entire shelf life (60 days), 4 °C for the first 20 days, and 8 °C for the remaining shelf life (thermal abuse, 40 days). Analyses were performed on three biological replicates at 0, 10, 20, 30, 40, 50 and 60 days. In addition, duplicate technical replicates were performed for each of the 3 biological replicates per trial stored at both temperatures.

2.5. Microbiological Analysis

Each sample (100 g) was completely diluted with saline-peptone water (peptone, 1 g; NaCl, 7 g; distilled H2O, 1000 mL) at a 1:1 ratio in Stomacher bags. After homogenisation for 2 min in Stomacher bags (PBI, Italy), decimal dilutions were prepared. Lactic acid bacteria (LAB) were counted by inoculation of 1 mL of each serial dilution in MRS medium (Oxoid, Italy) using the double layer technique. The plates were incubated at 37 °C for 2 days, after which the grown colonies included in the agar medium were counted. The total bacterial count (CBT) was monitored by plating 0.1 mL of each serial dilution on Plate Count Agar (Oxoid, Italy), followed by incubation at 25 °C for 2 days. Listeria spp. and L. monocytogenes were determined using the ISO 11290-2 method [54].

2.6. Physicochemical Analysis

Samples from trials NI, BOX74 and BOX57 were also subjected to physicochemical analysis as follows: (a) determination of water activity (aw), carried out at each sampling point by the use of an AquaLab device (Decagon, USA), according to the manufacturer’s instructions, and after appropriate calibration; (b) pH determination using a glass electrode mounted on a pH meter (Crison Basic 20, Italy); and (c) colour determination using the Minolta Chromameter CR-200 and CIE Lab system after calibration. Ten different positions on the surface of each sample were immediately evaluated after opening the tray. In particular, parameters a*, b*, L*, and ΔE were evaluated [55]. Moisture, proteins, fat, sugar and ash were determined by AOAC [56].

2.7. Sensory Analysis

Ten additional samples of the (a) control trial (not inoculated, NI); (b) samples inoculated with only the Sacco BOX-74 bioprotective starter culture (BOX74); and (c) samples inoculated with only the Sacco BOX-57 bioprotective starter culture (BOX57) were prepared for sensory analysis. The analysis was performed by 12 untrained panellists, representing real consumers. The samples were evaluated by tasters, who were asked to identify the products in ascending order from worst to best, keeping in mind the following parameters established by Baublis et al. [57] and Vàlcovà et al. [58]: smell (fermented, rancid), flavour (sweet, salty, fresh pungent, meat and rancid) and aroma (ammoniacal, sweet, salty and bitter). Panel members were trained according to the consensus method, in six to eight sessions of at least 90 min each. During each session, subjects were trained, using a roundtable discussion, to achieve (a) lexicon development, (b) training in intensity scaling and (c) reference association with aroma, flavour and mouthfeel attributes. The scaling of attributes on unstructured line scales was practiced, to calibrate the panel until a consensus was reached amongst the panellists.

2.8. Statistical Analysis

Data were subjected to two analyses. In the first, the differences between means within storage day were tested by one-way ANOVA, where the experimental group was considered as fixed factor. In the second analysis, the differences between means within experimental group were tested by one-way ANOVA, where storage day was considered as fixed factor. For both analyses, Tukey’s test was used as post-hoc test (p < 0.05).

3. Results

3.1. Epidemiology of L. monocytogenes in Cooked Cubed Ham Packages

The physicochemical characteristics demonstrated the high nutritional value of cooked cubed ham. The results represent the means of all the samples. As shown in Table 1, the samples were rich in proteins, and included low concentrations of fat and sugars. No significant differences were observed among the lots of production and brands, thereby demonstrating the use of the same processes and production technology. These characteristics make the product excellent for human health and diet. The study demonstrated the presence of Listeria monocytogenes in cooked cubed ham (Table 2). Listeria monocytogenes was isolated in eight out of the 180 investigated samples (4.44%), as well as in some samples analysed at either 0 or 60 days. At time 0, the pathogen was only detected in two samples after using the enrichment method. At 60 days, L. monocytogenes was detected in five samples by enrichment, and in one sample by direct dilution at a concentration of 2.1 log CFU/g. The observed values exceeded the limit proposed by Reg. CE 2073/05 [59]. The results demonstrated that L. monocytogenes is widespread and contaminates food randomly. The distribution of Listeria monocytogenes is not homogeneous, and depends on the contaminated environment, considering that the cooked ham bars were pasteurised at 85 °C for 5 min, cooled, unpackaged and then diced. Finally, contamination was found in each lot from the factories.

3.2. Physicochemical Characteristics of Cooked Cubed Ham

The cubed cooked hams were produced in one of the main Italian factories, but the technology is largely used in Italy. The aw value remained within the range of 0.986 ± 0.004 and 0.994 ± 0.003 for the entire monitored period in all trials (Table 3). No significant differences were observed in the samples stored at 4 °C (p > 0.05). Conversely, a significant difference was observed among the aw values of the samples stored for 20 days at 4 °C, and then at 8 °C for the remaining 40 days (p < 0.05). In particular, the significance is most evident at 0, 30 and 50 days. Comparing the aw at level of both temperatures and days storage significative differences were observed at day 30, 40 and 50. In fact, this difference cannot be considered a variation due to the storage temperature, but instead, is an intrinsic variation of the analysed samples, which were different at each analysed time. Moreover, at 50 and 60 days, the aw values were similar to those assessed at 0, 10, 20 and 30 days.
The trends of the pH of the inoculated and non-inoculated samples of cooked cubed ham are shown in Table 4. At 30 days after production, a decrease in pH can be seen in both the inoculated and uninoculated samples. This decrease is related to the metabolism of LAB, which are considered to be mainly responsible for acidification. The presence of the bioprotective starters produced a significant decrease in pH values, particularly in products stored first at 4 °C, and then at 8 °C. In fact, the observed variation was particularly significant (p < 0.05) in samples inoculated with the bioprotective cultures, as well as both in samples stored at 4 °C for the entire period, and those stored for 20 days at 4 °C, and then 40 days at 8 °C. In the uninoculated samples, a significant pH variation was observed at 30 and 40 days in samples stored at 4 °C, as well as at 30 days in samples stored at 4 °C and then at 8 °C. However, it is possible that the pH variation depends not only on the growth of the bioprotective starter LAB, but also on the samples analysed, which were different at all times. Comparing the pH fate at level of each temperature and each treatment (Table 4), significant differences were observed after 30 days storage (p < 0.05). In particular the difference was higher at level of 20 days at 4 °C, and then 40 days at 8 °C storage. It seems that BOX-57 starter produced a higher pH decrease than BOX-74 starter (p < 0.05) in samples stored over 30 days at 4 °C, and then 40 days at 8 °C (p < 0.05). However, it is possible that the pH variation depended not only on the growth of the bioprotective starter LAB, but also on the samples analysed, which were different at all times.
Considering the colour changes, there was no significant difference over time among all the investigated samples (Table 5). In fact, the a*, b* and L* values showed wide standard deviations, which influenced the significance (p > 0.05), regardless of the storage temperatures and the analysed times. The lack of an important change was due to the MAP system used, which included a low oxygen concentration (less than 0.2%), and, consequently, prevented any oxidation or browning (Figure 1). In addition, starter cultures, which are microaerophilic, produced a reducing potential that limited any oxidation. In any case, the colour variation was not visible to the naked eye (Figure 1), given that the ΔE value was lower than 2.0 [55].

3.3. Microbial Evolution and Interaction between Bioprotective Culture and L. monocytogenes

Autochthonous CBT and LAB grew over time and reached final values higher than 7 log CFU g−1, independent of the storage temperature (Figure 2 and Figure 3). The growth of both CBT and LAB increased in the samples stored under thermal abuse (4 °C and 8 °C) and exceeded 8 log UFC g−1 (Figure 3). These control cooked cubed hams were also naturally contaminated by L. innocua, which was not identified at 0, 10 and 20 days of storage, due to being present lower than the detection limit of 10 CFU g−1. Subsequently, during storage, L. innocua grew to a final concentration of 2.5 ± 0.5 log CFU g−1 in NI samples at 4 °C and even reached 7.1 ± 0.3 log CFU g−1 in NI samples stored at 4 and then 8 °C, despite the competition of autochthonous LAB (Figure 2 and Figure 3). These data suggest that the shelf life of the cooked cubed ham should not exceed 30–40 days. During a longer shelf life, in the case of accidental contamination, L. monocytogenes could grow and reach concentrations dangerous for consumer health.
Considering the results of the coinoculation of the BOX-74 bioprotective culture and Listeria monocytogenes (Figure 4), a significant growth of the bioprotective culture, and a clear inhibition of the inoculated L. monocytogenes can be observed.
Given the wide standard deviation observed, L. monocytogenes reduction was not statistically significant, (p > 0.05) within the same temperature, in respect to that at day 0, although a slightly decreasing trend can be observed over time (Table 6). Vice versa, when the bioprotective culture was not inoculated (LM trial), L. monocytogenes grew and reached final values higher than 8 log CFU g−1, thereby becoming a serious health risk for consumers, even though these products are usually cooked before consumption.
Therefore, the bioprotective culture was effective. L. monocytogenes strains were not completely eliminated, but their growth capability was inhibited. When the bioprotective culture BOX-74 was inoculated in the absence of L. monocytogenes, abundant growth was observed and counts close to 9.0 log CFU g−1 were observed (Figure 4). Almost the same counts were observed in the case of the co-inoculated samples (BOX74-LM), confirming that an inhibitory effect was shown, or rather, a small increment in the counts of the co-inoculated samples could be observed. The growth of the bioprotective culture, either alone or with L. monocytogenes inoculation, was also confirmed by the pH trend, which changed from 6.35 units (day 0) to 5.69 units at the end of shelf life (60 days). The CBT increased over time, and concentrations reached were slightly higher than those of the bioprotective culture. As a matter of fact, the CBT counts included the LAB counts in both the inoculated and non-inoculated samples (natural LAB contamination), which explained the particularly high value obtained. For confirmation, five colonies were isolated for each plate count of PCA, and the presumptive identification of LAB species was performed.
In the case of storage under thermal abuse conditions (from 4 to 8 °C), a similar trend was observed (Figure 5). A clear inhibition of the growth of L. monocytogenes was obtained, due to the presence of the BOX-74 bioprotective culture. The value of decrease was not significative and was similar to the one observed in the samples stored at 4 °C (p > 0.05, Table 6). BOX-74 inhibited L. monocytogenes growth, despite the higher temperature maintained for the last 40 days of storage. In the trials in which BOX-74 was not inoculated, L. monocytogenes reached final concentrations higher than 8.4 log CFU g−1. Therefore, the selected bioprotective culture BOX-74 was effective in preventing the development of 6 logs of L. monocytogenes, and resulting in a consistent bacteriostatic effect. Consequently the L. monocytogenes concentration remained < 2 log CFU g−1, as accepted by Reg. CE 2073/05 [59]. The LAB bioprotective culture, when inoculated in isolation, grew abundantly, and reached values of approximately 8.6 ± 0.1 log CFU g−1. These values are not significantly different from those observed in the samples with the BOX-74/L. monocytogenes co-inoculation. This finding confirms that the presence of L. monocytogenes did not affect the viability of the bioprotective culture. The active metabolism of the bioprotective culture was also demonstrated by the pH trend, which decreased to 5.4 units at 60 days of storage. This value is lower than that observed in samples stored at 4 °C, until the end of shelf life. Thermal abuse (8 °C) increased bioprotective culture activity. CBT increased over time, and reached concentrations similar to those observed for the bioprotective culture.
As far as the BOX-57 bioprotective culture is concerned, L. monocytogenes growth was inhibited and the level of inhibition was statistically significant (p < 0.05), beginning as early as 20 days after the co-inoculation. This inhibitory effect was visible in both trials at 4 °C, and also when the cooked cubed hams packages were stored under thermal abuse conditions (4 to 8 °C) (Figure 6 and Figure 7). This result confirms the higher effectiveness of BOX-57 than BOX-74. On the other hand, L. monocytogenes was able to grow and reached final concentrations higher than 8.7 log CFU g−1 at 50 days after inoculation in cooked cubed ham, without the addition of the bioprotective culture. Therefore, BOX-57 was also effective and prevented the growth of inoculated L. monocytogenes strains. When inoculated alone, BOX-57 grew and reached concentrations slightly higher than 8.0 Log CFU g−1 at 4 °C and even greater than 9.0 Log CFU g−1 at 4 to 8 °C at 30 days. These values were lower than those reached by the starter inoculated with L. monocytogenes. In fact, at 4 °C, the sample grew quickly, even within 40 days, reaching 9.0 Log CFU g−1. However, at 50 and 60 days, the concentrations slightly decreased. The development of the starter, either added as a single culture or in a mix with L. monocytogenes, was confirmed by the pH trend. CBT increased over time and reached concentrations similar to those of the LAB (Figure 6 and Figure 7).
The storage temperature of approximately 8 °C, which occurred from the 20th day until the end of shelf life, did not favour the development of L. monocytogenes in the presence of BOX-57. However, L. monocytogenes was not completely eliminated, although its growth was prevented.
Comparing the levels of L. monocytogenes reduction in samples treated with BOX-74 and BOX-57, it appears that BOX-57 is more effective. In particular, at 20, 30 and 60 days storage, BOX-57 produced a significative L. monocytogenes reduction respect to BOX-74 (p < 0.05) in samples stored 20 days at 4 °C and 40 days at 8 °C (Table 6). Conversely, at 4 °C, the significative difference using BOX-57 was observed only at 60 days storage (Table 6). Consequently, it seems that the best performances in L. monocytogenes reduction can be obtained by BOX-57 starter.

3.4. Sensorial Analysis of Cubed Cooked Ham Samples Treated or not with BOX-74 and BOX-57 Starter Cultures

Sensory analysis was carried out by a panel of non-professional panellists on the uninoculated control samples, as well as the samples inoculated with only BOX-74 and BOX-57. The starters did not profoundly change the sensorial characteristics of the product (Table 7; Figure 1). In fact, cooked cubed ham treated with bioprotective LAB presented neither typical odours and flavours of spoilage nor white/viscous patinas, slime, discoloration or browning. The lack of change was expected, because both starters were selected for their antagonist activity versus spoilage and pathogenic bacteria, without off-odour and off-flavour production. Thirty-one out of 33 control samples did not present any spoilage, with only two (6%) among the samples stored at 8 °C being blown, due to heterofermentative LAB growth. Conversely, none of the samples inoculated with the bioprotective cultures were swollen. The LAB cultures were homofermentative and prevented the development of heterofermentative contaminants by substrate competition. Therefore, starter addition can have a dual purpose, preventing the growth of either L. monocytogenes or spoilage bacteria, as represented by the heterofermentative LAB.
The panel did not identify any difference in the colour of the samples, regardless of the presence or absence of the starters or the storage temperatures. The presence of slime, discoloration or browning produced by the spoiler bacteria or by the indigenous LAB has not yet been highlighted (Table 7). The only difference between inoculated or non-inoculated samples was a change in pH; however, this change was not deemed significant by the panellists. The panel identified a slightly acidic, non-disturbing taste in the samples added with the starters, and among these, cooked cubed ham inoculated with the BOX-57 was the most appreciated.

4. Discussion

Cooked ham is a meat product that, at the end of production, is usually L. monocytogenes-free. The production technology allows for the cooking of brined meat in moulds and its pasteurisation or sterilisation after packaging in aluminium or plastic bags. These treatments eliminate the asporogenous pathogenic microorganisms that may be present in the meat or derived from the production environment after the pre-moulding of ham before final packaging [20,21]. Nevertheless, it is also a good ecosystem for microorganisms that can contaminate the product to make it unsafe. For this reason, the choice of a specific method of preservation that guarantees increasing attention for controlling the shelf life and safety of this new generation of minimally processed ready-to-eat food products of extended durability under refrigerated conditions is fundamental. Cooked cubed ham is produced from pork meat, which is brined, packaged under vacuum, cooked in an oven, pasteurised, and then diced and packaged in MAP. The heat treatments eliminate or reduce asporogenous microorganisms. However, during dicing, recontamination occurs and defines the spoilage flora, which mainly consist of LAB and are responsible for the decay of the shelf life of the product. In addition, pathogenic microorganisms, including L. monocytogenes, can re-contaminate the product. L. monocytogenes is a ubiquitous microorganism and can be found in various substrates, including meat and meat products. The level of contamination of L. monocytogenes is always limited and usually less than 1 CFU g−1 [20,21,60]. However, it is possible that during storage in the refrigerators of supermarkets, L. monocytogenes may develop and reach values capable of producing illness in the consumer. The growth of L. monocytogenes is favoured by the thermal abuse or long shelf lives to which these products are subjected (from 23 to 30 days for cold cuts and to 60 days for cooked cubed ham). Our study demonstrated that dicing permits L. monocytogenes to contaminate cooked cubed ham and that its presence in 25 g of product was found in eight out of 180 samples harvested from retail locations. In these specific samples, the concentration of L. monocytogenes was found to consistently be < 100 CFU g−1. Only in one out of the 180 samples, the concentration of L. monocytogenes was detected at a level of 2.1 Log CFU g−1 at 60 days of storage. This value demonstrates that L. monocytogenes can grow during a long storage period at 4 °C. In Europe, the presence of L. monocytogenes has been described in cooked meats [6,7,20,21,26,61]. L. monocytogenes was present in 1.65% of cooked ham and in 6.65% of cooked ham slices [26]. Data confirmed that slicing, even if performed in clean rooms with a high level of hygiene, produces contamination or recontamination and, consequently, poses a health risk. Cooked ham is a ready-to-eat (RTE) product, and is normally eaten without cooking; therefore, it can become dangerous to eat if it harbours L. monocytogenes [62]. Several studies have been performed to evaluate the survival or growth of this microorganism in RTE meat products [21,63,64,65,66]. Again, mathematical models were also used to describe the behaviour of L. monocytogenes in RTE meats, taking into account their intrinsic and extrinsic parameters, such as the pH; acidity; aw; salt ratio; presence of nitrite, polyphosphates, lactic acid and diacetate; packaging under vacuum or in a modified atmosphere; and storage temperature [67,68]. Data showed that L. monocytogenes can widely grow in cooked ham due to the pH value (>6.0), aw (>0.98), additives (nitrite, sugar, milk powder, proteins), brine and storage times and temperatures used.
Ingredients and additives cannot hinder the microorganism’s growth, and post-processing heat treatments after dicing or slicing cannot be applied. Consequently, to solve the L. monocytogenes problem, the use of others innovative technologies based on natural additives (e.g., essential oils) or bioprotective cultures are required [69].
Bioprotective selected starters represent a valid way to eliminate or prevent the development of L. monocytogenes in products that support its growth. The use of bioprotective cultures, represented by two types of cultures, including Lyocarni Sacco BOX-74 (Carnobacterium divergens, Carnobacterium maltaromaticum and Lactobacillus sakei) and Lyocarni BOX-57 (Carnobacterium divergens; Carnobacterium maltaromaticum and Lactobacillus sakei bacteriocin producer), demonstrated efficiency in inhibiting L. monocytogenes growth either at 4 °C, or at 4 °C for 20 days, and then at 8 °C for 40 days. The activity of these bioprotective cultures is mainly based on competition at the substrate level and also, in the case of BOX-57, bacteriocin production.
Microbial starters are usually added to raw meat to promote its ripening or shelf life, because they develop and predominate against pathogenic and spoiler microorganisms. In this work, the added cultures not only inhibited L. monocyogenes, but also prevented heterofermentative LAB growth, which was responsible for spoiling some cooked cubed ham by slimes and blown package production. The presence of autochthonous (natural) LAB, as demonstrated in the experiment, is not sufficient to inhibit L. monocytogenes. In the control samples, in which only L. monocytogenes was inoculated, a quick development of the pathogen was observed. Several authors demonstrated the effectiveness of selected cultures to avoid L. monocytogenes growth in cooked meats. Bredholt et al. [52,53] and Schobitz et al. [70] showed that the growth of L. monocytogenes can be inhibited by Lactobacillus sakei in vacuum packages of sliced cooked ham and by Carnobacterium piscicola in vacuum packed meats, respectively. Both strains were isolated by meat product. Vermeiren et al. [71] confirmed the importance of culture selection, and their research demonstrated that only 92% of the isolated LAB strains were able to inhibit L. monocytogenes growth. Amezquita and Brashears [61] demonstrated that selected Pediococcus acidilactici, Lactobacillus casei and Lactobacillus paracasei strains, isolated from meat based RTE products, showed bacteriostatic activity in cooked meats and bactericidal activity towards L. monocytogenes in frankfurters stored at 5 °C.
In this case, 2 Log CFU g−1 was chosen as the ideal L. monocytogenes concentration, and the inoculated samples were then packaged in MAP in plastic trays. This concentration was intentionally high to simulate a high level of recontamination during dicing and packaging. The bioprotective cultures, represented by the Carnobacterium divergens, C. maltaromaticum and Lactobacillus sake strains, were selected from meats and therefore used in these products. All the strains were able to develop at storage temperatures (4 °C) and compete both in vitro and in situ against L. monocytogenes. In particular, BOX-57 contains a Lactobacillus sakei strain known to be a bacteriocin producer.
It is well-known that LAB can produce bacteriocins, which are peptides or glycopeptides that contain structures that are closely related to the strain producer. Biochemical and molecular characteristics identify their type [69]. Several bacteriocins isolated from various food sources have been identified, studied and used in meat and meat products, including pediocins, lactocin 705, sonorensin, plantarocins and enterocines [44,69]. Bacteriocins are often purified from LAB and directly inoculated in raw or processed meats. However, direct inoculation of the bacteriocin producer LAB strain is preferred [69]. In recent years, combined techniques have been increasingly used. Bacteriocins are often associated with additives or other technologies (sodium nitrite, heat treatments, high pressures, etc.), which produce a synergistic effect [44,69]. Such combinations can solve many problems related to the presence of L. monocytogenes in cooked ham. Wu et al. [44] inhibited L. monocytogenes in cooked ham using physicochemical techniques combined with plantaricin.

5. Conclusions

The physicochemical parameters of the tested product (aw > 0.98 and pH > 5.0) were optimal for the growth of L. monocytogenes. To prevent the growth of L. monocytogenes, the use of bioprotective starters is recommended. In fact, the LAB bioprotective cultures grew throughout the storage period, independent of the storage temperature, leading to a significant decrease in the concentration of the pathogen. In all the samples with added bioprotective starter cultures, an increase of L. monocytogenes was observed, surpassing a 4 Log development that was conversely observed in the non-inoculated control samples. Moreover, the bioprotective cultures resulted in effective elimination of the risk of spoilage due to the growth of autochthonous heterofermentative LAB, as observed by the lack of swollen packs and/or slime production on the product. Therefore, according to the obtained data and despite the fact that cooked cubed ham did not show pH ≤ 4.4 or Aw ≤ 0.92, or pH ≤ 5.0 and Aw ≤ 0.94, as cited in the EC Regulation 2073/2005 [53], it can be scientifically stated that cubes of cooked ham with the addition of bioprotective starters cultures do not constitute a favourable substrate for L. monocytogenes growth. Consequently, these products can easily fall into category 1.3 (ready-to-eat foods that are not favourable to L. monocytogenes growth, other than those for infants and for special medical purposes), in which a maximum concentration of L. monocytogenes of 100 CFU g−1 is allowed. Finally, the use of the starters is also suggested because they do not change the sensorial quality of cooked cubed ham.

Author Contributions

All authors contributed equally to the planning, the testing, reporting findings and discussion of the work. All authors have read and agreed to the published version of the manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Acknowledgments

The authors gratefully acknowledge Fabio Dal Bello and Alessandro Pozzo of Sacco srl (Via Alessandro Manzoni 29/A, 22071 Cadorago, CO, Italy) for the donations of starters and Nadia Bortolussi and Matteo Gasparetto of Bechér S.p.a. (Paderno di Ponzano Veneto, Treviso, Italy) for cubed cooked ham production.

Conflicts of Interest

None of the authors of this paper has a financial or personal relationship with other people or organisations that could inappropriately influence or bias the content of the paper.

References

  1. Berger, S. Listeriosis: Global Status; GIDEON Informatics. Inc.: Los Angeles, CA, USA, 2016. [Google Scholar]
  2. Posfay-Barbe, K.M.; Wald, E.R. Listeriosis. Semin. Fetal Neonatal Med. 2009, 14, 228–233. [Google Scholar] [CrossRef]
  3. Szczawinński, J.; Szczawinńska, E.M.; Łobacz, A.; Tracz, M.; Jackowska-Tracz, A. Modelling the growth rate of Listeria monocytogenes in cooked ham stored at different temperatures. Vet. Res. 2017, 61, 45–51. [Google Scholar] [CrossRef] [Green Version]
  4. EFSA: European Food Safety Authority. Analysis of the baseline survey on the prevalence of Listeria monocytogenes in certain ready-to-eat (RTE) foods in the EU, 2010–2011 Part A: Listeria monocytogenes prevalence estimates. EFSA J. 2013, 11, 3241. [Google Scholar] [CrossRef]
  5. EFSA: European Food Safety Authority. The European Union Summary Report on Trends and Sources of Zoonoses, Zoonotic Agents and Food-borne Outbreaks in 2017. EFSA J. 2018, 16, 5500. [Google Scholar]
  6. Ahmed, O.; Pangloli, P.; Hwang, C.; Zivanovic, S.; Wu, T.; D’Souza, D.; Draughon, F.A. The occurrence of Listeria monocytogens in retail ready-to-eat meat and poultry products related to the levels of acetate and lactate in the products. Food Control 2015, 52, 43–48. [Google Scholar] [CrossRef]
  7. Angelidis, S.; Koutsoumanis, K. Prevalence and concentration of Listeria monocytogenes in sliced ready-to-eat meat products in the Hellenic retail market. J. Food Prot. 2006, 69, 938–942. [Google Scholar] [CrossRef] [PubMed]
  8. Dalgaard, P.; Jorgensen, L.V. Predicted and observed growth of Listeria monocytogenes in seafood challenge tests and in naturally contaminated cold-smoked salmon. Int. J. Food Microbiol. 1998, 40, 105–115. [Google Scholar] [CrossRef]
  9. Gómez, D.; Iguácel, L.P.; Rota, M.C.; Carramiñana, J.J.; Ariño, A.; Yangüela, J. Occurrence of Listeria monocytogenes in ready-to- eat meat products and meat processing plants in Spain. Foods 2015, 4, 271–282. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Hoelzer, K.; Pouillot, R.; Gallagher, D.; Silverman, M.B.; Kause, J.; Dennis, S. Estimation of Listeria monocytogenes transfer coefficients and efficacy of bacterial removal through cleaning and sanitation. Int. J. Food Microbiol. 2012, 157, 267–277. [Google Scholar] [CrossRef]
  11. Kushwaha, K.; Muriana, P.M. Adherence characteristics of Listeria strains isolated from three ready-to-eat meat processing plants. J. Food Prot. 2009, 72, 2125–2131. [Google Scholar] [CrossRef]
  12. Lunden, J.M.; Miettinen, M.K.; Autio, T.J.; Korkeala, H.J. Persistent Listeria monocytogenes strains show enhanced adherence to food contact surface after short contact times. J. Food Protect. 2000, 63, 1204–1207. [Google Scholar] [CrossRef] [PubMed]
  13. Mafu, A.A.; Roy, D.; Goulet, J.; Savoie, L. Characterization of physic—Chemical forces involved in adhesion of Listeria monocytogenes to surfaces. Appl. Environ. Microbiol. 1991, 57, 1969–1973. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Cox, L.J.; Kleiss, T.; Cordier, J.L.; Cordellana, C.; Konkel, P.; Pedrazzini, C.; Beumer, R.; Siebenga, A. Listeria spp. in food processing, non-food and domestic environments. Food Microbiol. 1989, 6, 49–61. [Google Scholar] [CrossRef]
  15. Norwood, D.E.; Gilmour, A. Adherence of Listeria monocytogenes strains to stainless steel coupons. J. Appl. Microbiol. 1999, 86, 576–582. [Google Scholar] [CrossRef] [PubMed]
  16. Ray, B.; Bhunia, A. Control by low pH and organic acids. In Fundamental Food Microbiology; CRC Press by Taylor & Francis Group: Boca Raton, FL, USA; London, UK; New York, NY, USA, 2008; pp. 394–410. [Google Scholar]
  17. Laguerre, O.; Derens, E.; Palagos, B. Study of domestic refrigerator temperature and analysis of factors affecting temperature: A French survey. Int. J. Refrig. 2002, 25, 653–659. [Google Scholar] [CrossRef] [Green Version]
  18. Likar, K.; Jevsnik, M. Cold chain maintaining in food trade. Food Control 2006, 17, 108–113. [Google Scholar] [CrossRef]
  19. Bortolussi, N.; Sponchiado, A.; Gottardo, F.; Iacumin, L.; Comi, G. Controllo e sopravvivenza di Listeria innocua in fase di cottura e pastorizzazione di prosciutti cotti. Ing. Aliment. 2010, 32, 27–34. [Google Scholar]
  20. Comi, G.; Ottaviani, S.; Patthey, C.; Mitri, E.; Menardi, G.; Andyanto, D.; Iacumin, L. Miglioramento della qualità di prosciutti cotti affettati e confezionati in MAP. Ing. Aliment. 2011, 12, 14–16. [Google Scholar]
  21. Comi, G.; Urso, R.; Vailati, E.; Ottaviani, S. Controllo e sopravvivenza di Listeria monocytogenes in fase di cottura e pastorizzazione di prosciutti cotti. Ind. Aliment. 2005, 446, 368–373. [Google Scholar]
  22. Aarnisalo, K.; Sheen, S.; Raaska, L.; Tamplin, M. Modelling transfer of Listeria monocytogenes during slicing of ‘gravad’ salmon. Int. J. Food Microbiol. 2007, 118, 76–78. [Google Scholar] [CrossRef]
  23. Chaitiemwong, H.; Hazeleger, W.C.; Beumer, R.R.; Zwietering, M.H. Quantification of transfer of Listeria monocytogenes between cooked ham and slicing machine surfaces. Food Control 2014, 44, 177–184. [Google Scholar] [CrossRef]
  24. Frustoli, M.A.; Cigarini, M.; Garritani, A.; Garulli, S.; Bovis, N.; Schivazzappa, C.; Barbuti, S. Fate of Listeria monocytogenes during shelf-life of pre-sliced bresaola packaged under modified atmosphere. Ind. Conserve 2007, 82, 325–330. [Google Scholar]
  25. Hoelzer, K.; Sauders, B.D.; Sanchez, M.D.; Olsen, P.T.; Pickett, M.M.; Mangione, K.J.; Rice, D.H.; Corby, J.; Stich, S.; Fortes, E.D.; et al. Prevalence, distribution, and diversity of Listeria monocytogenes in retail environments, focusing on small establishments and establishments with a history of failed inspections. J. Food Prot. 2011, 74, 1083–1095. [Google Scholar] [CrossRef] [PubMed]
  26. Little, C.L.; de Louvois, J. The microbiological examination of butchery products and butcheries’ premises in the United Kingdom. J. Appl. Microbiol. 1998, 85, 177–186. [Google Scholar] [CrossRef]
  27. Uyttendaele, M.; De Troy, P.; Debevere, J. Incidence of Listeria monocytogenes in different types of meat products on the Belgian retail market. Int. J. Food Microbiol. 1999, 53, 75–80. [Google Scholar] [CrossRef]
  28. Humphrey, T.J.; Worthington, D.M. Listeria monocytogenes of retail meat slicers. PHLS Microbiol. Dig. 1990, 7, 57. [Google Scholar]
  29. Hudson, J.A.; Mott, S.J. Presence of Listeria monocytogenes, motile aeromonads and Yersinia enterocolitica in environmental samples taken from a supermarket delicatessen. Int. J. Food Microbiol. 1993, 18, 333–337. [Google Scholar] [CrossRef]
  30. Pérez-Rodríguez, F.; Valero, A.; Carrasco, E.; García, R.M.; Zurera, G. Understanding and modelling bacterial transfer to foods: A review. Trends Food Sci. Technol. 2008, 19, 131–144. [Google Scholar] [CrossRef]
  31. Vorst, K.L.; Todd, E.C.D.; Ryser, E.T. Transfer of Listeria monocytogenes during mechanical slicing of turkey breast, bologna, and salami. J. Food Prot. 2006, 69, 619–626. [Google Scholar] [CrossRef]
  32. Sheen, S. Modelling surface transfer of Listeria monocytogenes on salami during slicing. J. Food Sci. 2008, 73, E304–E311. [Google Scholar] [CrossRef]
  33. Vorst, K.L.; Todd, E.C.D.; Ryser, E.T. Transfer of Listeria monocytogenes during slicing of turkey breast, bologna, and salami with simulated kitchen knives. J. Food Prot. 2006, 69, 2939–2946. [Google Scholar] [CrossRef] [PubMed]
  34. Montville, R.; Schaffner, D.W. Inoculum size influences bacterial cross contamination between surfaces. Appl. Environ. Microbiol. 2003, 69, 7188–7193. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Dickson, J.S. Transfer of Listeria monocytogenes and Salmonella typhimurium between beef tissue surfaces. J. Food Protect. 1990, 53, 51–55. [Google Scholar] [CrossRef] [PubMed]
  36. Kusumaningrum, H.D.; Riboldi, G.; Hazeleger, W.C.; Beumer, R.R. Survival of foodborne pathogens on stainless steel surfaces and cross-contamination to foods. Int. J. Food Microbiol. 2003, 85, 227–236. [Google Scholar] [CrossRef]
  37. Lin, C.M.; Takeuchi, K.; Zhang, L.; Dohm, C.B.; Meyer, J.D.; Hall, P.A.; Doyle, M.P. Cross-contamination between processing equipment and deli meats by Listeria monocytogenes. J. Food Prot. 2006, 69, 71–79. [Google Scholar] [CrossRef]
  38. Atkins, A.G.; Xu, X. Slicing of soft flexible solids with industrial applications. Intern. J. Mech. Sci. 2005, 47, 479–492. [Google Scholar] [CrossRef]
  39. Dawson, P.; Han, I.; Cox, M.; Black, C.; Simmons, L. Residence time and food contact time effects on transfer of Salmonella Typhimurium from tile, wood and carpet: Testing the five-second rule. J. Appl. Microbiol. 2007, 102, 945–953. [Google Scholar] [CrossRef]
  40. Cao, Y.; Gu, W.; Zhang, J.; Chu, Y.; Ye, X.; Hu, Y.; Chen, J. Effects of chitosan, aqueous extract of ginger, onion and garlic on quality and shelf life of stewed-pork during refrigerated storage. Food Chem. 2013, 141, 1655–1660. [Google Scholar] [CrossRef]
  41. Jasour, M.S.; Ehsani, A.; Mehryar, L.; Naghibi, S.S. Chitosan coating incorporated with the lactoperoxidase system: An active edible coating for fish preservation. J. Sci. Food Agric. 2015, 95, 1373–1378. [Google Scholar] [CrossRef]
  42. Luo, W.; Lin, H.S.; Ren, F.; Wu, L.; Chen, L.; Sun, Y. Antioxidant and antimicrobial capacity of Chinese medicinal herb extracts in raw sheep meat. J. Food Prot. 2007, 70, 1440–1445. [Google Scholar] [CrossRef]
  43. Pattanayaiying, R.; Kittikun, A.H.; Cutter, C.N. Incorporation of nisin Z and lauric arginate into pullulan films to inhibit foodborne pathogens associated with fresh and ready-to-eat muscle foods. Int. J. Food Microbiol. 2015, 207, 77–82. [Google Scholar] [CrossRef] [PubMed]
  44. Wu, S.; Zhang, H.; Zhou, H.; Junhua, J.; Xie, Y. Synergistic Effect of Plantaricin BM-1 Combined with Physicochemical Treatments on the Control of Listeria monocytogenes in Cooked Ham. J. Food Prot. 2017, 80, 976–981. [Google Scholar] [CrossRef]
  45. Woraprayote, W.; Malila, W.Y.; Sorapukdee, S.; Swetwiwathana, A.; Benjakul, S.; Visessanguan, W. Bacteriocins from lactic acid bacteria and their applications in meat and meat products. Meat Sci. 2016, 120, 118–132. [Google Scholar] [CrossRef] [PubMed]
  46. Gàlvez, A.; Abriouel, H.; Lòpez, R.L.; Ben Omar, N. Bacteriocin-based strategies for food biopreservation. Int. J. Food Microbiol. 2007, 120, 51–70. [Google Scholar] [CrossRef] [PubMed]
  47. Zhou, H.; Xie, Y.; Liu, H.; Jin, J.; Duan, H.; Zhang, H. Effects of two application methods of plantaricin BM-1 on control of Listeria monocytogenes and background spoilage bacteria in sliced vacuum-packaged cooked ham stored at 4 °C. J. Food Prot. 2015, 78, 1835–1841. [Google Scholar] [CrossRef] [PubMed]
  48. Leroy, F.; Verluyten, J.; De Vuyst, L. Functional meat starter cultures for improved sausage fermentation. Int. J. Food Microbiol. 2005, 106, 270–285. [Google Scholar] [CrossRef]
  49. Schillinger, U.; Kaya, M.; Lücke, F.K. Behaviour of Listeria monocytogenes in meat and its control by a bacteriocin-producing strain of Lactobacillus sake. J. Appl. Bacteriol. 1991, 70, 473–478. [Google Scholar] [CrossRef] [PubMed]
  50. Hugas, M.; Neumeyer, B.; Pagés, F.; Garriga, M.; Hammes, W.P. Antimicrobial activity of bacteriocin-producing cultures in meat products: 2. Comparison of bacteriocin producing lactobacilli on Listeria growth in fermented sausages. Fleischwirtschaft 1996, 76, 649–652. [Google Scholar]
  51. Kotzekidou, P.; Bloukas, J.G. Effect of protective cultures and packaging film permeability on shelf-life of sliced vacuum-packed cooked ham. Meat Sci. 1996, 42, 333–345. [Google Scholar] [CrossRef]
  52. Bredholt, S.; Nesbakken, T.; Holck, A. Protective cultures inhibit growth of Listeria monocytogenes and Escherichia coli O157:H7 in cooked, sliced, vacuum- and gas-packaged meat. Int. J. Food Microbiol. 1999, 53, 43–52. [Google Scholar] [CrossRef]
  53. Bredholt, S.; Nesbakken, T.; Holck, A. Industrial application of an antilisterial strain of Lactobacillus sakei as a protective culture and its effect on the sensory acceptability of cooked, sliced, vacuum-packaged meats. Int. J. Food Microbiol. 2001, 66, 191–196. [Google Scholar] [CrossRef]
  54. ISO 11290-1,2:1996 Adm.1. 2004 Microbiology of Food and Animal Feeding Stuffs e Horizontal Method for the Detection of Listeria Monocytogenes; ISO: Geneva, Switzerland, 2004. [Google Scholar]
  55. Mokrzycki, W.S.; Tatol, M. Colour difference ΔE—A Survey, Mach. Gr. Vis. 2012, 20, 383–411. [Google Scholar]
  56. AOAC. Official Methods of Analysis, 16th ed.; Association of Official Analytical Chemists: Washington, DC, USA, 1995. [Google Scholar]
  57. Baublis, R.T.; Meullenet, J.F.; Sawyer, J.T.; Mehaffey, J.M.; Saha, A. Pump rate and cooked temperature effects on pork loin instrumental, sensory descriptive and consumer rated characteristics. Meat Sci. 2005, 72, 741–750. [Google Scholar] [CrossRef] [PubMed]
  58. Vàlkovà, V.; Salàkovà, A.; Buchtovà, H.; Tremlovà, B. Chemical, instrumental and sensory characteristics of cooked pork ham. Meat Sci. 2007, 77, 608–615. [Google Scholar] [CrossRef] [PubMed]
  59. Regolamento (CE) n. 2073/2005. Available online: http://europa.eu.int/eurex/lex/LexUriServ/site/it/oj/2005/l_338/l_33820051222it00010026.pd (accessed on 25 April 2020).
  60. Hwang, C.A.; Sheen, S. Growth characteristics of Listeria monocytogenes as affected by a native microflora in cooked ham under refrigerated and temperature abuse conditions. Food Microbiol. 2011, 28, 350–355. [Google Scholar] [CrossRef] [PubMed]
  61. Amezquita, A.; Brashears, M.M. Competitive inhibition of Listeria monocytogenes in ready-to-eat meat products by lactic acid bacteria. J. Food Prot. 2002, 65, 315–325. [Google Scholar] [CrossRef]
  62. Hwang, C.A.; Sheen, S.; Juneja, V.; Hwang, C.-F.; Yin, T.-C.; Chang, N.-Y. The influence of acid stress on the growth of Listeria monocytogenes and Escherichia coli O157:H7 on cooked ham. Food Control 2016, 37, 245–250. [Google Scholar] [CrossRef]
  63. Luchansky, J.B.; Call, J.E.; Hristov, B.; Rumery, L.; Yoder, L.; Oser, A. Viability of Listeria monocytogenes on commercially-prepared hams surface treated with acidic calcium sulfate and lauric arginate and stored at 4 °C. Meat Sci. 2006, 71, 92–99. [Google Scholar] [CrossRef]
  64. Samelis, J.; Bedie, G.K.; Sofos, J.N.; Belk, K.E.; Scanga, J.A.; Smith, G.C. Combinations of nisin with organic acids or salts to control Listeria monocytogenes on sliced pork bologna stored at 4 °C in vacuum packages. Lebensm. Wiss. Technol.-Food Sci. Technol. 2005, 38, 21–28. [Google Scholar] [CrossRef]
  65. Samelis, J.; Sofos, J.; Kain, J.; Scanga, M.; Belk, K.; Smith, G. Organic acids and their salts as dipping solutions to control Listeria monocytogenes inoculated following processing of sliced pork bologna stored at 4 °C in vacuum packages. J. Food Prot. 2001, 64, 1722–1729. [Google Scholar] [CrossRef] [PubMed]
  66. Stekelenburg, F.K. Enhanced inhibition of Listeria monocytogenes in frankfurter sausage by the addition of potassium lactate and sodium diacetate mixtures. Food Microbiol. 2003, 20, 133–137. [Google Scholar] [CrossRef]
  67. Hwang, C.A.; Tamplin, M.L. Modelling the lag phase and growth rate of Listeria monocytogenes in ground ham containing sodium lactate and sodium diacetate at various storage temperatures. J. Food Sci. 2007, 72, M246–M253. [Google Scholar] [CrossRef] [PubMed]
  68. Legan, J.D.; Seman, D.L.; Milkowski, A.L.; Hirschey, J.A.; Vandeven, M.H. Modelling the growth boundary of Listeria monocytogenes in ready-to-eat cooked meat products as a function of the product salt, moisture, potassium lactate, and sodium diacetate concentrations. J. Food Prot. 2004, 67, 2195–2204. [Google Scholar] [CrossRef] [PubMed]
  69. Comi, G.; Iacumin, L. The use of bioprotective cultures. In Strategies to Obtaining Healthier Foods; Nova Science Publishers, Inc.: New York, NY, USA, 2017; pp. 1–40. [Google Scholar]
  70. Schobitz, R.; Zaror, T.; Leon, O.; Costa, M. A bacteriocin from Carnobacterium piscicola for the control of Listeria monocytogenes in vacuum-packaged meat. Food Microbiol. 1999, 16, 249–255. [Google Scholar] [CrossRef]
  71. Vermeiren, L.; Devlieghere, F.; Debevere, J. Evaluation of meat borne lactic acid bacteria as protective cultures for the biopreservation of cooked meat products. Int. J. Food Microbiol. 2004, 96, 149–164. [Google Scholar] [CrossRef] [PubMed]
Microorganisms 08 00898 g001 550
Figure 1. Cooked ham cubes with (a) and without (b) starter.
Figure 1. Cooked ham cubes with (a) and without (b) starter.
Microorganisms 08 00898 g001
Microorganisms 08 00898 g002 550
Figure 2. Evolution of microbial population in not-inoculated cubed cooked ham (control—NI), and stored at a controlled and constant temperature of 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 2. Evolution of microbial population in not-inoculated cubed cooked ham (control—NI), and stored at a controlled and constant temperature of 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g002
Microorganisms 08 00898 g003 550
Figure 3. Evolution of microbial population in not-inoculated cubed cooked ham (control—NI) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 3. Evolution of microbial population in not-inoculated cubed cooked ham (control—NI) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g003
Microorganisms 08 00898 g004 550
Figure 4. Evolution of microbial population in cubed cooked ham inoculated with BOX-74 starter (BOX74), Listeria monocytogenes (LM), and both BOX-74 starter and L. monocytogenes (BOX74 + LM) and stored at 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 4. Evolution of microbial population in cubed cooked ham inoculated with BOX-74 starter (BOX74), Listeria monocytogenes (LM), and both BOX-74 starter and L. monocytogenes (BOX74 + LM) and stored at 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g004
Microorganisms 08 00898 g005 550
Figure 5. Evolution of microbial population in cubed cooked ham inoculated with BOX-74 starter (BOX74), Listeria monocytogenes (LM), both BOX-74 starter and L. monocytogenes (BOX74 + LM) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 5. Evolution of microbial population in cubed cooked ham inoculated with BOX-74 starter (BOX74), Listeria monocytogenes (LM), both BOX-74 starter and L. monocytogenes (BOX74 + LM) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g005
Microorganisms 08 00898 g006 550
Figure 6. Evolution of microbial population in cubed cooked ham inoculated with BOX-57 starter (BOX57), Listeria monocytogenes (LM), and both BOX-57 starter and L. monocytogenes (BOX57 + LM) and stored at 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 6. Evolution of microbial population in cubed cooked ham inoculated with BOX-57 starter (BOX57), Listeria monocytogenes (LM), and both BOX-57 starter and L. monocytogenes (BOX57 + LM) and stored at 4 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g006
Microorganisms 08 00898 g007 550
Figure 7. Evolution of microbial population in cubed cooked ham inoculated with BOX-57 starter (BOX57), Listeria monocytogenes (LM), and both BOX-57 starter and L. monocytogenes (BOX57 + LM) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Figure 7. Evolution of microbial population in cubed cooked ham inoculated with BOX-57 starter (BOX57), Listeria monocytogenes (LM), and both BOX-57 starter and L. monocytogenes (BOX57 + LM) and stored at 4 °C for 20 days and then at 8 °C till the end of shelf-life. Data were expressed as mean ± standard deviation of the technical (2) and biological replicates (3); LAB, lactic acid bacteria; CBT, total bacterial count.
Microorganisms 08 00898 g007
Table
Table 1. Physico-chemical characteristics of the cubed cooked ham.
Table 1. Physico-chemical characteristics of the cubed cooked ham.
Parameters%
Moisture73.4 ± 0.4
Proteins18.5± 1.0
Lipids3.7 ± 0.3
Ash3.2 ± 0.2
Sugar1.2 ±1.1
aw0.992 ± 0.002
pH 6.35 ± 0.11
Table
Table 2. Presence of L. monocytogenes in cooked ham cubes packaged in Modified Atmosphere Packaging (MAP) and stored at 4 °C. Samples were randomly collected at retail.
Table 2. Presence of L. monocytogenes in cooked ham cubes packaged in Modified Atmosphere Packaging (MAP) and stored at 4 °C. Samples were randomly collected at retail.
Brands123
Days060060060
Lot 10/50/50/51/50/50/5
Lot 20/50/50/50/50/51/5
Lot 30/50/50/50/50/51/5
Lot 40/51/51/50/51/50/5
Lot 50/50/50/51/50/50/5
Lot 60/51/5 *0/50/50/50/5
Total0/302/301/302/301/302/30
Legend: Presence < 100 CFU g−1; * presence: 2.1 log CFU g−1.
Table
Table 3. aw values in cubed cooked ham stored at 4 °C, at 4 °C for 20 days and then at 8 °C for 40 days (end of shelf-life).
Table 3. aw values in cubed cooked ham stored at 4 °C, at 4 °C for 20 days and then at 8 °C for 40 days (end of shelf-life).
Daysaw (4 °C)aw (4 → 8 °C) *
00.986 ± 0.001 aw0.986 ± 0.001 aw
100.992 ± 0.002 av0.992 ± 0.002 av
200.994 ± 0.003 av0.994 ± 0.003 av
300.990 ± 0.001 av0.986 ± 0.001 bw
400.990 ± 0.003 av0.997 ± 0.001 bv
500.993 ± 0.004 av0.983 ± 0.003 bw
600.993 ± 0.001 av0.992 ± 0.001 av
Legend: * 20 days at 4 °C and 40 more days at 8 °C. Data were represented as the mean ± standard deviation; Different letters (a and b) indicate significant differences between the two temperatures (p < 0.05); the other set of different letters (v trough z) indicate differences among aw within each temperature (p < 0.05).
Table
Table 4. Trend of pH in cubed cooked ham with or without starter addition.
Table 4. Trend of pH in cubed cooked ham with or without starter addition.
Control NIBOX-74BOX-57
DayspH 4 °C* pH 4 → 8 °CpH 4 °C* pH 4 → 8 °CpH 4 °C* pH 4 → 8 °C
06.35 ± 0.11 av6.35 ± 0.11 av6.35 ± 0.11 av6.35 ± 0.11 av6.35 ± 0.11 av6.35 ± 0.11 av
106.35 ± 0.20 av6.35 ± 0.20 av6.21 ± 0.31 av6.21 ± 0.31 av6.18 ± 0.20 av6.25 ± 0.10 av
206.06 ± 0.11 aw6.06 ± 0.11 aw6.01 ± 0.21 av6.01 ± 0.21 av6.00 ± 0.11 av5.90 ± 0.25 aw
305.82 ± 0.06 ax5.94 ± 0.08 bw5.82 ± 0.06 aw5.94 ± 0.08 bv5.80 ± 0.06 aw5.75 ± 0.03 cx
406.12 ± 0.06 aw6.06 ± 0.06 aw6.02 ± 0.06 av6.06 ± 0.03 av5.80 ± 0.10 bw5.70 ± 0.01 cx
506.33 ± 0.05 av6.27 ± 0.01 bv5.59 ± 0.25 cw5.49 ± 0.07 cw5.48 ± 0.20 cx5.65 ± 0.22 dx
606.18 ± 0.05 ax6.17 ± 0.02 ax5.69 ± 0.10 bw5.14 ± 004 cx5.65 ± 0.30 bx5.33 ± 0.20 dy
Legend: Control NI, cubed cooked ham without inoculated starter; BOX-74, trial inoculated with BOX-74 starter only; BOX-57, trial inoculated with BOX-57 starter only; * 20 days of storage at 4 °C followed by additional 40 days at 8 °C. Data were reported as the mean ± standard deviation; Mean with different letters (a versus d) within the lines were significantly different (p < 0.05); the other set of different letters (v trough z) indicate differences among pH within each temperature and each treatments (NI, BOX-74 and BOX-57) (p < 0.05).
Table
Table 5. Colour changes in the different trials stored at different temperatures.
Table 5. Colour changes in the different trials stored at different temperatures.
L* (4 °C)L* (4 → 8 °C)
DaysNIBOX-74BOX-57 *NIBOX-74BOX-57
062.86 ± 2.90 av65.71 ± 1.43 av65.97 ± 0.72 av62.86 ± 2.90 av63.40 ± 5.39 av65.31 ± 0.79 av
6065.02 ± 2.81 av64.66 ± 2.77 av66.83 ± 0.58 av65.72 ± 1.03 av65.74 ± 2.03 av67.22 ± 1.03 av
a* (4 °C)a* (4 → 8 °C)
NIBOX-74BOX-57 *NIBOX-74BOX-57
014.20 ± 0.23 av14.84 ± 0.32 av14.24 ± 0.40 av14.20 ± 0.23 av13.98 ±1.15 av14.65 ± 0.18 av
6014.23 ± 0.65 av14.29 ± 0.31 av14.65 ± 0.11 av14.26 ± 0.35 av14.57 ±0.22 av13.23 ± 0.61 av
b* (4 °C)b* (4 → 8 °C)
NIBOX-57BOX-57NIBOX-57BOX-57
03.60 ± 0.50 av3.66 ± 0.13 av3.51 ± 0.20 av3.60 ± 0.50 av3.07 ± 0.74 av3.62 ± 0.10 av
603.59 ± 0.28 av3.36 ± 0.12 av3.51 ± 0.26 av4.36 ± 0.56 av3.76 ± 0.23 av3.48 ± 0.45 av
Legend: Control NI, cubed cooked ham without inoculated starter; BOX-74, trial inoculated with BOX-74 starter only; BOX-57, trial inoculated with BOX-57 starter only; * 20 days of storage at 4 °C, followed by additional 40 days at 8 °C. Data were reported as the mean ± standard deviation; mean with different letters (a versus d) within the lines were significantly different (p < 0.05); the other set of different letters (v trough w) indicate differences among days within each temperature and each treatments (NI, BOX-74 and BOX-57) (p < 0.05).
Table
Table 6. Fate of L. monocytogenes coinoculated with BOX-74 and BOX-57 starter cultures in cubed cooked ham stored in MAP at 4 °C, at 4 °C for 20 and then for 40 days at 8 °C.
Table 6. Fate of L. monocytogenes coinoculated with BOX-74 and BOX-57 starter cultures in cubed cooked ham stored in MAP at 4 °C, at 4 °C for 20 and then for 40 days at 8 °C.
Temperature 4 °C
4 → 8 °C
Days* LMBOX-74 + LMBOX-57 + LMBOX-74 + LMBOX-57 + LM
02.1 ± 0.3 av2.1 ± 0.1 av2.1 ± 0.1 av2.1 ± 0.1 av2.1 ± 0.1 av
102.8 ± 0.1 bw2.2 ± 0.2 av2.3 ± 0.1 av2.2 ± 0.2 av2.3 ± 0.1 av
204.9 ± 0.7 ax2.2 ± 0.4 bv1.6 ± 0.2 cw2.2 ± 0.4 bv1.6 ± 0.2 cw
308.2 ± 1.3 ay1.7 ± 0.4 bv1.6 ± 0.1 bw1.9 ± 0.3 bv1.4 ± 0.1 bcx
407.9 ± 0.5 ay1.5 ± 0.5 bv1.3 ± 0.2 bw1.6 ± 0.5 bv1.1 ± 0.5 by
508.5 ± 0.2 ay1.5 ± 0.2 bv1.3 ± 0.1 bw1.5 ± 0.1 bv1.3 ± 0.2 bx
608.4 ± 0.3 ay1.8 ± 0.2 bv1.2 ± 0.4 cw1.9 ± 0.3 bv1.4 ± 0.1 cx
Legend: Data were reported as the mean ± standard deviation; mean with different letters (a versus e) within the lines were significantly different (p < 0.05); the other set of different letters v trough z) indicate differences among days within each treatments (* Listeria monocytogenes (LM), and BOX-74 or BOX-57 starter inoculated with L. monocytogenes) (p < 0.05).
Table
Table 7. Sensorial evaluation by untrained assessors.
Table 7. Sensorial evaluation by untrained assessors.
ParameterControlStarter 1Starter 2
Fermented0/120/120/12
Rancidity0/120/120/12
Sweet10/129/129/12
Acid0/121/121/12
Meaty12/1212/1212/12
Fresh6/126/127/12
Bitter0/120/120/12
Ammoniacal0/120/120/12
Slime0/120/120/12
Final value *222
* Scores: 1 (excellent). 2 (good). 3 (sufficient). 4 (scarse).

Share and Cite

MDPI and ACS Style

Iacumin, L.; Cappellari, G.; Colautti, A.; Comi, G. Listeria monocytogenes Survey in Cubed Cooked Ham Packaged in Modified Atmosphere and Bioprotective Effect of Selected Lactic Acid Bacteria. Microorganisms 2020, 8, 898. https://doi.org/10.3390/microorganisms8060898

AMA Style

Iacumin L, Cappellari G, Colautti A, Comi G. Listeria monocytogenes Survey in Cubed Cooked Ham Packaged in Modified Atmosphere and Bioprotective Effect of Selected Lactic Acid Bacteria. Microorganisms. 2020; 8(6):898. https://doi.org/10.3390/microorganisms8060898

Chicago/Turabian Style

Iacumin, Lucilla, Giorgia Cappellari, Andrea Colautti, and Giuseppe Comi. 2020. "Listeria monocytogenes Survey in Cubed Cooked Ham Packaged in Modified Atmosphere and Bioprotective Effect of Selected Lactic Acid Bacteria" Microorganisms 8, no. 6: 898. https://doi.org/10.3390/microorganisms8060898

APA Style

Iacumin, L., Cappellari, G., Colautti, A., & Comi, G. (2020). Listeria monocytogenes Survey in Cubed Cooked Ham Packaged in Modified Atmosphere and Bioprotective Effect of Selected Lactic Acid Bacteria. Microorganisms, 8(6), 898. https://doi.org/10.3390/microorganisms8060898

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop