development of a tool for effective diagnostic monitoring of honey bee colonies.

Final Report Summary - SWARMONITOR (development of a tool for effective diagnostic monitoring of honey bee colonies.)

Executive Summary:
There are over half a million beekeepers in Europe, mainly small family-owned companies with less than ten workers. Beekeepers are still using traditional low technology labour intensive methods as it is an absolute necessity for beekeepers to pay regular visits to their apiaries. These can be situated far apart from each other and every single colony must be opened and visually inspected, one by one, whether it requires intervention or not. This is the only way beekeepers can attempt to control swarming and monitor health disorders. The need for modern tools and technology is recognised within beekeeping to improve both efficiency and profitability. Currently there are a range of products based around hive scales from different manufacturers that also include measurements such as temperature and humidity. The aim of the Swarmonitor project was to develop monitoring tools that could detect vibrations from within a bee hive, characteristic of different honey bee activities, and inform the beekeeper if a specific hive requires intervention. These tools will allow beekeepers to have a diagnosis of colony status without the need for frequent visits to the apiary or invasive opening of hives for physical inspection. This will enable more effective and less labour intensive apiary management, by allowing the beekeeper to selectively intervene at an appropriate time. As part of Swarmonitor, a procedure for analysing the vibrational data and attributing it to a colony condition has been described in a patent application (GB 1409807.3 ) and an example of using this for the prediction of swarming has been demonstrated. This technique is extendable to allow other conditions to be diagnosed and work on this continues particularly on the level of Varroa destructor mite infestation. Additional tools for the monitoring of the brood cycle of the honey bee queen and the state of the colony over winter time when the hives cannot be opened have also been demonstrated. The key to adoption of any technique is the affordability and inexpensive routes to suitable hardware have also been demonstrated. Swarmonitor has been a collaboration between the Bee Farmers Association of the United Kingdom (BFA, UK), the Research and Information Centre for Bee Culture (CARI, Belgium), the European Professional Beekeepers Association (EPBA, Germany), CAPAZ GmbH (Germany), SZOMEL Services and Trade LLC (Hungary), Arnia Ltd. (UK), the National Institute for Agricultural Research (INRA, France) and Nottingham Trent University (UK).

Project Context and Objectives:
One-third of the human diet can be traced directly, or indirectly, to insect pollination. The role of bees as a pollinator remains the most important economic contribution, far outweighing the importance of hive products such as honey and wax. Although the number of managed honey bee colonies has increased globally over the last half century this has been accompanied by an even greater need for insect pollination. Over the last 15 years, dramatic colony losses over winter have been reported in some regions of the world. Many factors may account for this decline in managed honey bee colonies in the US and Europe amongst them bacterial and viral infections and the widespread increase Varroa destructor mite. There are more than half a million beekeepers in the European sector, mainly SME family-owned companies with less than ten workers. Beekeepers are still using traditional low-tech labour-intensive methods as it is an absolute necessity for beekeepers to pay regular visits to their apiaries, that can be far apart from each other and from home, to open and visually inspect every single colony, one by one, whether it requires intervention or not. This is the only way beekeepers can try to manage swarming and monitor health disorders. The need for modern tools/technology is recognised within beekeeping to improve both efficiency and profitability however this must also be affordable technology. The EU has seen a worrying drop in the number of beekeepers and an increase in the age of beekeepers, losing as many as 10% of beekeepers annually without corresponding new entrants to the business. In general it is difficult to find new people interested in beekeeping as it is recognised as a no longer economically viable activity due to the need to use high labour intensive methods. It is against this backdrop of an increasing need for managed pollination and declining numbers of people willing to undertake this type of work that new tools and methods are being sought. Honey bees live in hives where little or no light penetration occurs rendering visual communication virtually impossible. In order to communicate effectively with one another bees employ alternative signalling methods for communication. These include chemical signalling, through the emittance and reception of differing hormones, as well as mechanical signalling whereby sound and vibrations are transmitted from sender to receiver. Honey bees detect airborne sound through antennal flagellum located on the forehead and it has been suggested that they sense vibrations transmitted through the substrate via the subgenual organ, proposed to be located between the tibia and the femur. Currently there are a range of products based around hive scales from different manufacturers that also include measurements such as temperature and humidity. In addition, microphone based sound measurement has received significant interest but microphones are subject to deterioration by coating of propolis by the bees. Accelerometers measure vibrations and can be embedded into the wall or wax of the honey comb and so are resistant to propolising. The aim of the Swarmonitor project was to develop monitoring tools using accelerometers that could detect vibrations from within a bee hive, characteristic of different honey bee activities, and inform the beekeeper if a specific hive requires intervention. These tools will allow beekeepers to have a diagnosis of colony status without the need for frequent visits to the apiary or invasive opening of hives for physical inspection. The major challenge of this type of work is to measure the vibrational information whilst at the same time having confirmation about the actual status of the honey bee colony. Furthermore it requires sufficient numbers of colonies in each of the particular conditions to be monitored to make the study meaningful. Providing a sufficient large number of closely monitored colonies was one of the main aims of the Swarmonitor project as such levels of resources are beyond the scope of a single producer of hive monitoring products.

Project Results:
The technology developed within the Swarmonitor research project has pioneered the extensive use of accelerometers to obtain diagnostic information on honey bee colonies. This type of sensor exhibits several strong advantages in terms of monitoring honey bees, in particular (i) they do not have any moving parts exposed to the outside, (ii) they are particularly good at monitoring low frequency signals in the range of those relevant to honey bees, (iii) inexpensive types are readily available, (iv) they directly collect vibrational signals of a physical nature that is highly relevant to honeybee communication processes, (v) they can reside in a honeybee hive for several years with negligible effect, due to their small volume, and their performance remains intact irrespective of any propolis coating. Crucial to the outcome of the measurement delivered by an accelerometer, is the substrate on which the sensor is secured. The vibrations propagating in the substrate will be those measured by the sensor, and the geometrical and physical features of the substrate, as well as the vicinity of the sensor to the vibration source (the honeybee/honeybee colony) will have a pronounced effect on the outcome of the measurement. The success of our project's aims and goals is therefore intimately linked to the careful choice of the substrate onto which our sensors are attached, and their location within the colony/hive. Although substrates that are hard (high Young modulus) and light (low mass density) are inherently good at conveying vibrations, honeycomb is (i) in the immediate vicinity of honeybees, (ii) inherently insensitive to vibrations induced externally by outside noise (e.g bird songs) and vibrations (rain) and (iii) benefiting from an 'time-averaged' colony signal by means of different bees cyclically returning to the centre of the colony. The identification of the optimum location for our accelerometers therefore required experimental exploration, and the final choice is eventually moderated by the elegance of the solution, from the point of view of a commercial product.
Ultra-high performance accelerometers have been trialled (i) in the centre of the central honey comb, (ii) in the centre of eight neighbour honeycombs, (iii) in the top wooden bar of the central frame, (iv) in the centre of the wooden face of the hive, (v) on a metal resonance plate of a modified brood box. In our UK apiary in 2013 summer, one hive was set up with a high performance (1000 mV/g) Brüel and Kjær accelerometer in the centre of the honey comb of the central frame, and another identical accelerometer in the wooden top bar of the same frame. In our French apiary at the Avignon INRA research centre, during the summer of 2013, four hives were set up in an identical way, but with ten times less sensitive, high performance Brüel and Kjær accelerometers. The rest of the 20 INRA hives had the same accelerometers embedded in the central comb. This remained the case until the autumn of 2014 when 20 accelerometers were moved to the outside wall of the hives. At the same time the UK apiary had 10 hives with the walls monitored with the higher sensitivity Brüel and Kjær accelerometers.
For the accelerometers in the comb, signal strength above 1kHz is negligible and two common peaks at 125 Hz and 250 Hz are dominant. The long term analysis shows that the signal in the wooden bar tends to follow the trend of that seen in the honey comb, which is severely modulated by the 23-days long brood cycle in the frame under consideration. They tend to be 20 times less than in the honeycomb, and do not always end up in a distribution exhibiting a clear peak, unlike in the case of the honeycomb
In the wooden face, signal strength is typically one order of magnitude lower than the hive. Signal strength above 1kHz is negligible and spectral responses are totally different to the comb accelerometers. Vibrational amplitude deviations are much stronger in the wooden face and of the two usual peaks at 125 Hz and 250 Hz found in the honey comb only the 250 Hz peak can be seen in the wooden face. Massive, short duration honeybee exodus (around mid-day) results in a very large signal strength enhancement in the wooden face, highly specific to a bandwidth around 500 Hz, accompanied by an overall reduction in signal strength in the honeycomb and a reduction in signal strength at 250 Hz in the wooden face. Enhanced honeybee activity can be seen on both accelerometer positions in the afternoon but spectral responses are very different, with honeybee signals of interest focused in the 0 - 500 Hz band for the honey comb, and the 200 Hz - 600 Hz band for the wooden face. Amplitude deviations in the wooden face are huge (the highest seen in any of the accelerometer locations that have been explored), with standard deviation several times larger than the mean amplitude, in most frequencies. Although mean signal is typically 10 times lower in the wooden face, peaks in vibrational activity in the wooden face can reach amplitudes as high as in the honey comb but spectral shape changes are much more varied in the wooden face than in the honey comb, with highly specific enhancements or decreases at specific times of the day, promising high discriminatory power when the origin of the phenomena triggering the spectral changes are identified. The modified brood box with a resonance plate exhibits signal strength twice as high as when using an accelerometer mounted on the hive's wall. It exhibits high sensitivity to foraging activity. It is unfortunately unexpectedly highly sensitive to external sources of sounds and vibrations.
Low cost accelerometers were tried on different locations to capture the honeybee signal. Although these sensors have a sensitivity (300 mV/g) intermediate to those (100 mV/g and 1000 mV/g) of the ultra-high performance accelerometers tried in the previous sections, their intrinsic poor SNR yielded no useful measurement when mounted (i) in the middle of the honeycomb, (ii) on the queen excluder or (iii) on the wooden wall of the hive. On any of these three configurations, honeybee signal was only seen in instances of extremely high activity. Provided that the ADXL 335 is mounted on a light enough substrate, they can still provide useful signal, even in cases of low honeybee activity. Best results were obtained when mounting them on thin acrylic membranes. Low cost accelerometers mounted on thin acrylic membranes residing in the back wall of a brood boxes at the CARI apiary in Belgium, at the INRA apiary in France and Brackenhurst apiary in the UK. An alternative to the acrylic membrane in the walls of brood boxes is to include a dummy board that incorporates the same acrylic membrane as used in the walls. The quality of the signal is similar to that obtained from the back of the hive. Since dummy boards are peripheral to the colony, the nature of the signal is also similar to the other configuration. The use of dummy frames with embedded sensors allows, however, the use of a product that does not require modification of the brood box. Most beekeepers use dummy boards, and they can be made much slimmer than a normal frame. They are inherently close to the honeybees and immune to external sounds and vibrations since they reside within the brood box.
The main advantages of using accelerometers in the honey comb are (i) outstanding immunity to external sounds and vibrations, (ii) reliability of the spectral response, (iii) sensing important changes in the load of the honeycomb, in particular that of the brood, (iv) high enough SNR to allow meaningful winter monitoring. The main drawbacks are (i) sensitivity highly local to the frame under investigation, not necessarily yielding an image of the entire colony, (ii) cumbersome installation/use for a commercial product, (iii) presently not compatible with the use of inexpensive accelerometers, (iv) rather insensitive to foraging activity. At the expense of lower signal levels, similar conclusions can be drawn from using an accelerometer in the wooden bar of a frame.
With greater cost and sophistication, placing an accelerometer in each individual frame allows one to combine the previously mentioned advantages, together with the ability of yielding a global picture of the entire colony. The use of an accelerometer in the outside wall of a hive requires minimal changes, if any, from the beekeeper to allow him/her to use the sensor. The SNR is insufficient for winter monitoring but is sensitive to colony bee total numbers and therefore inherently provides greater signal in the close vicinity of a primary swarm. It provides highly specific sensitivity to foraging and seems mildly sensitive to the structural vibrational modes of the inside of the hive, such as frame load, and the presence of a super. It suffers from vibrational modes that are specific to the hive itself, due to the wood used to make it, and/or the way it has been assembled (see also Deliverable 1.3). At the expense of a change in the structure of the brood box, it may be possible to use external accelerometers with high immunity to external noises and vibrations, although our attempt to demonstrate this has only yielded higher collected honeybee signal. Provided that they are used in combination with a thin membrane, inexpensive accelerometers can be used to monitor honeybees. The solution that we have explored is (i) suitable for winter time monitoring, (ii) highly immune to external sounds and vibrations, (iii) suitable for long term monitoring (we have multiple evidence that the membranes are not propolised by the honeybees). We are pro-actively working at further identifying the advantages of this type of set up.

In deliverable report D1.2 we have investigated the link between vibrational signal and the ‘history’ of a honey bee hive. This included a laboratory investigation of the effects of both simulated and actual hive changes on measured artificially introduced vibrational signals starting with foundation comb and moving onto built and loaded comb. Different positions of accelerometer and vibration generator were used along with a range of amplitudes and frequencies. We have shown that the overall comb response is complex, comprising frequency dependent peaks and dips. This complexity remains following building and loading, although the response becomes more symmetrical with respect to a centralised source of vibrations after the comb has been built naturally. We are able to draw general conclusions about the effect of natural hive changes on measured signals. As the comb is naturally built, very low frequency signals are suppressed and higher frequencies (above about 250 Hz) are easily transmitted. The process of loading the comb with honey, however, results in a significant reduction in broadband transmission efficiency. This reduction appears to be drastic across honey loaded but uncapped cells. The signal measured at the top strut of the frame of a foundation comb was far weaker than that measured at the comb centre. The relative strength measured in this position improved, however, following natural building, although the absolute strength was reduced. Following loading, the situation was not as clear cut, because there were strong fluctuations in relative and absolute strength depending on the source-detector distance and orientation. We have also shown that attenuation of vibrations is rapid on moving to adjacent frames, and drastic on travelling beyond the neighbouring frame to that into which they were introduced. To this extent, vibrations introduced via the external wall of the hive register minutely at the central comb and vice versa.
There are numerous different designs of honey bee hives, used by commercial and amateur beekeepers. These hives will exhibit different vibrational modes, due to both their geometry and type of wood and more recently even hives made of plastic. Within our project, it is desirable to come up with a monitoring device that would work equally well on several or all hive types, and it is therefore essential to quantitate the extent to which our measurements are affected by the hive type. In deliverable report 1.3 we presented work in which we have quantitated the effect that hive type has on our vibrational measurements, for different hive types, and for different location of accelerometers. First, deviations were explored within a collection of hives of the same type. Deviations within the bee induced signals will come both from the differing bees, and from the differing wood in each hive. Deviations within a man-made shaker induced signals will originate only from the differing wood in each hive. We have shown that within any hive type, important deviations can be seen in the vibrational modes that the external wall exhibit. The source of the deviations probably reside both in the wood used to make the hive and the content of the hive. We have shown that within any hive type, important deviations can be seen in the spectral features of honey bee induced vibrations, and these tend to be larger than the deviations seen in the vibrational modes. We have shown that different hive types have different vibrational modes, although all hive types exhibit enhanced vibrations around 500 Hz and around 2 kHz. We have shown that different hive types exhibit similar SNR when external noise/vibrations interfere with the measurement.
As an alternative to comb or wall mounting we have investigated using an accelerometer secured to an acrylic membrane glued into one wall of the brood box. We have shown that within any hive type, and between hive types, negligible deviations are seen on the spectral responses: the cavity resonator that the hives produce are not strongly affected by the hive type, but more probably by honeybee race and/or local climate. The extent to which our project's results are affected by hive type is discussed in the relevant deliverable. At the expense of having to modify existing brood boxes, the acrylic membrane solution is one way forward to minimise deviations due to hive type. At the expense of using sensors inside the colony, the use of an accelerometer embedded in the honey comb itself is another way forward to achieve the same outcome.

Originally, the project was intended to benefit from 20 colonies undergoing the swarming season three times, in 2013 and 2014 and 2015. Due to technical difficulties we missed the 2013 swarming season measurements, and actions were rapidly discussed and taken to compensate for the data loss. An apiary additional to the French INRA one was established at the UK Brackenhurst campus in March 2014, comprising ten hives. A further additional apiary was set up in Belgium by CARI in the same spring, comprising ten hives. An excellent array of swarming events were captured this way, both in 2014 in 2015, more than compensating for the data loss from 2013. One of the important achievements of the Swarmonitor research project is the spectacular collection of swarming events that have been captured and recorded. A good collection of data was collected in the case of (i) high performance accelerometers secured to the centre of the central honey comb of a colony, (ii) high performance accelerometers secured to the front face of the brood box of a colony, and (iii) inexpensive accelerometers secured to an acrylic disc residing at the back of the brood box of a colony. Some data was collected in other configurations, for example with an accelerometer embedded in the wooden bar of a frame, but numerous repeats of swarming events, with the same measurement conditions, are necessary to confirm that a signature is generic to the preparation for swarm. If only a single measurement or too few repeats are available, any vibrational signature obtained by the numerical search could be erroneously identified and associated with the preparation for the swarm. Some results were obtained using ultra-high performance Brüel and Kjær accelerometers. Twenty four of these were purchased for the French INRA apiary with a high sensitivity at 100 mV/g. Twelve of them were purchased for the UK hives with an ultra high sensitivity at 1000 mV/g. These accelerometers were mainly used in two different configurations, (i) in the centre of the central frame of the brood box and (ii) mounted externally in the middle of the front face of the broodbox. The Brüel and Kjær software PULSE® was used to digitise and collect these measurements, in the form of power spectra averaged for 3 minutes, with bandwidth of 5500 Hz and frequency resolution of 3 Hz. For use in the honeycomb, a cavity (1 cm3) was made in the honey comb in the middle of the central frame of each colony, an accelerometer placed in it, and a small amount of liquid wax poured on it to avoid any direct exposure of bare metal. We know from experience that a colony will live many years in a hive with accelerometers that are set up in this way, and that bees will use cells in the immediate vicinity of the accelerometer normally for pollen, brood or honey. The colonies that were preparing to swarm or had swarmed often, but not necessarily, left visual evidence of these, and this is fully logged in Deliverable 2.1 which provides all the beekeeper's observations following any regular visual inspection. Three security cameras were installed on April 10th for evidencing swarm events. Ten frames per second are recorded, with a resolution of 1024x768, and they were stored using an external device.
Although some swarming events are fully corroborated by video evidence from the cameras setup at INRA, a primary swarm also leaves an unmistakable trace on vibrational measurements that are undertaken in the honeycomb. Furthermore, additional accompanying features help further establish the log of a swarm: (i) swarms usually take place around lunch time or early afternoon. (ii) they are preceded by a substantial increase in the bee sounds frequency over approximately two to three hours before the lift-off. (iii) they exhibit a huge vibrational peak at ultra-low frequencies, around 60 Hz, at the time of the lift-off. (iv) they are usually followed by "queen pipes" a few days later (see Deliverable 1.1) (v) they are followed by a large signal drop as the queen and worker bees capable of flight leave the hive. If the queen is unclipped (capable of flying) this signal drop will remain for several days, but build up slowly after some days as the young bees mature. However, if the queen was clipped (incapable of flight) there is a large signal enhancement shortly after the drop in signal as the worker bees return to the hive. The clipped queen may or may not return with them. (vi) they are followed by an unusually long absence of worker brood, (typically around 14 days) as the new queen needs time to be born, to successfully mate, to become mature, and to start laying brood. The entire collection of primary, secondary and tertiary swarms captured in this way are shown in the Deliverable 4.3 and 4.4. In another setup, an accelerometer is secured, externally, in the middle of the front face of the broodbox of a hive that may or may not have a super (a smaller box of frames for collecting honey placed above the broodbox). Extensive work showing the effect of securing the accelerometer to the front face of a hive is shown in Deliverable 1.1 and 1.3. Although a primary swarm leaves an unmistakable trace on vibrational measurements that are undertaken from within the honeycomb, the features that match a primary swarm logged from within the outer face of the broodbox are more subtle and make primary swarm identifications harder (when using the vibrational logs only). Fortunately, many primary swarms have been captured both from within the honeycomb and the wall, and this gives us a reliable set of primary swarms to look at, to learn these features. The frequency increase of the background honeybee vibration in the hours preceding the swarm is not usually clearly detected when measured from within the wall, although some recordings show a faint trace of the phenomenon. The signal intensity increase at the time of the swarm lift off is much more abrupt, short (approx. 20 mins) and pronounced, when measured from within the wall. Additional figures are therefore supplied here, with a horizontal axis cropped to shorter time scales. The logarithmic scale's range has been increased, compared with the honeycomb measurements, to cope with the extraordinary dynamic range of the signal. A strong and short lived peak at 250 Hz appears specifically at the time of the swarm lift off (and this does not take place in any other instance). A swarm, as for the case of logs from within the honeycomb, usually takes place around lunch time or early afternoon. A primary swarm is usually (but not always) followed by "queen pipes" a few days later (see Deliverable 1.1) and queen pipes can be logged from within the front face of hives, but with lower sensitivity than from within the honeycomb. A swarm is usually followed by (a) a large signal drop if the queen is unclipped (capable of flying) as many bees will leave with the queen, and by (b) a large signal enhancement if the queen was clipped (bees come back to the hive). The entire collection of primary, secondary and tertiary swarms captured in this way are shown in the Deliverable 4.3 and 4.4. Overall, Swarmonitor has provided an outstanding array of swarming events, mostly in the second period, logged with numerous different configurations and instrumentations, within a range of colonies spanning the strongest and the weakest individuals, and including cases of 'clipped' queens and 'unclipped' queens. A lot of variability in the colony swarming and health in 2014 and 2015 was seen, which is exactly what is needed to use vibration patterns to predict swarming and honey bee colony health.
The overall core aims in swarmonitor are to monitor vibrations with accelerometers and, from these, to develop numerical methods to determine the health status and swarming status of the apiary.
The indicators of health status that turned out to be successfully diagnosed by our accelerometer measurements are those relating to (i) winter weakness leading to colony failure, (ii) brood cycle abnormalities, and (iii) varroa infestation levels. An extensive study of the detection of the intention for swarming has also been undertaken.
Because the species of honey bee studied in this work (various subspecies of Apis mellifera) remain in hives overnight, we concern ourselves only with data logged throughout this time from either sunset to sunrise or from midnight to 06:00 to ensure they contain no contributions from foraging activity, for all of the studies that we undertook.

What is particularly insidious about winter losses – and what largely underpins the rationale for this work – is that safe monitoring through visual inspection is not normally possible in-situ due to the danger of over-cooling the colony by opening hives during this period, though the extent to which bees cope with winter, i.e. ‘overwinter’, varies markedly from one subspecies to another. Here we describe how to diagnose honey bee colony status by logging and analysing vibrations measured using accelerometers embedded in the wax of hive combs.
Our main experimental apiary comprised twenty honey bee colonies that were moved to an INRA field experiment site in Avignon in March 2013 and were placed in ten-frame Langstroth hives, a very common hive type worldwide. The INRA apiary was up and running several months in advance of the first measurements being recorded, and its hives were inspected in September 2013 to estimate the extent of pollen, honey and brood and bee number. Because of the consequences of significant Varroa mite and Nosema parasitism on colony survival, these levels on adult bees were estimated using standard methods. Seven colonies were treated with APIVAR against the mite in October. INRA hives were also inspected to determine whether the hives were live the following winter. We also analysed data from two British National hives at the Clifton campus of Nottingham Trent University from winter 2013/2014 and supplemented our study with data from these hives from the following spring and summer.
Vibrational accelerations generated by honey bees domiciled within given hives were measured from sunset to sunrise with a resolution of 3.125 Hz using Brüel and Kjaer Type 4508 piezoelectric accelerometers. Twenty and two hives were observed at INRA and NTU, respectively, and over 30 million individual frequency resolved data points were logged and analysed. The accelerometers were positioned in the wax at the centre of the central comb of each INRA colony over winter 2013/2014 and of all-INRA-colonies-but-one over the winter of 2014/2015 (the accelerometer of colony 19 was placed in an adjacent frame over winter 2014/2015) and of our NTU hives. Accelerometers were also placed in the wax at the centre of seven other combs of one of our NTU hives.
By the time measurements came to be recorded in October 2013, six colonies of the INRA apiary were found to have died. By January 2014, only one colony of this apiary had survived but all four colonies monitored in the wax at INRA survived the following winter. Both NTU colonies survived the winter of 2013/2014. This gives a total of seven colonies that survived at least one winter.
There were no bees in the hives of the initially dead colonies, though they still contained food (honey and pollen). These losses resulted in what we consider to have been naturally produced control hives for the purposes of this work.
Qualitatively we found that averaged accelerometer signals considered for the purpose of this study, comprising both amplitude and frequency information, changed little on reducing the upper frequency threshold dramatically from 5 kHz to 300 Hz. We infer from this that the vibrational activity of interest lies overwhelmingly below 300 Hz. We also found a lower threshold around 20 Hz to be necessary to eliminate external ambient vibrations, as without one they dominated much of our analysis whether or not bees resided in the hive. All our analyses from this point on, therefore, are limited to the frequency range bounded by these thresholds (i.e. from 22 to 300 Hz) or a range within it.
The results, when signals from colonies that were determined by physical inspection by INRA personnel to have been lost before the start of this study were averaged daily over frequency and time in the same way.

We found that colonies that fail exhibit a mean vibrational amplitude below that of any surviving colony. We found that, two to three weeks prior to failure, the main spectral feature at 125 Hz exhibits an increase superior to any colony that survived. We have optimised our results by reducing the significance of vibrations that have not resulted from the presence in hives of live honey bees to the overall signal recorded whilst maximising the power to discriminate between signals from live and failed colonies. This discrimination was successful over different winter periods in very different climates (those of the south of France and northern England), evidencing successful generalisation in keeping with Deliverable 4.2. We have shown the potential of both the frequencies and strengths of suitably processed vibrational signals to indicate if a hive no longer contains a colony and – furthermore – provide advance warning of colony loss over winter. Such advance warning would alert beekeepers to the need for remedial intervention in order to save colonies that would otherwise be lost over the winter. A weakening colony could be fed using specialised insulated board, merged with another to form a larger joint cluster or superimposed on a strong one to benefit from its heat. Even if remedial action is not deemed commercially justified and so not taken, knowledge of the likely extent of winter colony losses – months before the new season starts – would afford the advantage of forward planning in preparing/purchasing the hardware and colonies required for a successful new season. In any event, we submit that the increased frequency of the nominal 125 Hz line in colonies that go on to fail is of intrinsic – possibly biophysical – interest. We note that neither of the indicators of likely eventual colony failure were present in either the results we obtained following the same processing of the signals from NTU 1 or INRA 3. This suggests wider diagnostic potential, though more work would be required to confirm this.

We have also shown that suitable vibrational data processing allows highly sensitive monitoring of the brood cycle in the vicinity of the sensor. We have explored the minimum data that are required, when frequency information is included, to accurately determine the current point in the brood cycle.
As in the case of the winter losses, in order to minimise the effect of daytime foraging activity of the bees and occasional high amplitude spikes in the data set, for example caused by human intervention in the hive, only vibrational frequency spectra measured between midnight and six a.m. were considered. A histogram of the amplitudes was produced and this then lost any frequency information contained within the data. More than one hundred spectra were available each night resulting in a good estimate of the amplitude most often logged during the night. When a suitable range of vibrational frequencies is considered, the histograms exhibit a single, pronounced maximum, which oscillates with a remarkably regular period, closely matched to, although slightly greater than, that of the worker bee brood cycle. The regular oscillation is disturbed after a primary swarm or when a colony has lost its queen. It is also disturbed prior to summer colony failure. It is absent in the winter time and the phase of the oscillation is frame-dependent. The periodically repeating maximum can be extracted, for example by fitting an analytical function to the distributions and the period of the oscillation quantitated by Fourier transformation of the time series of these coordinates. When shown as a function of the period, the spectra exhibit clear maxima between 21 and 26 days. The ability to determine the position in the brood cycle from a single averaged spectrum has been investigated by means of a simple two step clustering exercise; note that we are now including frequency information as well as the amplitude. Five colonies were selected, on the basis of the clarity and regularity of the oscillations that their distributions exhibit, as candidates to identify an algorithm that would further allow predictive discrimination on the other colonies. Although strict validation of the outcome is not possible (honeycomb loads in the vicinity of the accelerometer are not known), it is useful to explore whether frequency-resolved accelerometer data carries the oscillating information seen in the amplitude data. Spectra were selected at some of the maxima and minima of their respective oscillations, and underwent Principal Component Analysis (PCA). For each spectrum, the corresponding dominant 10 PCA scores were further fed into a Discriminant Function Analysis (DFA) for supervised discrimination based on troughs and peaks. In order to find discriminant functions with generic effectiveness to all five colonies in the data set, the best results were obtained when the PCA scores were collapsed onto three DFA scores, and when measurements from all five colonies in the 'low vibrational amplitude' state were clustered into a single cloud, whilst measurements in the 'high vibrational amplitude' states were clustered into separate clouds for different colonies. When raw spectra, containing both amplitude and frequency information were fed into the algorithm, suitable clustering could be achieved provided the measurement comprised at least 30 minutes of averaging. Numerous features of the vibrational amplitude oscillations that we have highlighted directly point to those of the honeybee brood cycle. These are detailed in the Deliverable 4.1 and 4.2. The honeycomb may be seen as a vibrating substrate, stimulated by sounds and vibrations originating from honeybees in the colony. As the comb content changes, its corresponding transfer function changes too, and in spite of variations in the stimulating signal (evidenced e.g. by the data coming from the wall of the hive) it is the vibrational transmission changes that dominate the modulation of the measured substrate acceleration. The maxima of the amplitude oscillation most probably correspond to the intervals between brood-rearing (when cells in the vicinity of the sensor are empty). This correspondence is consistent with (i) with the observed unusually long high amplitude sections taking place after a primary swarm or a drone laying colony, (ii) the ColEval data, (iii) the period closely matching that of the brood cycle, and finally (iv) Newton's second law. The relatively disappointing correlation with the recorded brood levels may be attributed to errors arising from (i) recordings relying on visual estimates, (ii) recordings referring to an entire frame, whilst our accelerometers are more sensitive to solid structures closer to them, (iii) some measurements being affected by deviations in the source of the measured vibrations, i.e. activities of the bees themselves. The outcome of the vibrational amplitude spectral shape analysis is encouraging, in suggesting that the brood cycle could be monitored using only one hour of night time measurements, in a way less sensitive to drifts than when using amplitude alone, more generic to multiple colonies, and more specific to the brood cycle. The exploitation of the results by other researchers requires no sophisticated numerical analysis. The relevant discriminant curves simply need cross-correlating with the measured averaged spectrum to give a set of 3D coordinates used to compute the distance to the centroid of interest. Therefore, all that is needed by others is the discriminant curves and the coordinates of the centroid(s) of interest.
Improved brood cycle monitoring could be obtained by applying an artificial stimulus to the frame with known amplitude and frequency rather than relying on the bees to generate the vibrations, and potentially allowing further specific sensitivity to brood, honey and pollen to be identified. Abnormal brood cycle may result from diseases, swarming, queen failure, pesticide exposure or lack of room in the hive. In case of a disease, it can be controlled using medicine or beekeeping techniques or destroying the colony; this would be required in the case of American Foulbrood to avoid the disease spreading to other colonies. If the abnormal brood cycle is due to swarming, then the beekeeper will have to check for a new queen and follow the development of the colony. Detecting a queen failure will be useful so that the beekeeper can replace it or introduce the workers of the colony to another one with a queen so as to save the queenless workers. Abnormal brood pattern might also be due to prolonged adverse weather, meaning no flow, meaning queen goes off lay. This happened this year (2015) in the UK.

We have also shown that suitable vibrational data processing allows monitoring of the varroa infestation levels.Each vibrational frequency is multiplied by the corresponding component of a given eigenspectrum to give the principal component (PC) for that frequency and rank of eigenspectrum (we have generated these eigenspectra using the same raw spectral data). These frequency-resolved PCs are usually summed to give the PC score for the order of eigenspectrum concerned, and these scores are then used as input to successive algorithms – to perform discriminant function (DF) analysis (DFA), for example. We begin by regressing on the average of all time-resolved principal component scores (up to the order concerned, dictated – as we later explain – by the number of observations comprising our dataset) over all frequencies up to the point at which the corresponding component of the eigenspectra is not significant. We then
1) use DFA to identify those frequencies with the greatest potential for discriminating between Varroa-infested and clean colonies (with respect to Varroa) and then
2) regress once again on daily-averaged PC scores, but using the scores that have been obtained after concentrating on these frequencies with greatest potential and discarding the rest.
We compare the results of regressing on targeted frequencies in this way with doing so on the complete frequency range covered by our dataset.
We used a generalised linear model for regression assuming either Gaussian or Poisson distributions (a routine from the Matlab statistics toolbox was used and the other distributions it can consider cannot be applied to zero-valued data). We begin by assuming a Gaussian distribution for our initial investigations.
We found a definite positive correlation, as quantified by R2 – here, the coefficient of multiple regression. We use this initially as a guide to the goodness of fit, but comment later on the need to interpret it with caution. In an attempt to improve on this and investigate which frequencies have the greatest potential to discriminate between colonies with and without Varroa present we considered the signals from the extreme cases where no Varroa and very high levels of Varroa were detected, respectively, and fed them to a discriminant analysis routine.
To check that our discriminant analysis algorithm was actually discriminating, we simply re-applied it to the data used to train it – i.e. we performed an a posteriori analysis. The results show a clear demarcation between the colonies having zero and high infestation.
The best correlations with measured Varroa levels were obtained with the daily (overnight) averages of the first three principal component scores considering a frequency range of 140 to 200 Hz. They improved when we considered the averages of a subset of the greatest PC scores rather than of all of those calculated at each three minute interval overnight between data being logged. The optimal number we considered was ten (this equates to 30 minutes of data per day, but it may be that if our averaging time was reduced to 30 seconds similar results would still be obtained but with only 5 minutes of data). At this optimal value when assuming the Varroa levels follow a Gaussian distribution, however, a second marked linear relationship seems to be forming.

We have also shown that suitable vibrational data processing allows monitoring of the colony's intention for swarming.
The details of the numerical method that we have used is disclosed in our patent GB1409807.3 , and in our interim report which also comprises a copy of the patent. It consists of successively applying a unsupervised clustering exercise (Principal Component Analysis, or PCA) followed by a supervised clustering exercise (Discriminant Function Analysis or DFA) on a selection of PCA scores.
In order to give the numerical search the chance to best identify measurements that correspond to (i) a colony intending to swarm and (ii) a colony not intending to swarm, measurements must be supplied as 'training data set' where these two conditions are those that most heavily modulate the spectra. For example, if information about day to day deviations in foraging are removed, the discrimination between 'intending to swarming' and 'not intending to swarm' will be clearer.
* The first choice that proved helpful was to remove day-time data. By focusing on night time data, (i) information about foraging is somewhat removed (although work on the honeycomb that follows a successful foraging afternoon can still be seen late at night, see Deliverable 1.1) and (ii) since the entire colony is back in the hive, all individuals contribute to the collected signal.
* The second choice that proved helpful was to remove spikes in the signal. Occasionally, very high amplitude spectra are acquired, for example due to a visit by the beekeeper or due some object colliding with the hive. These spikes, if accounted for in the numerical search, provide additional new information in the data set, resulting in higher rank PCA scores required to describe the same data, and a weaker discrimination afterwards.
Having removed these affects to our best ability, we gathered the spectra, up to four days before a primary swarm, and labelled them as corresponding to 'intending to swarm'.
Spectra gathered after the last queen pipe of a swarming colony were further labelled as 'not intending to swarm', except for colonies that failed during the summer. Those spectra coming from a failing colony were dismissed, as they carry information about a colony suffering from additional abnormal conditions and would, again, only weaken the outcome of our discrimination exercise.
Spectra gathered at any time of a colony that did not swarm were also considered in the numerical search, as they also contribute to a colony 'not intending to swarm'.
The numerical search that we have performed is entirely self-contained within the matlab® high level programming language that we are using routinely, from the data upload to the swarming signature extraction. The signal processing comprises numerous steps, many of which can be tailored by the user to refine the outcome of the search.
This allows us to run optimisation loops, where a specific parameter along the processing stages is incrementally increased or decreased, until the extracted swarming signature provides the best outcome.
This numerical exercise, which we call 'parametric optimisation', will work provided that we have a scalar quantity which, at the end of a specific processing chain, will provide an estimate of the 'quality' or 'performance' or 'effectiveness' of our swarming alert.
We have called this scalar quantity the 'reliability of the swarming alert', and have computed it in the following manner:
* within the group of colonies under investigation, the threshold dictating whether a measurement results in 'intending to swarm' or 'not intending to swarm' is set so that the percentage of false positives (swarming alerts that are triggered for a colony that is 'not intending to swarm') is very low (usually 0.5 % in most of our work).
* using that threshold, the percentage of true positives (swarming alerts that are triggered for a colony that is 'intending to swarm') within the four nights that precede the primary swarm is then calculated.
The value of that second percentage is called the 'reliability'. The closer the 'reliability' is to 100 %, the more robust the alert.
When/if the 'reliability' actually reaches 100 %,
* the threshold can be re-adjusted, to make the false positives percentage even lower, and the optimisation loop repeated, or
* the number of colonies under investigation can be increased to make the numerical search to output a signature more 'generic' to differing colonies.
Within the optimisation loop, parameters that can be changed independently from each other are:
* the lower frequency of the spectral data below which vibrational data is dismissed (too low will cause irrelevant information to blur the discrimination, too high will remove important information),
* the higher frequency of the spectral data above which vibrational data is dismissed (too high will cause irrelevant information to blur the discrimination, too low will remove important information),
* the range of days, preceding a primary swarm, that are chosen to provide spectra corresponding to the condition 'intending to swarm' (too low will remove important information from the numerical search, too high will incorrectly bring 'not intending to swarm' data into the 'intending to swarm' category),
* whether the spectra are normalised to their maximum amplitude (to remove the contribution of signal strength and make a search purely considering spectral shape) or not,
* whether the covariance matrix (emphasizing spectral information with the highest amplitudes) or the correlation matrix (exploiting spectral information irrespective of its local amplitude) is diagonalised during the PCA search,
* whether the mean spectrum is used solely, or whether the instantaneous 'history' of the spectral changes is also used, e.g. using either the standard deviation, or the spectrum of a limited collection of contiguous spectra,
* the time duration over which the 'history' of the spectral changes must be considered,
* the number of PCA scores contributing to the DFA search (too low will remove important information, too high will bring in 'noise' and blur the discrimination),
* the specific spectra fed to the PCA algorithm to get the PCA scores (this will dictate the relevance of the first PC scores, as discussed earlier),
* the number of DF scores (usually 2 or 3) onto which the selected PCA scores are collapsed,
* the number of categories, or clusters, that the DFA algorithm attempts to achieve.
With regards to this last parameter, the minimum is obviously 2, those clusters corresponding to 'not intending to swarm' and 'intending to swarm'.
However, substantial beneficial outcomes were obtained in our study when more than 2 clusters were attempted. As discussed in Deliverable 4.1 and 4.2 we know that our spectra are strongly modulated by the brood levels of the frame under investigation. In the four days of data considered within the 'intending to swarm' spectra, it may therefore be that one colony under scrutiny will have a 'high brood' frame load whilst another one will have a 'low brood' frame load.
In this scenario, we have found that excellent discrimination to swarming will still be obtainable, if three clusters are set within the DFA search,
* one for 'not intending to swarm',
* one for 'intending to swarm with a low brood level frame load' and
* a third one for 'intending to swarm with a high brood level frame load'.
In the same frame of thinking, using more than three clusters can be beneficial to account either for a broader range of brood levels or for colonies that have some feature within their swarming fever that makes them different from others (e.g. 'varroa infested' and 'not varroa infested ' .
If a single cluster is used for the 'not intending to swarm' state, a simple alert can still be triggered by considering a measurement's Euclidean distance to that cluster, rather than to a cluster corresponding to an 'intending to swarm' state, as there will be multiple such clusters.
Our studies soon showed that instantaneous spectrum analysis is not sufficient to identify a good signature to the swarming fever. Some colonies preparing for the swarming process have a night-time instantaneous spectrum that can in some instances be strictly identical to the night-time instantaneous spectrum of a colony that will not swarm in that season.
The results are therefore all including in one way or another information regarding the 'history' of the overnight changes of the vibrational spectrum during all stages of the analysis.
The outcome of the application of the algorithms is a mixture of promising and disappointing results. With regards to accelerometers placed directly in the honeycomb:
It cannot be pure coincidence that the indicator works so well on hive B2 and UK hive 1 frame 8 (both on season 2014) and also quite well on UK hive 2 central frame. Clearly the pattern identified does also work on other colonies (but not all of them), resulting in an indicator highly specific to the intention for swarming, and absent elsewhere, even on colonies in a separate country, much further North on earth, and residing in a different hive type.
Note, in particular, that our UK hive has more than a year of recorded data and that, due to the distance between their site (Nottingham) and that of the French site (Avignon), must comprise substantially different bees from those residing on the French site. The correct triggering of the primary swarm indicator in frame 8 and in the second colony is therefore truly remarkable, particularly in the case of using the standard deviation of successive spectra in the computation of the alarm.
Unfortunately some false positives and false negatives are also often triggered on other colonies.
The UK colony No 2 exhibits some false positives after the primary swarm (although in a season very far away from the swarming season), the INRA colony A10 does not result in a high indicator (although we have shown that for this particular colony, even considering it on its own does not result in a suitable swarm alarm) and numerous false positives and false negatives are registered on the other frames that were logged on UK colony No. 1 and the indicator does not go up in the 2015 INRA colonies that did swarm.
It is clear from looking at the data in the UK colony 1 (see Deliverable 1.1) that substantial deviations are seen, within a specific colony, in data collected in different frames.
We therefore assume that the incorrect alarms are mainly caused by the strong modulation that the vibrational modes of frames suffer from, when their load changes with honey, brood, or pollen. This is also well corroborated by our study on the brood cycle. We also propose that the vibrational signature for the preparation for swarming is perhaps frame-specific, and that, by chance, in some of our measurements the accelerometer was located in the 'right' frame.
When testing the generic alarm against colonies that have not contributed to its identification, the fact that the indicator works better when using the standard deviation of consecutive spectra rather than their spectra suggests that the one hour regular oscillation highlighted on the 2D DF spectra is perhaps not as generic as originally thought. Clearly, a larger statistical group of measured swarms would help clarify this point. On the INRA site however, the 2D spectra work equally well, suggesting that this one hour long oscillation is perhaps site-dependent or longitude-dependent.
We also analysed the data collected from within the walls of honeybee hives. In this setup, an accelerometer is secured, externally, in the middle of the front face of the broodbox of a hive that may or may not have a super (a smaller box of frames for collecting honey placed above the broodbox).
With regards to the study of the primary swarm alarm measured from within the front face of Dadant hives, we have found that
* normalising all the vibrational spectra to a maximum value of 1 improves the alarm, demonstrating that vibrational strength is not important/central to the swarming signature,
* high quality alarms can be identified in 5 of 12 colonies using a single generic set of DF curves and centroids, with negligible false positives,
* 7 out of the 12 swarming colonies fail to yield alarms prior to the primary swarms,
* by undertaking a separate numerical search on the 7 'difficult' swarming colonies, their alarms can be improved, but correct 'positives' do not necessarily take place in the four days preceding the primary swarm.
o the 2D spectroscopy technique suggests that a very slow oscillation of the vibrational spectra must be monitored to detect the intention for swarming, suggesting that a few (perhaps six), well spaced out (say every 30 minutes), 3-minute samples of the over-night vibrational spectrum would be enough to compute a reliable alarm.
These results are likely to be further improved
* by running a new numerical search in which the training data supplying the 'intending to swarm' state will not necessarily come from the last four days preceding the primary swarm. Instead, the software will pick a collection of a few (say four) days in the two weeks preceding the primary swarm and will systematically explore all possibilities until the best days, yielding the best overall performance of the alarm, will be identified for the 'difficult' colonies. We expect the exploration to converge within less than a month of numerical computation, although a robust matlab® software will first have to be written up.
* by further expanding the time duration (presently three hours) of the night-time data set over which 2D spectroscopy is undertaken.
* by further coarsening the vibrational spectra (presently 10 Hz spectral resolution is used) to remove the wooden box spectral 'signature'.
* by undertaking one or more of the possible 'parametric optimisation' strategies suggested in this Deliverable.
The results look presently promising. In particular: (a) the fact that the colony 'strength' can be removed from the data whilst still providing a good alarm demonstrates that colony 'bee saturation' is not the only parameter to consider when assessing whether a colony will swarm or not.
(b) the fact that the alarm can occasionally be successfully applied on colonies that have not contributed to the numerical search demonstrates the fact that we have identified, for some colonies, a numerical strategy that is likely to work in the future, on more colonies without the need to upgrade the firmware of the monitoring tool.
The results, however, are not promising enough to allow the SME's to launch a reliable-enough product.
In this phase of the work we have demonstrated that the method described in the Swarmonitor patent provides a viable technique for swarming prediction.
This has also shown that a particular hardware arrangement (type and positioning of accelerometer) requires its own set of DFA parameters.
To take this work to a commercial product then requires a commercial decision as to hardware choice and positioning before the specific algorithm can be developed based on this methodology.

Arguably the most important part of any measurement system is the transducer that converts the physical parameter, in our case acceleration, into an electrical signal. The accelerometers used in the monitoring system set up at INRA were single axis B&K accelerometers which cost around $900 (US) each. Such accelerometers are required for a scientific study of the hive vibrations as they provide a linear response over a very wide frequency range and high sensitivity however they are not a viable option for a low cost monitoring system. Most portable computing devices such as tablet computers and smart phones incorporate accelerometers to perform tasks such as rotation of the display orientation when the device is turned. These are much smaller and lower cost but do not have such a linear frequency response. A detailed review of current accelerometer chips available on the market was undertaken and the most promising of these in terms of speed, sensitivity and bandwidth was investigated in detail.
For the analogue to digital conversion and digital signal processing part of the initial prototype we chose the mbed NXP LPC1768 microcontroller as our starting platform. This is a single board device based on the NXP LPC1768, with a 32-bit ARM Cortex-M3 core running at 96MHz. It includes 512KB FLASH memory, 32KB RAM and its interfaces include a built-in USB, SPI, I2C, 12 bit ADC and a real time clock, all of which have been required in the first prototype design. To maximise the use of the ADC input range an amplifier circuit was developed for use between the low cost accelerometer and the mbed. Initially the mbed was programmed to collect the raw ADC data and save it to a flash memory drive. This was tested and shown to work adequately for a capture rate of 4k samples per second which is sufficient for the frequency spectra up to 2kHz. This resulted in very large data files (660MB per day) but it was found that logging for 4 minute periods and then saving to file with a short break gave more consistent results. To progress the ‘on board’ signal processing further, a routine was developed to perform the on-line FFT of the data, and save (every 2 to 4 mins) its average to a single file (rather than the raw data). The software could computes 337,500 spectra over a day and save their partial averages over 4 minutes as a single file with a date stamped file name. This work represented a considerable step towards the final goal however the power consumption for the mbed was significantly higher than is acceptable requiring about 250 mA including a USB drive, and around 150 mA without the USB drive.
The details of the mbed based system represents the development of Swarmonitor prototype A. The approach to the development of prototype A was to reduce risk and ensure that a suitable system was produced in time to allow the data required for developing the predictive algorithms to be collected in time to test them in subsequent years. As such, the choice of the mbed microcontroller that was used was based on existing examples in the developer community that ensured that data collection at the anticipated rates and the ability to deliver FFT code would be possible. The mbed system, whilst relatively power hungry was, at the time, one of the few microcontrollers able to offer the large memory for our requirements. As Swarmonitor progressed we discovered that such high frequency components of the vibrational signals are not a requirement for an adequate swarming algorithm and that the averaged fast Fourier transform (FFT) rather than the raw data is all that must be stored. Mbed based systems were deployed at the INRA and CARI apiaries and provided a significant wealth of data using prototype A. An alternative to the ‘power hungry’ mbed was required for prototype B and this needed to provide sufficient memory and preferably at a reduced price. In early 2014 it became possible to purchase in the UK the Teensy 3.1 microcontroller. This offered more memory than the mbed, much lower power consumption, a smaller foot print and half of the price. The use of the acrylic disc to act as a ‘resonance plate’ and enhance the low cost accelerometer performance in this application proved very successful as part of prototype A. The bees ignore the change of material from wood to acrylic inside the hive and even after a year there is no contamination by propolis or beeswax observed. With the parameters now better defined and new microcontrollers available on the market, prototype B was set to deliver the significant power improvements needed for commercialisation. Microcontrollers such as the mbed and Teensy are designed to be general purpose and are therefore usually plugged into electronic circuits that fulfil a specific application requirement; such circuits are referred to as shields. Various developments of Teensy shield were undertaken with the incorporation of uSD card socket, real time clock crystal and battery backup and a very efficient buck regulator with switch. The revised version of the software captures the data and calculates the FFT as in the previous prototype but now stores the result of the FFT every two minutes and also calculates the standard deviation of the calculated spectra which it also saves to the uSD card. A multi-channel data logger has also been developed to allow a comparison of the results when placing accelerometers in different location in one hive e.g. disc, honeycomb, wall etc. Using a sample rate of 4000 samples per second, after 1024 points (256 ms) the FFT is applied to the data of each channel in turn to determine the frequency spectrum, which is averaged over 4 minute intervals before being logged. For each channel there are two data buffers of 1024 points to enable logging to continue while FFT is performed. The FFT takes 46ms on the Teensy with the highest available clock speed, so we would expect to be able to sample up to 5 channels reliably which was tested and demonstrated to be correct.
A swarming algorithm was developed based on the method described in the patent and using the previous data collected using the mbed and acrylic discs. The Teensy 3.1 code for testing for an alarm condition consists of the following: when a new averaging loop starts the day is checked. If this is not the day on which the swarming algorithm (SA) was last run, it runs the SA, i.e. it runs once a day about midnight. Data was collected for a pre-determined period of ten minutes. The discriminant function (DF) curves are read from the text files on the uSD card. These undergo a point by point multiplication with the normalised (maximum amplitude equal to one) measured spectrum, and the resulting coordinate compared to the threshold value held in the file, centre.txt which has the cloud centre co-ordinates and max distance parameter. The spectrum, DF parameters, distance and decision to raise an alarm or not are then logged to a text file and if appropriate the energising of an output pin that controls the power to a relay activating the SMS text system as described in the next section.
Prototype B was a ‘hybrid’ in that when not processing the swarming algorithm, the code logged data continuously, saving the spectrum every two minutes to a binary data file. This meant that it would require significantly more power than one that only tested for a swarming condition for ten minutes a day. One of the requirements of the Swarmonitor system is to alert the bee keeper when a condition that requires attention has been detected. The most straightforward way to do this is to send a text message (SMS) to a predetermined telephone number. The Swarmonitor SME participants already operate GSM in their existing products and are most likely to integrate Swarmonitor product into their existing GSM systems. For this reason a commercial ‘off the shelf’ GSM card was used to demonstrate the ability of Swarmonitor prototype B to send warning messages and these were incorporated into the hardware tested by the Association volunteers. The hardware chosen was the Arduino GSM shield driven from an Arduino Uno microcontroller. The simple idea was that, for power saving, the Arduino Uno with GSM shield remains powered down unless a text message must be sent and the power line is connected via a relay to a buck regulator for providing the 5V required. The relay is activated by one of the lines on the Teensy at which point the Arduino runs through a predetermined code to send the text message “LOL we are going” to the telephone number in the code and then signals on completion to the Teensy by setting output line 6 high, allowing the Teensy to power it down again. The Arduino Uno with GSM shield draws around 150mA but it only takes around two seconds to actually send the text message. Whilst this worked ‘in the lab’, water ingress compromised the relay on some of the volunteers systems and they did not function correctly.
Power consumption for the Teensy in low power mode is 0.12W and using this mode for 23.8 hours a day requires 2.856 W-hr. The Teensy in measurement mode uses 0.24W and would run for 0.1 hours giving 0.024 W-hr and the Teensy + SMS system uses 1.8W for 0.1hours giving 0.18W-hr. The total daily consumption is 3.06W-hr which could be provided by a 10W solar panel in Nottingham (UK) mounted flat on the roof of a hive in January (such a panel would provide 6.4 W h/day). An alternative would be disposable batteries such as Energizer D cells which give around 18000 mAh and typically can be purchased in bulk for around two Euro each. This would cost around 40 Euro per year in disposable batteries or less if the monitor was not used during the winter months. Clearly there is a trade-off between the initial outlay on equipment for continuous off grid supply and the inconvenience of regular battery changing and both options should be considered where appropriate. so should be adequate for the application of the Teensy 3.1 data logger used at Brackenhurst.
Part of the project aim was a cost effective prototype product with minimal energy consumption that outputs and wirelessly communicates diagnostic information to a remote user. The Teensy 3.1 at 28 euro for one off purchases, forms the processing basis of prototype B and competently handles all the computing requirement even with the multichannel logging system. Power consumption was significantly less than the prototype A based on the mbed and within the range of economical use of disposable batteries. The new low cost accelerometer board on the acrylic disc allows the swarming prediction algorithm previously developed for prototype A to be employed in prototype B but in a much lower power and more cost effective package. The inclusion of the swarming algorithm data on the uSD card allows this to be further refined when improved swarming data becomes available. The use of a commercial board for the sending of the SMS message has demonstrated the potential whilst not restricting it to any specific implementation of this technology All the developments on prototype B can be found in deliverable report 5.2.
One of the most critical observations, and contrary to our understanding before Swarmonitor began, is that night time vibration monitoring provides the most reliable source of data as it contains relevant information but excludes bee keeping activities and foraging activities. The power of our algorithm to distinguish between swarming and non-swarming states is insensitive to the time at which the data is obtained but such daytime activities do represent noise. Furthermore, reducing the averaging duration drastically from the full 6 hour period considered when training the algorithm to 15 minutes when considering standard deviations (and 4 minutes when not doing so) degrades the swarming alarm only minimally. Having established that night time was most appropriate, the length of time used to generate each spectrum investigated using 30 seconds, 3 minutes and 10 minutes. Moderately good agreement in the first eigenspectra for 3 minutes and 10 minutes was observed but 30 was clearly insufficient. The bandwidth of the data required for a good alarm, and hence the speed at which data must be collected, was also considered in deliverable report 4.3. Processor clock speed determines the maximum capture rate however is the clock speed can be reduced, as with managed at the CARI apiary, a significant power saving can be achieved. A swarming algorithm was demonstrated using a bandwidth of only 1000 Hz indicating that significant power gains can be achieved through this technique.

Potential Impact:
The impact of the project so far has been mainly in coverage from the press releases and conference presentations as detailed later in this section. The nature of the project means that only certain times of the year has it been possible to collect ColEval data (spring and summer when the hives can be opened) and crucial data has been collected up to the closing weeks of the project. This data is still undergoing analysis and it is expected that several journal publications and corresponding press releases will follow, including journal articles on swarming prediction, winter losses and levels of Varroa destructor mite infestation. A major publication regarding the sensing of the brood cycle has occurred in the journal PLOS ONE (http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141926).
Immediately after funding has finished, the project is in a phase in which the SME members of the consortium evaluate more closely the information presented in the deliverable reports produced towards the end of the project and decide what aspects, if any, they wish to take forward into products.
A time scale of three months has been set for this process after which the remaining information will be made publically available. This is described in the deliverable report - final plan for dissemination and use of the foreground. Although the project is yet to generate significant socio-economic impact, it is clear that there is good scope for significant impact in this area. The project has provided pioneering methods for assessment that are so far not available to the commercial bee keeper and with the challenges facing the sector, it is likely that more use of technology will become a standard requirement.
When the algorithm for the detection of the swarming intention is improved to a standard suitable for implementation into an existing commercial product such as the CAPAZ or the ARNIA products, there is a high likelihood for significant socio-economic impact as the demand for this feature is very high and the additional cost to the existing products minimal. Such a product will substantially reduce the need for beekeepers to physically inspect their hives in the spring time. Those hives that do not intend to swarm can be visited on a much scarcer rota than those that do intend to swarm, and can be left to develop with minimal disturbance/stress. The impact will therefore be more efficient beekeeping and more healthy colonies. Those hives that do intend to swarm can be dealt with, and, following intervention from the beekeeper, their motivation to keep intending to swarm could be further monitored.
The method and algorithm for the detection of the brood cycle in a frame is of a standard suitable for implementation into an existing commercial product such as the CAPAZ or the ARNIA products, but
• requires the embedding of a sensor into a frame, and
• still requires optimisation when using an inexpensive accelerometer, and
• still requires clear identification of net advantages (test and validate the claim for picking up the queen's 'dynamism' or 'ageing') compared with results using a thermometer sensor.
Such a product would allow beekeepers to act quickly on colonies that could fail due to the absence of the brood cycle or due to an abnormal brood cycle. This information can be helpful to the beekeeper as they can visit the colony to diagnose and solve the problem. Abnormal brood cycle may result from diseases, swarming, queen failure, pesticide exposure or lack of room in the hive. In case of a disease, it can be controlled using medicine or beekeeping techniques or destroying the colony; this would be required in the case of American Foulbrood to avoid the disease spreading to other colonies. This tool also opens new promising opportunities for the testing of toxicological effects at the colony level, enabling long-term, non-invasive observations and bridging the gap between laboratory and field tests. If the abnormal brood cycle is due to swarming, then the beekeeper will have to check for a new queen and follow the development of the colony. Detecting a queen failure will be useful so that the beekeeper can replace it or unite the workers of the colony with another one with a queen so as to save the queenless workers.
Monitoring of the brood cycle is an interesting scientific tool to measure population dynamics of honeybee colonies. As brood development is closely linked to the climate, it can be performed in different regions of the world or of a country to look at differences in brood development. There are 28 different geographical bee subspecies, some of which have a very different brood cycle. Some ecotypes have even been identified to have a brood cycle linked to the blooming of local flowers. Measuring the evolution of the brood cycle of those honeybees is of important evolutionary and ecological interest.
The method and algorithm for the detection of the dangerous deterioration of the colony's condition in the winter is of a standard suitable for implementation into an existing commercial product such as the CAPAZ or the ARNIA products, and will be an exciting additional application when an accelerometer type of sensor is made available in the future in these existing products. The advance warning given to the beekeeper could alert him/her to the need for remedial intervention in order to save colonies that would otherwise be lost over the winter. A weakening colony could be fed using specialised insulated board, merged with another to form a larger joint cluster, or placed above a strong one to benefit from its heat. Even if remedial action is not taken, knowledge of the extent of winter colony losses – months before the new season starts – would afford the advantage of forward planning in preparing/purchasing the hardware and colonies required for a successful new season.
The method and algorithm for the detection of the varroa mite infestation levels requires more work before it is of a standard suitable for implementation into an existing commercial product. When it works, it would allow the targeted intervention of the beekeeper, it would reduce the overall quantities of medication required to treat against varroa infestation, and therefore provide more efficient beekeeping practice.
The cost basis of this needs to represent commercial sense and the development of the microcontroller based systems have considerable merit here.
Media coverage of the Swarmonitor project can be found on the following, non-exhaustive list of websites:

http://www.farminguk.com/news/Honeybees-buzzing-can-shed-light-on-Queen
-s-fertility-study-finds_37852.
html http://www.farmingmonthly.co.uk/farming-environment/10285-honeybees
-buzzing-can-shed-light-on-queens
-fertility-study-finds/
http://www.southwestfarmer.co.uk/news/14110175.HONEYBEES____BUZZING
_CAN_SHED_LIGHT_ON_QUEEN___S_FERTILITY__STUDY_FINDS/?ref=rss
http://www.smallholder.co.uk/news/14110176.HONEYBEES____BUZZING
_CAN_SHED_LIGHT_ON_QUEEN___S_FERTILITY__STUDY_FINDS/?ref=rss
http://www.eurekalert.org/pub_releases/2015-11/asoa-vbt102815.php
http://phys.org/news/2015-11-vibrating-bees-state-hive.html
http://www.middaydaily.com/bees-vibrations-give-clues-about-their-hives/23253/
http://www.newswise.com/articles/view/642201/?sc=rsla
http://www.csmonitor.com/Science/2015/1108/Vibrating-bees-give-dizzying
-clues-about-their-hives
http://www.telegraph.co.uk/gardening/beekeeping/9950045/Listening-to
-bees-buzz-can-help-spot-disease.html
http://www.telegraph.co.uk/science/science-video/9949337/Could-the
-Waggle-dance-hold-the-key-to-bee-health.html
http://www.thetimes.co.uk/tto/environment/wildlife/article3702422.ece
http://www.express.co.uk/comment/columnists/john-ingham/382662/Beekeepers
List of Websites:
www.Swarmonitor.com
THE BEE FARMERS ASSOCIATION LIMITED LBG
8 Olivers Close, West Totton, Southampton, Hampshire
SO40 8FH United Kingdom

Final Report Summary - SWARMONITOR (development of a tool for effective diagnostic monitoring of honey bee colonies.)

Share this page

Download