Results online test #1

version française, cliquez ici

On this page you can find the video and the results of the blind online listening challenge (survey).
This test was online from 08.06.17 until 26.06.2017 and was completed by 226 respondents.
Go to this page to see how the test was presented to the public.


INTRODUCTION

1
In 2014 the Leonardo Guitar Research Project (LGRP) conducted its first study using blind and non-blind playing and listening tests. 10 classical guitars made from non-tropical woods and 5 classical guitars made from tropical woods were assessed, using a number of different methods, by 3 guitarists/3 listeners and 2 audiences (66 listeners). Click here for the research report

2
At the beginning of 2017 a new study was carried out, this time using 44 guitars (22 guitars made from tropical wood and 22 guitars made from non-tropical wood). 20 guitar players were involved in the testing of the instruments.
This second study was designed to re-examine the findings of the first study. On this occassion we chose to (a) use a larger number of guitars and players, (b) execute the tests in 3 different countries, (c) test ‘pairs’ of guitars, one from tropical wood and the other from non-tropical wood, made by the same builder. (This had not been the case in the first study).
Click here for the report.

3
For this online blind test we employed some of the guitars that had been used during the second study. This time, however, the listening test was carried out using an online survey. Over a two week period 226 people participated in the test.

Go to this page to see how the test was presented to the public.


MATERIAL

The audio used for the blind online test:


The survey asked questions such as:
• How many guitars did you perceive ?
• How many of the perceived guitars are made from non-tropical wood
   species
?

a Pdf format of the survey can be seen or downloaded here






The Video
• Number of guitars in the audio/video : 16 
• Number of builders who made the guitars: 8
   Each builder made a pair of guitars consisting of one made from tropical woods (T) and one made
   from non-tropical woods (
NT). The pairs appear consecutively in the track.
• All guitars were made within the same time span, 2015/2016.
• All guitars are of the same model and have the same strings.
• The wood species are variable for parts such as: back/sides, bridge, fingerboard and neck but NOT for the top plate;
   all guitars have European spruce tops and the same bracing pattern.
• Order of the 8 T’s and the 8 NT’s in the track:
   T / NT / T / NT / T / NT / T / NT / T / NT / T / NT / T / NT / T / NT
• The 15 transition time points:
   0:10 / 0:23 / 0:36 / 0:51 /  1:03 / 1:22 / 1:28 / 1:43 / 2:00 /  2:10 / 2:18 / 2:31 / 2:42 /  2:50 / 2:59


The Guitars



All guitars were made in the Centre for Musical Instrument Building (CMB, Belgium) under the guidance of master luthiers and guitar making teachers Walter Verreydt and Karel Dedain. All guitars are of the same model. They all have European spruce tops, the same bracing pattern and the same strings.
These 16 guitars were also used for blind and non-blind playing and listening tests in the LGRP Phase #2 study.
Click here for the results (abstract) of that study.

Recording / Mastering

The musical piece - Tristorosa by H. Villa Lobos - was recorded on each of the 16 guitars.
These recordings were edited into sections of the desired length and then pasted together into one musical piece.
All guitars were recorded with a flat EQ. For the mastering, no audio effects were added.
The guitars were recorded in stereo (X-Y technique) using two cardioid Neumann KMi microphones.
The reverb is natural: large room, 9m long x 6m wide x 5m high.
To minimise the risk of deviation, the position of the left hand, the position of the guitar in relation to the microphones, and the playing style were strictly controlled throughout the 16 recordings.
The player, Gaëlle Solal, is world-class. Her masterful control guarantees consistent and even playing on all the instruments.  http://www.gaelle-solal.com/



Overview of the MAIN RESULTS / detailed results will be published later


Respondents
• Number of respondents: 226

Quality of audio used by the assessors
Question 1: Please check the option, which best describes the audio quality you used for listening.

Answers/results:

Awareness
Question 2: If you had NOT known that there were several guitars in the track, would you have noticed it ? (*this question requires an answer)

Answers:

Perceived guitars
Question 3: How many guitars did you perceive ? (*this question requires an answer)

• Average number of perceived guitars over 226 respondents: 5 (out of 16)


Transition time points
Question 4 : If you perceived more than one guitar, please tell us where the transition(s) between the guitars took place. Note the transition time point(s) (when one guitar follows another) in the box below. For example, if you detected 5 guitars you need to fill in 4 time points, separating them with a slash,  e.g.  0:28 / 0:56 / 2:16 / 2:56  (this question can be skipped)

• 80 respondents (out of 226) provided time points / 146 skipped the question
• 8 respondents failed to provide the information in the form requested
• 72 respondents noted a total of 420 transition time points (average: 5,8 time points out of 15)
• Total number of correctly detected time points over 72 respondents = 312 (average: 4,3 time points out of 15)
   N.B.: a margin of error of 2 sec. before and 2 sec. after the actual second where the transition took place was applied.
• Total number of correctly detected time points excluding the margin of error = 135  (average: 1,8 time points out of 15) 
• Total number of wrongly detected time points out of 420 (i.e; where no transition took place) = 108.

Nature of the guitars (T or NT)

Question 5 : Can you distinguish the tropical wood guitars from the non-tropical guitars? this question requires an answer

Answers:
Question 6 : If you answered YES to question N° 5, please tell us the nature of the guitar (T= tropical; NT=non-tropical) according to the order in which you detected them (for example, if you detected 5 guitars: T, T, NT,T, NT). This question can be skipped.

• 29 of the 226 respondents answered / 197 skipped
• 6 respondents failed to provide the information in the form requested
• 23 respondents provided the information in the form requested
• Total number of identified natures (T or NT) by the 29 respondents = 119 (average: 4)
• 9 identifications correctly matched the nature of a guitar with its corresponding timeslot (transition points)
• 110 identifications failed to correctly match the nature of a guitar with its corresponding timeslot. Either the timeslot was
   correct but the nature was wrong, or the timeslot given encompassed 2 or more guitars.
 


The survey pdf with the raw data, graphs and all comments can be seen or downloaded by clicking here.
(mail addresses of participants are left out)





Supplementary findings concerning the transition time points and the builders

The 16 guitars played on the track alternated throughout according to their nature (T/NT; T/NT etc.) with a total of 15 transition points. Because the guitars were presented in pairs according to their builder, there are also 7 transition points between the different builders' guitars. (These 7 points mark the transition between one builder's NT guitar and the following builder's T guitar). The results show that the average number of transitions detected between the guitars of different builders was higher than the average number of transitions detected between the T and NT guitars of individual builders.

For example; transition 6 (between builder 3 and 4) was detected by 51 respondents. The following transition - between builder 4's two guitars -  was only detected by 3 respondents.
All correctly detected time points were compiled into an excel file which was used to make a graph showing the results per transition time point and guitar builder.



 



CONCLUSIONS

• All 216 respondents perceived combinations of several guitars (including both T’s and NT’s) as being ONE guitar.

• In this test it was very difficult to differentiate one guitar from the other, and virtually impossible to distinguish between
   guitars made from tropical wood species from those made from non-tropical wood species.

• Although several people demonstrated outstanding listening abilities (by indicating 7, 8 to 9 correct transition time points),
   the ability to detect the nature of the guitars was notably less pronounced.

• This test shows that the distinctive sound qualities and the supposed nature of T’s and NT’s were not distinguishable one
   from the other.

• This test implies that neither group (Tropical or Non-Tropical) possesses inherently distinctive, readily identifiable sound
   qualities.
 
• Indeed, as there are clearly more time points detected between T’s and NT’s made by different builders than time points 
   between T’s and NT’s made by a given builder, it would appear that the builder may have a more pronounced effect on
   differences in sound quality than the wood species used for back/sides, bridge, fingerboard and neck.

    We should, however, exercise caution as some respondents indicated in their comments that they were able to detect
    transition points based on "clicks" caused by editing rather than on a perceived difference in sound quality between guitars.
    We are still in the process of analysing whether or not there were more detectable "clicks" or other editing phenomena
    between different builders than between guitars from the same builder.

• Furthermore, if we consider other studies on this subject (see extra info), the question has to be asked as to whether the   
   woods for back/sides, bridge, fingerboard and neck are really as important as has been previously assumed.


DISCUSSION

The discussion part will be build up here.
You can send your input and comments to 
leonardo.guitar.research@cmbpuurs.be
Subscribe to the LGRP Newsletter to stay updated




EXTRA INFORMATION

Click here to read the 71 comments on the test  (before the results were published )

• See the results of the LGRP studies Phase#1 and Phase#2

• All 16 guitars were subjected to tests (acoustic signature) by François Gautier, Professeur en Acoustique/vibrations
   Laboratoire d'Acoustique de L'Université du Maine. The report of that study will be published later.











• A comparable video with 12 steel-string guitars (6 tropical and 6 non-tropical guitars) / see below.

• A video where Gaëlle Solal plays a guitar with back and sides made from 40 layers of newspaper glued together.
   See below. This 'newspaper guitar' was used as a control guitar in the LGRP Phase#2 study. More details will follow.


• A study conducted at Lancaster University by Samuele Carcagno, Roger Bucknall, Jim Woodhouse, Claudia Fritz, and Chris Plack.
   Guitarists' evaluation and discrimination of steel-string acoustic guitars built with back/side woods of varying price, prestige,   
   and sustainability
http://asa.scitation.org/doi/10.1121/1.4988661


• Other complementary online tests, using steel-string guitars, that can be completed HERE !
These online tests are part of an ongoing study on the preferences of guitar players for different woods used for the back and side plates of guitars. These tests are open to both musicians and non-musicians, but guitar players are especially encouraged to take part. These tests may give you an idea of what’s your favourite guitar wood. More importantly, the results of these tests will provide precious data to understand the impact of guitar wood choice on guitar sound quality. This is important from an environmental and economic perspective, because the most prized and expensive woods used in guitar manufacturing (e.g. Brazilian Rosewood, which is an endangered species) are scarce and from unsustainable sources. The results of these tests may allow us to identify woods that are cheap and sustainable, but that don’t compromise on sound quality. These results will be made publicly available on there website at the end of the study.
Please click on the following link to take part : http://www.psych.lancs.ac.uk/hearing/the-guitar-experiment/online-tests/
The more people who participate in different tests with different methods the stronger will be the results.



• Video with 12 steel-string guitars of the same model in one song.
Fabien Degryse plays 'in my solitude", Duke Ellington.
The audio of this video was used to carry out 2 audience tests. One in Rome (It)  and one in Le Mans (Fr).
The public was asked to listen very attentively to the audio. The following questions were then asked:
• Did you notice anything unusual during the track ?
• Who perceived more than one guitar in the recording ?
• How many guitars did you perceive ?
Almost all listeners on both occasions thought that they had been listening to one guitar.


• Gaëlle Solal plays the 'Newspaper Guitar' / Choro N°1 - H. Villa Lobos
This guitar was made by Luthier Walter Verreydt within the framework of the Leonardo Guitar Research Project.
A modern take on an experiment by Antonio de Torres, the back and sides of this guitar were made by gluing together 40 layers of newspaper using  a mold. It seriously raises the question as to how much the materials used for backs and sides contribute to the overall sound of an instrument.




You can send your comments to 
leonardo.guitar.research@cmbpuurs.be
Stay updated by our Newsletter.

Thanks !
 
Ċ
Jacky Walraet,
27 Jun 2017, 23:34
Ċ
Jacky Walraet,
27 Jun 2017, 09:20