Electronically aided violin practice / PAR 4/2017 / 2017 / Archiwum / Strona główna | PAR Pomiary - Automatyka - Robotyka

Pełen tekst

(1)Pomiary Automatyka Robotyka, ISSN 1427-9126, R. 21, Nr 4/2017, 83–87, DOI: 10.14313/PAR_226/83. ; ! & " % %-%*% ? " # @ * '; % * $--&0$+$ . 1 % " # B 'I %a OJ-+0,+& . &%&)The project described in the paper rely on creating a small device mounted under violin strings between a fingerboard and a bridge to help beginning violinists in instrument practice on its first stages of music education. This device helps to identify mistakes of wrong intonation and wrong bowing. It notifies the mistakes to the violinist via acoustical signalisation. The working principle of the device, some technical details of its construction and first practical tests results are presented in the paper. The proposition of the development of the prototype device is described. 8 X ! * " " . 1. Introduction Regular musical education in the music school is usually connected with playing an instrument. No matter what instrument, first steps in this field are connected with proper sound emission. Depending on the kind of instrument, the way of sound generation is different and a proper sound emission poses to the player specific challenge. As in many other aspects of our life, first steps in some new activity can influence further development very much, so gaining proper habits just from very beginning is very important. Best solution to this problem is playing the new instrument only under control of a master – music teacher. Of course it is not always possible. So we propose electronically aided instrument practice. In this paper we will look at the problems connected especially with playing violin. With some modification the described device can be used with all strings. The violin is a bowed string instrument held between a chin and shoulder. Its left side props left side of chin on chin-rest (placed at the left side of tailpiece, on the edge of violin) and usually a rib is used (putted on the back on lower bout of violin). The player’s left hand has to embrace neck of the violin between thumb and index finger without squeezing it on its bottom side in such a way that there is a clearance. The spine of violinist should be straight, shoulders situated naturally and legs in little straddle position. In this place we should say, that the playing position described above is not very natural for human body. Playing violin take place with a bow held in right hand or plucking the strings with right hand fingers (pizzicato). Left hand fingers press strings to the fingerboard in appropri-. # & *:X B 8 ",$JD,)% % #) ' ,-%&D%,&+-% &.%++%,&+-% ! "" # $%&. ate places to shorten the string and the tune the instrument for various notes. Playing violin is a complicated process of the synchronous cooperation of human muscles, auditory system and mind [2]. Violinist has to concentrate on drawing a bow across strings correctly, having good stature, playing in tune and often on reading notes. Beginning violinist has problem with coordinating of all those actions in the same time. Most important things are proper bow drawing (right hand) and good intonation of notes (left hand). Bow should be drawn perpendicularly across the strings what is obtained by moves of the elbow and wrist when shoulder moves only up and down while changing the string. Wrong technique (which is however natural for human body) is to rigidify elbow trying to move bow mainly by moves of arm [1]. At the first stages of musical education all these components are not developed and need education. The main role in the process of education plays the master-teacher, who controls the pupil’s efforts, shows the ways of development and point out the mistakes [11]. Unfortunately learning violin (like other instruments) is not only spending time on classes with teacher but mainly on hours spent alone or with inexperienced parent on daily practice routines. And during this homework a very common situation is, that student can not realise that there is something wrong while practicing. The own practicing routines at home are dangerous especially for beginners in gaining wrong habits because of the lack of control from the master side. In these moments modern technology can help. We came up with an idea to create a small electronic appliance – Electronic Teacher – which can help in unassisted learning. During the exercises with Electronic Teacher the student may get information about committed mistakes and refine his technique of playing without a risk of remembering mistakes undetectable for her/him on that level of studying. The regular use of the new device can tellingly rev up a processes of learning music. In the paper the general idea of the device is presented. The device shall generate acoustical and visual signals providing information about the playing quality in two main aspects: bowing quality and intonation. These two aspects are analysed and signalled independently. The paper is aimed at presenting of new approach to getting these skills. The intonation con-. 83.

(2) Electronically aided violin practice. {.}

(3)

(4) . trol is supported by giving the player reference tones and this approach is presented in the second part of the paper. The bowing supervision is based on the use of matrix of optoelectronic sensors. This idea will be described in the third part of the paper. The prototype of the device was presented to the student of Primary National Music School in Szczecin, Poland. Basing on these tests some improvements and modifications are introduced, what is presented in the fourth part. Finally, general concept of microcontroller based mobile system is presented.. Straight bowing, which is keeping the bow perpendicular to the string and parallel to the bridge of the violin while playing, is considered as a basic skill to learn by the novice violin player. Firstly let us look at the proper rules of bow moving. When the player is moving the bow on the string, the angle between them must be equal nearly to 90 degrees. Movement the bow left and right is acceptable if aforementioned angle remains still equal to 90 degrees. Angle between the surface of violin deck and the bow depends on the played string and can change during playing. The good skills in proper bow moving has a major consequences for the achievement of next mastery levels of violin playing. Initially, we were trying to use MEMS (Micro-Electro-Mechanical-System) sensors – 3-axis analog output accelerometers type ADXL335 [5] and 2-axis analog output gyroscopes type LPY550AL [6] placed on the bow, to follow the motion of it. The frequency analyse of acquired data proved that the motion of a bow had generated 3 main frequency component groups: very low frequency motion components, low frequency bow vibration components and middle frequency sound components. The motion components were difficult to acquire with sufficient accuracy to retrieve the bow 3D trajectory with simple hardware. The sensor placement directly to the bow was also inconvenient because of changing the bow mechanical balance. Thus, the dynamic detection of the bowing quality using cinematic sensors was very difficult and we decided to find another solution. We decided to follow the position of the bow in reference to the strings using a static matrix of optoelectronic sensors placed on the violin below the strings, what is shown in fig. 1. Parallel analysis of the signals obtained from the sensors gave us the information of the position of the bow. The dimensions of the sensor matrix (separation of the sensors in rows and columns) was elaborated taking into account the strings separation, the bow width and required angular resolution. In the first version we assumed that there are three quality levels which specify the correctness of the bow position: ‘cor-. Y.7

(5) &&. The proper intonation is the effect of control process in closed feedback loop consisting of the mechanical system of playing person, her/his auditory system and the brain regions responsible of the musical consciousness. At the first stages of musical education all these components are not developed and need education. A special interest have to be put during home practice routines. In the case of intonation control the widely used electronic instruments are called chromatic tuners. They usually show the intonation shift in cent scale (1 cent = 1/100 of semitone) on graphic indicator. This is, however, the main disadvantage because they may get the pupil the habit of visual control of intonation, but it should be exclusively by hearing sense. On the other hand, the young musician often doesn’t know, if the played notes have correct intonation, because he/ she is not familiar with sound phenomena that indicate the bad intonation. Assuming the student possesses a relative musical hearing, the intonation control could be more effective if the student will be able to constantly compare by ear the pitch of played notes with the acoustic pattern. In addition, depending on the progress, this pattern may be the note of the same nominal pitch as played at a given time or a note, which remains in a particular interval relationship (third, fifth, etc.) [1]. The above idea accomplishes the control intonation part of the presented system. The final hardware is going to based on standard microcontroller system equipped with analog to digital and digital to analog conversion and small display. The system will be complemented with front-end analog part allowing direct connection of microphone and earphone. The block diagram of the system is shown in fig. 7 and will be described later. The algorithm of the system operation was tested in the MATLAB environment and is as follows. The signal from the instrument is acquired by the microphone. After preamplification and denoising the signal is analysed in the frequency domain and the fundamental frequency of the note is estimated. The value of the instantaneous frequency is compared with the frequency database of the notes in equal temperament system. As the result of the comparison, the tone with nominal pitch is emitted. The player has the opportunity to correct the position of the finger on the string based on what she/he hears. Achieved are therefore two objectives: the student learns the correct position of the fingers on the strings and at the same time, develops the sense of hearing - learns to respond to the acoustic phenomena, which are characteristic for the correct and incorrect intonation. The created application can also visualise intonation deviations on the display screen, what may be needed in the beginning stage, but should be avoided as soon as possible. The application also provides the emission of pattern tones, which can stay in the relation of any interval to the played note. This can help to develop the auditory sensitivity to correct the intonation on the base of other intervals than an unison.. 84. P. O. M. I. A. R. Y. •. A. U. T. O. M. Fig. 1. Presentation of positions of fiddlestick and matrix of sensors Rys. 1. Przedstawienie różnych pozycji smyczka i położenia matrycy czujników. A. T. Y. K. A. •. R. O. B. O. T. Y. K. A. N R 4 /201 7.

(6) /

(7) B*

(8) m' rect’ – angle deviation less than about 10 degrees, ‘little incorrect’ – angle deviation between about 10 and 20 degrees, ‘very incorrect’ – angle deviation more than about 20 degrees. In the figure 1. these levels are presented in the illustrative manner. The position of the bow is detected by a matrix of 9 (3 × 3) transoptors (type KTIR0711S). Each of them includes an infrared electroluminescent diode and a photo-transistor. To power up the sensor and obtain the voltage output signal from it, very simple electronic circuit is needed [7]. Realized matrix of sensors is presented in the fig. 2.. (2) It is obvious that high accuracy was not necessary, although the number of possible combination drastically increase. In this way, manually creation of table of truth and control algorithm was impossible. Thus we introduced an extra stage of system learning. In this level, the person who had good playing skills (a teacher for example) was moving the bow properly. Simultaneously, the algorithm was creating a database of ‘correct’ positions. Next, the same person started to move the bow ‘little incorrect’. The database of this positions were created by system. The same operation was repeated for ‘very incorrect’ positions. Collected data were treated to eliminate ambiguities and were then used in the main algorithm to assess bowing quality.. ^.|

(9)

(10) &4. Fig. 2. Realised matrix of sensors Rys. 2. Widok zrealizowanej matrycy czujników. When there is no object above the sensor, the output voltage is near to positive supply voltage. When an object appears above the sensor, the output voltage will start to decrease. Collector current (and output voltage) is depended of the distance between object and sensor and its reflectance. During the first stage of the system development the output voltage was measured by National Instruments Data Acquisition Card (DAQ) type NI – USB 6251. Every sensor was connected to one Analog Input (AI) of the DAQ. Its resolution was 16 bit and maximum sampling frequency 1.25 MHz [8]. Discretised values of output voltage from all sensors were received by the DAQ and send to the MATLAB environment. We considered two possibilities of analysing collected data. First approach was easier and assumed that the information from sensor is restricted only to two levels. We introduced a threshold to classify only two states: ‘bow is present’ – 1, and ‘bow is absent’ – 0. Next, analysing the matrix pattern, we created a table of truth, which described which positions are ‘correct’, which are ‘little incorrect’ and which are ‘very incorrect’. The number of sensors is 9, thus the number of possible combinations is calculated in equation (1). N = 29 = 512. Described device has been tested in the laboratory. The prototype was created based on data acquisition card and a PC. In tests a schoolgirl of primary music school has participated. The primary purpose of the tests was to estimate the ease of operation and ergonomics of the new device. In order to improve the functionality of the intonation-support-part, the simple sinusoidal tones were replaced with the synthetic violins sounds. Such pattern significantly facilitate orientation in the pitch imperfections compared with a pure sine tone. This confirms the role of the higher harmonics contained in the signal in the process of tuning to the correct intonation. So pure sine tone generator was replaced with wavetable synthesis. The part of the device responsible for the bowing quality signalization also required corrections. Although the sensors. Fig. 3. Examples of data classified as ‘impossible’ position of fiddlestick Rys. 3. Przykłady danych zaklasyfikowanych jako ‘niemożliwe’ położenie smyczka. Fig. 4. Examples of data classified as ‘correct’ position of fiddlestick Rys. 4. Przykłady danych zaklasyfikowanych jako prawidłowe położenie smyczka. (1). It is obvious that not all combinations are possible. In the figures 3–6 the examples of data pattern classified as: ‘impossible’, ’correct’, ’little incorrect’ and ‘very incorrect’ are shown. Data is presented in matrix form of dimensions 3 × 3 and every element could be only ‘1’ or ‘0’ (rows are parallel to violin strings, and columns parallel to the properly guided bow). Creation of table of truth and control algorithm was a result of theoretical assumptions and the diameters of bow and matrix of sensors. All conditions are fixed (constant). There was no the system learning stage in this approach. Second approach was to analyse the data more accurate. The analog output voltages from sensors varied from 0 V to 3 V during playing. The resolution of analog to digital converter (ADC) was 16 bits, thus it was possible to detect variable of output voltage given by equation (2).. Fig. 5. Examples of data classified as ‘little incorrect’ position of fiddlestick Rys. 5. Przykłady danych zaklasyfikowanych jako częściowo nieprawidłowe położenie smyczka. Fig. 6. Examples of data classified as ‘very incorrect’ position of fiddlestick Rys. 6. Przykłady danych zaklasyfikowanych jako całkowicie nieprawidłowe położenie smyczka. 85.

(11) Electronically aided violin practice were working in infrared range the impact of kind of light in the room on them was considerable. It was the reason to set properly value of threshold before start to use the device. It is possible to create an automatically self-calibration procedure. Another important question was selection the properly method to signal the position of bow. We had two options: visual or acoustic. As we know, hearing is heavily used by musician, thus our first approach was to present the information as graphical symbol in the display. Experiments with the musicians proved that the continuing need observation of display is very uncomfortable for them. It was the reason to try to use the acoustic signalization. The frequency and spectrum of signal were crucial. It had be completely different from sound generated by violin. After series of experiments, we decided to use simple sinusoidal signal of 800 Hz frequency. Signals from violin and system not interfered with each other. Algorithm written in MATLAB obtain detection of bow position every 0.5 second. This value was obtained after a series of experiments. After detection, the reaction of system is dependent of obtained position. If the position of bow is ‘correct’ none acoustic signal is generated. If the position is ‘little incorrect’ the slow intermittent beep is generated, and if the position is classified as ‘very incorrect’ the fast intermittent beep is generated. In this form the system is ready to build-in into a small, microcontroller based system with own source of sound.. Fig. 7. General concept of microcontroller based mobile system Rys. 7. Ogólna koncepcja mobilnego system mikroprocesorowego. operational amplifiers. Next, gained and filtered signal from microphone is converted to discrete-time domain by analog to digital converter (ADC 1). The sampling rate is set by the timer 2 (TIM 2). Samples are transferred to operational memory (RAM) by the Direct Memory Access module (DMA1) – channel 1 is used. Data processing algorithms (mainly 1024point FFT) are carried on collected samples by core with floating point unit (CPU+FPU). In the program memory (FLASH) the pattern table of tones is placed. This discrete values are transferred to digital to analog converter (DAC) via DMA 2 module with channel 4. DAC is timing by timer 4 (TIM 4) which set the sampling frequency. The small external power amplifier(Power Amp) is necessary to drive earphone. Data from matrix of sensors are obtain by sampling all 9 sensors by the 9 multiplexed inputs of the analog to digital converter 2 (ADC 2). Low sampling frequency is controlled by timer 3 (TIM 3). Role of DMA 2 module with channel 1 is to transfer data from ADC 2 to RAM memory. Core is responsible of data analyse. Signalisation of bad fiddlestick moving is realised by DAC, power amplifier and earphone.. Z.4& ,& & &

(12) 4

(13) Described experiments have been conducted on prototype system including: data acquisition card, matrix of sensors, external audio amplifier and PC class computer with MATLAB software. It is obvious that practical use of this system will be very uncomfortable. Therefore, we created a concept of microcontroller based, mobile device. Discussed solutions of bowing quality assessment and intonation control support, impose relative high requirements for used microprocessor system. System should simultaneously take data from matrix of sensors, acquire audio signals from microphone, process both streams of data, and communicate with musician. All of this operations must be in real-time executed. To reduce cost of device and decrease time of operations, almost system components should be included inside the microcontroller. As we can see, used microcontroller must have internal analog to digital (ADC) and digital to analog (DAC) converters, and high-performance core to data processing (e.g. Fast Fourier Transform (FFT) computing and learning algorithms have high computing cost [9,10]). All of this components are included in STM32F303 microcontroller, which we chose to release mobile system. It includes 4, 12-bit ADCs, 3 channels of 12-bit DAC and direct memory access module (DMA) [4]. Advanced RISC (Reduced Instruction Set Computing) Machine (ARM) architecture based core, cooperates with floating point unit (FPU) and radically increases the floating point calculations used in FFT. Reverse bit operation, useful in FFT calculation to divide samples in even and odds are hardware supported (RBIT instruction) by this core [3]. Another important aspect is present of programmable gain amplifiers (PGA) inside the microcontroller, which can cooperate with small condenser microphone (preamplifier and anti-aliasing filter). Use of DMA to transfer data form input to memory, and from memory to output without core activity is very favourable. Table of tones can be placed in flash memory (128 kB). If it will be not enough, table could be placed in external Secure Digital (SD) card. General concept of described system is presented in figure 7. Condenser microphone is connected to preamplifier antialiasing filter (Preamp Filter) based on internal microcontrollers. 86. P. O. M. I. A. R. Y. •. A. U. T. O. M. [.

(14) This article presents the concept and the results of prototype tests of the electronic device meeting the role of the electronic teacher who helps the self-development of young violinist. The presented idea of intonation tuner and optoelectronic bowing qualifier seems to work very well and really helps violin beginners. During the design phase of the device we put special attention to the way of error signalization to the user. Chosen acoustic signalling guarantees the student full concentration only on sense of hearing, which is most important for every musician. In addition, it eliminates the need to use of any display, what makes the miniaturization of the device easier. Modern microprocessor technology allows to integrate the device into a small item that can be put on an upper violin deck under strings. Thus the balance of the bow is not impaired. To communicate with the user only lightweight earphone is needed.. 1. van der Linden J., Schoonderwaldt E., Bird J., Johnson R., MusicJacket – combining motion capture and vibrotactile feedback to teach violin bowing. “IEEE Transactions on Instrumentation and Measurement”, Vol. 60, Iss. 1, 2011, 104–113, DOI: 10.1109/TIM.2010.2065770. 2. Schoonderwaldt E., Mechanics and acoustics of violin bowing: Freedom constraints and control in performance, KTHSchool Comput. Sci. Commun., 2009. A. T. Y. K. A. •. R. O. B. O. T. Y. K. A. N R 4 /201 7.

(15) /

(16) B*

(17) m' 3. PM0214, Programming manual, STM32F3, STM32F4 and STM32L4 Series Cortex®-M4 programming manual 4. Cortex-M4 Technical Reference Manual 5. Analog Devices Inc., Small, Low Power, 3-Axis ±3 g Accelerometer ADXL335, datasheet ,2009 6. STMicroelectronic, LPY550AL MEMS motion sensor: dual axis pitch and yaw ±500°/s analog output gyroscope, datasheet ID15808, 2009 7. Knigbright, KTIR0711S - SUBMINIATURE, HIGH SENSITIVITY PHOTOINTERRUPTER, datasheet. 8. National Instruments, NI 6251 Device Specifications, 2015 9. Lyons G.R., Understanding digital signal processing, third edition, Pearson Education, Inc.,2011 10. Rabiner L.R., Gold B., Theory and Application of Digital Signal Processing, Prentice Hall, 1975. 11. Galamian I., Sally T., Principles of violin playing and teaching, Courier Corp., 2013.. # "*Q * 0&%)W artykule opisano projekt i realizację prototypu miniaturowego urządzenia umieszczanego pod strunami skrzypiec, pomagającego w nauce gry na tym instrumencie podczas pierwszych etapów edukacji muzycznej. Opisywane urządzenie pełni podwójną rolę: wykrywa i sygnalizuje błędne ruchy smyczka oraz pomaga skorygować niewłaściwą intonację. W artykule przedstawiono podstawowe zasady nauki gry na skrzypcach, które były podstawą opracowania konstrukcji urządzenia. Zaprezentowano także ważniejsze szczegóły konstrukcyjne układu oraz zaproponowano koncepcję rozwoju prototypu. &

(18)

(19) XQ "* " 9 . !%*% ? . & " % %. ",$JD,)% %. %" )% %. /9 ' ; * " * # @ * * %@ Y " ,&+C% "*Y ,&+.%R9 0 E UY "F *F ; ' ; " " 0 * # @ * * % Q F" 0 "F " " * " " 0 *F%. /9 ' ; 0 * V"* Y%+DDOY%+DDDX ' U" * / "B V"*,&&&X%/ E UY "F *F; ';#@%0 Q FY P%R9 Q0 Y Q Q Q 9 F %. 1 % % ")*"% " /9 B U "%@ 0 * * 6U( " R*F Q * "% 70 * % R9 0 ' I " * # B 0 * %. 87.

(20) NR 3/2015. 88. P. O. M. I. A. R. Y. •. A. U. T. O. M. A. T. Y. K. A. •. R. O. B. O. T. Y. K. A. N R 4 /201 7.

(21)