Michael McAuliffe

Morphing

The most common way to use STRAIGHT in speech perception experiments is to morph between two similar sound files that differ in a key way.

In this page, we’ll morph between asi (/’ɑsi/) and ashi (/’ɑʃi/) with stress on the initial syllable. The wav files are asi_initialstress.wav and ashi_initialstress.wav, and the pre-analyzed mat files are StrObjasi_initialstress.mat and StrObjashi_initialstress.mat.

Load these files into the morphing menu, either as the mat files, or analyze the wav files.

alt text

There are three steps to creating a continuum:

  1. Alignment of the two files
  2. Setting up morphing rate
  3. Generating the continuum

Alignment of the two files

To begin with, once the sound files are loaded and analyzed, press the “Open anchoring interface” button to specify the alignment between the two. For details on how to perform various actions in the anchoring interface, please see here.

alt text

This will open a window with the two waveforms and spectrograms (asi on top as “Speaker A” and ashi on bottom as “Speaker B”).

alt text

Next, click on the “Calculate distance matrix” button, which will compare each frame of audio from one file to each frame in the other.

In the distance matrix on the right side, dark blue represents small distances (very similar frames of audio) and dark red represents large distances (very different frames of audio). In general, you’ll notice stripe or checkboard patterns, and the boundaries of these stripes correspond roughly to segment boundaries.

alt text

The initial alignment is the straight line across the two files. In general, this should be pretty close to the path we’ll eventually choose, because we want the sound files we’re morphing to be as similar as possible. In this case, I’ve produced the sounds with the same stress pattern and similar timings, though /s/ in general tends to be longer than /ʃ/.

The temporal anchors should be placed such that they line up to points in the sound file that have the same “meaning” in both files. So minimally there should be an anchor for the onset of the /ɑ/, the boundary between the /ɑ/ and the sibilant, the boundary between the sibilant and the /i/, and the offset of the /i/.

alt text

We also want to make sure that there are frequency anchors for the sibilant and the formant transitions, to make those more natural. Without frequency anchors, any differences will be mixed linearly in the frequency domain, similar to what you would get if you made the stimuli in Praat by extracting the /s/ and /ʃ/ and mixing them at different percentages. Instead we can shift the spectral peak around in STRAIGHT. The differences between mixing and shifting are not super pronounced when synthesizing sibilant continua, but make a huge difference in synthesizing vowel continua (shifting formants around).

alt text

Once you have an anchor in the middle of the sound to be morphed, we want to anchor the peaks in the frequency domain. To do so, hold alt and click on the point in the center of the sibilants (the cursor will turn to a pointing hand over the anchor). This will bring up a display of the spectra for the two wave files at that point in time.

alt text

In this case we’ve aligned the peaks of the sibilants, so that they’ll get shifted around when morphing, rather than averaged. See the interface controls page for more details on how to interact with frequency anchors.

Once you’re happy with the frequency anchors, press the “Update frequency anchors” button in the lower right of the window, and that will save what you have anchored and pass it back to the main anchoring dialog.

Once all the anchors are set up to your satisfaction, you can press the “Set up anchors” button to send the work back to the main morphing dialog. I also recommend saving the anchors via the “Save morphing substrate” button, like the asi-ashi_initialstress.mat file.

Setting up morphing rate

Once anchoring is finished, or the morphing substrate mat file from a previous session is loaded, you’ll have to set up the two end points of the continuum you want to generate.

To do so, first press the “Initialize morphing rate” button and then the “Edit morphing rate” button, which will open the morphing rate dialog.

alt text

In the dialog window, there are a lot of options for what can be done. For the continuum we’re creating, we’re going to be ignoring most everything here, but see the morphing rate interface page for more details on what aspects of it do.

alt text

What we want to do is create a substrate that is wholly “A” (i.e. asi) and another substrate that is wholly “B” (i.e. ashi). To do so, click on one of the points in the “Spec” figure (the blue boxes and lines should turn green) and drag it all the way up to the “A” side.

alt text

Once that’s done, press the “Set up morphing rate” button on the right to send it back to the main dialog. Click the “Save morphing substrate” button to save the A subtrate as a mat file, like asi-ashi_initialstress_A.mat.

alt text

Once the A substrate is saved, we’ll make the B substrate by clicking the “Edit morphing rate” button, and then clicking and dragging a point in the “Spec” figure (along with all the rest of the points) all the way to the B side.

alt text

Once more, click the “Set up morphing rate” button to send it back to the main morphing window, and then click the “Save morphing substrate” button to save the B subtrate as a mat file, like asi-ashi_initialstress_B.mat.

Generating the continuum

Once the end points of the continuum are all set up, we’re ready to generate the steps. From the main morphing menu, click the “Continuum” button at the bottom right of the menu.

alt text

This brings up a dialog to generate the continuum. You can edit where the continuum will be save here, but I find it easier just to use the defaults and move the continuum to a backed up location once you’re happy with the results. You might have to go back and edit the anchors a couple of times and STRAIGHT is not good about remembering paths you’ve entered before. You can also change the number of the steps that will be generated if you want more or less than the default 11. I do recommend changing the file name root to something more informative than “stimulus”, in this case I used “asi-ashi_initialstress”.

alt text

When you click the “Generate continuum” button, it will prompt you to find the A substrate mat file that we saved earlier, followed by the B substrate mat file. Once both have been selected, it will generate the continuum (and play them on top of each other). To actually get a sense of how well they turned out, I would recommend opening them in Praat and playing around with them there. The continuum steps that I’ve generated is below.

Step File Step File Step File
1 asi-ashi_initialstress001.wav 2 asi-ashi_initialstress002.wav 3 asi-ashi_initialstress003.wav
4 asi-ashi_initialstress004.wav 5 asi-ashi_initialstress005.wav 6 asi-ashi_initialstress006.wav
7 asi-ashi_initialstress007.wav 8 asi-ashi_initialstress008.wav 9 asi-ashi_initialstress009.wav
10 asi-ashi_initialstress010.wav 11 asi-ashi_initialstress011.wav    

I’ve written a batch script in Matlab for generating lots of continua given a folder with *A.mat and corresponding *B.mat substrates. If you’re generating a lot of continua, this might be faster and more flexible than using the dialog for each one. The script is available here