Challenge: Identifying Rhythmic Properties -
Beat-like-ness

In this video we will learn about one component of the prosodic dimension of gestures namely the rhythmic properties of gesture, focusing specifically on beat-like-ness. First, we will define what we mean by beat-like-ness. Then, we will explore a few examples together. Finally, we will describe how to annotate beat-like-ness.

The prosodic dimension: Rhythmic properties of gestures

Remember when we mentioned the prosodic dimension of gesture, we used the term “prosody” in a general sense to mean the raw organizational structure, that is, how phenomena may be grouped together at various levels, and how certain elements may be given additional prominence. Gestural prominence would generally refer to how salient a gestural movement is, and could be determined by a number of aspects of gesture, including kinematic measures such as velocity or acceleration, spatial aspects such as the occurrence is gesture space or size of the gestural excursion, or finally a concept we call “beat-like-ness”. 

To understand the concept of beat-like-ness, it is useful to have a little background knowledge. McNeill (1992) describes beat gestures as simple up-and-down gestures that align with prominence in speech. However, this type of gesture was largely defined in terms of its simplistic form and its direct relationship with speech prosody. Yet it is important to remember that gestures of all types tend to associate with prominence in speech, and beat gestures can also have more complex phasing and form than simple up and down movements. This is why in M3D, we propose assessing a gesture’s referentiality independent of its rhythmic qualities. Importantly, McNeill also described the work of Kevin Tuite, who proposed that there is an underlying “rhythmic pulse” that surfaces both in speech (in the form of prominent syllables) and in gesture (where gesture is produced to align with this pulse). Further, Tuite says that “beat” gestures are simply the gestural manifestation of this rhythmic pulse without any additional referential meaning superimposed on the gesture. Finally, McNeill also talks about “superimposed beats” where a clearly referential gesture (such as reading a book) can be produced with an additional “bounce”. For example, I could say something like “He was reading a book” and if my interlocutor were to react by saying “I thought it was a magazine”, I could then respond by saying “No, a book” and [speaker produced a superimposed beat].    

Beat-like-ness

Taking this as our starting point, we can now describe what we call in M3D beat-like-ness. Beat-like-ness is the assessment to which a gesture seems to be a prominence-lending movement. Another way of thinking about this, is to consider the degree to which Tuite’s “rhythmic pulse” surfaces in the gesture. M3D initially proposes that beat-like-ness be evaluated on a 3 point perceptual scale. First, a score of 0 would correspond to gestures that are not beat-like at all. That is, there is no “rhythmic pulse” that surfaces at all in the gesture. Let’s take a look at a gesture that conveys no beat-like-ness. [see example]. A score of 1 would correspond to a slightly beat-like movement. This could correspond to gestural movements that have sort of sudden stops or turns which seem to potentially align with a “rhythmic beat”. Here’s an example of a slightly beat-like movement [see example]. We see that the speaker’s hands move outwards and seem to stop at a fixed point, seeming to align with an underlying rhythmic pulse. However, the movement itself does not seem to be lending any sort of particular prominence to that rhythmic pulse. A strong beat-like movement would receive a score of 2. A strong beat-like movement would correspond to a typical McNeillian “beat” gesture, where the gesture clearly reflects the notion of an underlying rhythmic pulse and the gesture is highlighting that pulse as prominent. Here is an example of a speaker producing three non-referential gestures with a beat-like score of two. [see example]. 

There are two important things to remember. First, beat-like-ness is a measure of how much a rhythmic beat surfaces in the gesture. As such, other kinematic factors such as velocity or the size of a gesture are not accounted for in this score. For example, the gestures in this video [see example] and the ones in this video [see example] should both be considered “strong beat-like gestures” and receive a score of two, even though one may seem to be more “emphatic” or exaggerated than the other. A rhythmic beat or pulse clearly surfaces in the gestures of both videos. 

Second, remember that beat-like-ness is independent of referentiality. Referential gestures can also show varying degrees of beat-like-ness as well. For example, here, the speaker is talking about the importance of “having a sense of scale” and produces a weakly beat-like referential gesture [see example]. This gesture stroke would receive a score of one, weakly beat-like, because the gesture seems to convey a rhythmic pulse at the end-points, but is not necessarily highlighting or marking the pulse as prominent. Here is another example where the speaker is talking about inertia and produces a single referential gesture [see example]. A rhythmic pulse indeed surfaces in the production of this gesture stroke, so it should be given a score of two, strongly beat-like. Such examples would also include what McNeill described as “superimposed beats” or referential gestures that contain additional “bounces”, such as in this example, when the speaker says “really focused on the poles” [see example]. Finally, it should be noted that the audio is only present in these examples so as to highlight the referentiality of the gestures. But as you will see now, we usually annotate beat-like-ness without audio. 

How to annotate Beat-like-ness

So how do we annotate beat-like-ness? We generally annotate beat-like-ness after having annotated gesture phasing. In this way, the annotation for beat-like-ness will overlap with the annotation of the stroke. Beat-like-ness should be assessed without the audio, as we want to avoid bias from hearing the speech signal. Play the gesture at full speed, and assess whether the gesture is not beat-like at all, weakly beat-like, strongly beat-like or extremely beat-like. Once you have decided, select the stroke and create a new annotation on the beat-like-ness tier that perfectly aligns with the stroke annotation of the same gesture. You may then select the level of beat-like-ness from the drop-down menu that appears when you create the blank annotation. 

In this video, we’ve explored another aspect of the prosodic dimension of gesture, specifically beat-like-ness. Remember: Beat-like-ness refers to the assessment of how much gesture seems to be a prominence-lending movement. We can think of this as how strongly an “underlying rhythmic pulse” surfaces in the movement. There are 3 different levels of increasing beat-like-ness. It should be annotated without audio to avoid bias from speech. Thanks for watching!