CNMN > Projects > Audiovisual Mismatch

Guillaume Jabbour

  • Open (def: scores for unspecified instrumentation)
  • Digital devices
  • 5 to 12 years of age
  • 13 to 18 years of age
  • Adults

3-4 hours

  • Education

Audiovisual Mismatch

Description

Pic­ture a mut­ed video of a per­son walk­ing inside a house. Imag­ine what the foot­steps might sound like going up creaky steps or slight­ly drag­ging on the floor. Turn up the sound and become com­plete­ly dis­ori­ent­ed as you hear foot­steps walk­ing in crunchy snow, birds and a sym­pho­ny of frogs. Sound artist Guil­laume Jab­bour demon­strates how to use an iphone to shoot and edit a short video in iMovie and cap­ture, edit and mix real sounds in Garage­Band. The sounds are then synced with video to cre­ate a sur­re­al expe­ri­ence. The process offers insight into how movie sound effects are made by Foley artists.

Mate­ri­als: iPhone or iPad with Voice Memo, Garage­Band and iMovie

 

Tar­get Audi­ence: 10 years old +

 

Group Size: indi­vid­ual or pairs

Intro­duc­tion (15 min.)

  • Watch the sam­ple videos below with music AND with­out music.

  • Dis­cus­sion: 

    • What do you notice? (e.g. the audio does not match the visu­al, sounds like snow, nature).

      • ADAPTATION: The dis­cus­sion could be direct­ed by ask­ing “Com­plete these state­ments, “I see… I hear…”

    • How do you think this was done? (e.g. audio was made sep­a­rate­ly from video, every­thing was glued togeth­er after).

  • Watch the Foley Artists video below.

  • Dis­cus­sion:

    • What does a Foley Artist do?

 

Step 1: Cap­tur­ing and Edit­ing the Video (30 – 60 min.)

  • Open the cam­era app on your iPad or iPhone. Choose Video.

  • Choose a loca­tion indoors where you can walk for at least 1 min. with­out interruption.

  • Use your device to film a short scene (1 min. max). You can film a longer scene lat­er but for now, keep it short. Film your own legs if you are work­ing alone or take turns film­ing each oth­er walking.

  • Don’t wor­ry if there is extra footage at the begin­ning and end of your video; you can trim that out afterwards.

  • Watch your video and answer these questions:

    • Does the video look good?

    • Is the light­ing even? Is any­thing blown out (very bright and dif­fi­cult to look at) or too dark?

    • Is the image steady or is there lots of unwant­ed movement?

  • Con­sid­er the answer to these ques­tions and decide whether you need to reshoot your video or not to make it look as good as possible.

  • Once you are hap­py with the qual­i­ty of your video, open the iMovie app, select new project > Movie > choose the video you just shot and click on the + sign to add it to your iMovie project.

  • Tap the video in your iMovie project and at the bot­tom Tap Detach to detach the audio. Remem­ber, you will be over­lay­ing a dif­fer­ent audio track later.

  • When you see the sep­a­rate audio track below your video, tap and hold down to select it and press the delete key or flick it towards the top of your screen to delete it.

  • Trim the video to the desired length, cut­ting out any unnec­es­sary footage. 

  • To remove parts of your video in the mid­dle of the clip, place your cur­sor close to where you want to cut, tap the video region and choose the “Split” option. Then pull the lit­tle scis­sors down to split your clip into two regions. You can now trim out any unnec­es­sary footage on either side of the split.

  • Click on the Gear to access set­tings and tog­gle the slid­er to add a Fade in and out to your video to make it look smoother.

  • Once you have decid­ed on the final ver­sion of your video, you are ready to begin work­ing on the audio.

 

Step 2: Cap­tur­ing and Export­ing the Audio (30 min.)

  • Open the Voice Memo app on your iPhone or iPad – this app allows you to record even if you leave the app to watch your iMovie video at the same time. This is impor­tant for sync­ing the audio with the visual.

  • Fig­ure out where you will walk to cre­ate a dra­mat­ic mis­match with the video footage. Crunchy snow, leaves or shal­low water are great options.

  • When you are ready, start record­ing in Voice Memo, then leave the app and open your iMovie project. Hit play in iMovie and fol­low the video so that the audio cap­ture of your foot­steps match­es your video. Do this as many times as you need to, until you are happy.

  • Go back to the Voice Memo app and stop the recording.

  • Rename the file to a name that makes sense so that you can find it eas­i­ly lat­er on if nec­es­sary – e.g. Foot­steps crunchy snow

  • From the filelist in Voice Memo, click and hold down on your file, click Share > Save File > Garage­Band > Garage­Band File Transfer.

  • This is how you will access your file in Garage­Band so that you can mix the audio adding effects, pan­ning, EQ and compression.

 

Step 3: Mix­ing the Audio (30 – 60 min.)

  • Open the Garage­Band app, then tap new project > microphone

  • At the top right, tap the loops symbol

  • Tap Files > Garage­Band File Trans­fer > drag the file you just export­ed to the Garage­band workspace

  • Wear ear buds, air pods or head­phones and use your ears to edit the track until you are happy.

  • Use Track Pan to send your audio sig­nal to the left or right.

  • Use the EQ func­tion to bring out the high, mid­dle and low fre­quen­cies of your audio track. 

  • Use the Com­pres­sor to reduce the dif­fer­ence between your loud­est and qui­etest sounds.

  • Use Effects (Reverb, Echo) to enhance your track.

  • Once you are hap­py with your track, tap Projects at the top left, then rename your project to some­thing that makes sense like before (e.g. AV Mis­match Foot­steps). Then tap Done.

 

Step 4: Adding the Audio to the Video and Export­ing (15 – 30 min.)

  • In Garage­band, tap and hold down on your project then select Share > Song > High Qual­i­ty > Share > Open in… (this may take a few moments as your project must be export­ed) > Tap Open in iMovie > Choose your iMovie project

  • You will now see the audio and video in the same project

  • Place the audio where you want it.

  • You can add oth­er audio tracks using the same process

  • You can also use Garage­Band to cre­ate music to use as a sound­track for your video.

  • Tap Done.

  • Tap the Share arrow.

  • Decide how you want to export the file. A good option if you have access to a Mac­book or iMac is to Air Drop the Movie file to that and watch it there. Once the movie file is on a com­put­er, you can also upload it to your Youtube account.

 

Tak­ing it Further

What oth­er audio­vi­su­al mis­match­es can you imag­ine? Here are some examples:

  • Birds chirp­ing dur­ing a scene of a space­ship land­ing on the moon

  • A lizard walk­ing with foot­steps that sound like piano keys

  • Sounds of springs and saws dur­ing a scene of a doc­tor exam­in­ing a patient 

Hap­py edit­ing and have fun!

Read More +

Video