squiggy
version 1.5.2
(PPC R3 only)
an instrument for the BeOS.
by douglas irving repetto
screen shot
download squiggy
squiggy is a real-time spectral manipulation tool for the BeOS. with it you can create, record, manipulate and play back sound-images called sonograms.
squiggy is meant to be an instrument for live performance. with that in mind, i have gone for speed over accuracy in writing the processing routines. some of them are pretty noisy. become one with the pain. i've also chosen to go with a largely keyboard based interface. this gives squiggy a bit of a learning curve (you have to learn a bunch of key combinations) but i think it's worthwhile. you might want to keep "squiggy keyboard reference" (Styled Edit) or "skr.html" (NetPositive) open while learning to play squiggy.
new in squiggy 1.5:
mixer - allows you to play all 10 sonograms at once.
stereo - each sonogram can be individually panned.
time stretch/compress - each sonogram has its own frame rate.
window size - each sonogram has its own window size (which means they can all loop at different rates!)
stream to disk - squiggy's output can be saved to disk.
MIDI - mixing, panning, sonogram selection can be controlled via midi.
squiggy is now more stable and less RAM hungry. it should run better on a wide variety of machines. i'd still recommend at least 32MB of RAM to use squiggy.
playing squiggy:
the image in the black window is called a sonogram (or sometimes spectrogram.) it is a way of displaying information about the time-varying spectrum (frequency content) of a sound. when you draw or record a sonogram, what you end up with is a graph that shows how much energy (amplitude) the sound has in a number of frequency regions called bands. try this: record yourself counting or whistling, then try to associate the resulting image with the sound you are hearing. it's also nice to record from a CD and just watch as the sound passes by. sonograms are often quite beautiful.
in the sonogram, the x axis is time and the y axis is frequency. the grey level of a pixel is its magnitude (amplitude.)
what does what:
- mouse + button: draw in the black window.
- number keys: set the grey level (amplitude) for drawing.
- 'e': toggle the eraser. you can also press a number key to go back to drawing.
- delete or backspace: erase the entire image
- spacebar: toggle playback on and off.
- return: reset the playback position to the beginning of the image.
- 'r': toggle sonogram recording. you can use the mic or cd inputs on your computer. set which one to use in the "Sound" preferences panel - i've found that the input fader should be all the way down to avoid distortion. the "Aux" setting will not work. see "other stuff" below. the signal that you are recording is added to the mix as you're recording it. if you don't want this (it may feed back) just bring down the volume on that sonogram.
- command + 'r': toggle disk recording. you can stream squiggy's output to a file. right now there's no indicator that tells you you're recording to disk, other than your disk access light flashing!
- function keys F1-F10: switch between different stored images. you can have up to 10 sonograms stored at a time. controllable via MIDI.
- 'a': toggle autoswitching. autoswitching will automatically switch to the next image when the current image is done playing.
- 'A': toggle randomswitching. randomswitching will randomly switch between images. autoswitching and randomswitching can be used while recording -> this allows you to make one (or several) longer recording(s).
- 'm': toggle mixer. the mixer allows you to play all 10 sonograms at the same time (or as many as your machine can handle.) you can even record new sonograms while in mixer mode.
- 'v' or 'V': decrement/increment the number of active sonograms. default is 5. your machine will crash if you set the number too high. on an 8500/120 (604) i was able to mix 5 sonograms and record at the same time. on an 8500/233 (604e) i can mix all 10 sonograms and record and stream to disk, no problem.
- command key (the key right next to the space bar on most machines) + mouse button: set the current 'playback' point. if you keep the button down and move the mouse you can 'scrub' through the image!
- option (alt?) key + mouse button: explore the image with the pointer. information about the frequency/magnitude/location of the pixels you touch is displayed in the info window.
- '+': add sonograms. press an F key, press '+', then press another F key. the second image will be added to the 1st
- '-': subtract sonograms. see above.
- '*': multiply sonograms. see above. multiplication is pretty arbitrary, and mostly just makes noise.
- '/': divide sonograms. see above. division is pretty arbitrary, and mostly just makes noise.
- '=': copy a sonogram. press an F key, press '=' then press another F key. the first image will be copied to the second image (erasing anything that was there before). note that '=' works slightly differently that the other operations!
- up/down-arrow: change volume. only affects the sonogram's volume in the mix, not the actual sonogram magnitude data. controllable via MIDI.
- shift + up/down-arrow: change volume very quickly.
- control + up/down-arrow: frequency shifting. shifts all of the pixels (frequency bands) in the image up or down by one, changing the frequency of the sound. the pixels on the edges will wrap around to the bottom/top.
- control + shift + up/down arrow: frequency shift by 10 bands.
- left/right-arrow: pan. controllable via MIDI.
- control + left/right-arrow: decrease window size.
- control + shift + left/right-arrow: increase window size.
- '[' or ']': time stretch/compress. pressing the bracket keys will change the amount of time squiggy spends playing each frame of the sonogram, thereby stretching (more time) or compressing (less time) the sound in time. left-bracket stretches, right-bracket compresses.
- '(' or ')': gain change. left paranthesis lowers and right raises the gain (amplitude) of the whole sonogram. this is different than using the volume faders, which don't actually affect the sonogram's data.
- 'l': 5 loudest band filter. 'l' (as in liposuction) throws out all but the 5 frequency bands with the most energy.
- 'L': 10 loudest band filter.
-
'c': phase copy. press an F key, press 'c' then press another F key. the phase information from the second image will write over the phase information of the first. this does a strange type of mixing on the two sounds (really cheap cross-synthesis!) note that the image you see will not change - phase information is not encoded in the image.
-
'C': amplitude copy. like phase copy, only it replaces the amplitudes in image one with the amplitudes from image two. image one and image two will now look identical, but will sound different. spooky!
- 'd': deharmonize. what does 'deharmonize' mean? good question.
- 'D': deharmonize more.
- 'f': flip sonogram top-to-bottom. why? another good question.
- 'F': flip sonogram left-to-right, i.e. reverse it. god si dog.
- 's': skew the image down.
- 'S': skew the image up. just try it once, you'll understand!
- 'i': invert the image (sort of). try it on really noisy sounds.
- 'b': blur frequency (vertical). this is a pretty subtle effect. try many iterations of it to get some interesting sounds.
- 'B': blur time (horizontal.) this will eat any sharp attacks in the sound. try using it on some very percussive sounds - it will smear their attacks . . .
- 'w': filter weirdly (bands). the filter randomly fades bands in or out. do it many times for severe filtering. try it on noise to see how it looks.
- 'W': filter weirdly (frames). this version randomly fades each frame in or out. this effectively chops up the sound.
- 'n': fill the image with white noise.
- 'N': fill the image with random vertical stripes.
these are all just basic processes - the really interesting part is figuring out how to combine them to make wacky new sounds. dup-swap-drop!
MIDI control:
right now the MIDI setup is hard-coded. sorry. i'll change this in the future. all MIDI functions are accessible via keyboard commands, in case you can't use this MIDI setup.
- use MIDI port 2.
- all channels are active.
- controllers 0-9 are volume faders. you don't have to select a sonogram to fade it.
- controller 12 is the pan knob. it will pan the currently selected sonogram.
- MIDI notes 2-11 (D-1 to B-1) switch between sonograms.
neat things to try:
-
build up a bunch of loops. alter their frame increments and window sizes slightly so that they get out of phase with each other. listen for a long time.
-
when you draw, squiggy changes the magnitude of the pixels you touch, but keeps their old phase information. this means that you can draw with the phase from another image. do this: record something, use 'c' to copy its phase info to another (blank) image. now draw on the blank image - no more boring sine waves! if in the 2nd image you roughly fill in the places where the 1st image has a lot of energy, some timbral elements of the 1st image will start to appear . . .
-
buy the strangest fruit you can find. attempt to eat it. find out how you were supposed to eat it. buy another one.
-
record something, copy it to another spot, take the 10 loudest bands, then subtract them from the original. in general, subtracting original sounds from their processed offspring gives interesting results. wheehaw!
-
ride your bicycle backwards -- don't get a ticket like dominic!
-
record something, copy it to another location, use the arrow keys to shift its frequency, then add it to the original. repeat. again. maybe flip it over this time. or reverse it. etc.
-
record something, then use 'c' to copy the phase info from a blank image to what you just recorded. this wipes out the recorded phase info (assuming the blank image was really blank!) giving you some noisy clicky stuff.
-
make a smoothie: a glass of rice/soy milk, 1 banana, some frozen strawberries, some honey, some vanilla, maybe some coco powder or orange juice. whip in a blender. yumm.
-
record someone talking. press command and click the mouse in different spots to make them say the words in a different order . . . or press command and click and hold the mouse button, then move leftwards through the image to make them talk backwards. turn me on, deadman! practice secret messages.
-
use 'f' to flip a sound, then using control+up-arrow to shift it back into a more reasonable range. hmmm.
-
play with 'n' and 'N' to make some noise. try subtracting noises from one another (especially 'N' noise) and then blurring/detuning the results. apply the weird filter several dozen times. do lots of iterations of the same function. there are lots of neat sounds hiding in that thar noise . . .
-
listen to robert ashley's automatic writing. wonder what in the heck is going on. write some software to distract yourself.
-
record something. use 's' to skew the image. use 'w' a few times to apply the weird filter. now use 'S' to unskew it. pretty neat?
-
use 'w' and 'W' together to carve up an image into little squares. distortion is your friend. then use 'b' and 'B' to mush it around some . . .
- stop using the word 'stuff' so much.
other stuff (excuses, bugs, etc.):
squiggy was written and tested on an apple powermac 8500/120, 7600/132 and 8500/233. it should run great on faster machines, but i'm not sure how it will fair on slower ones, like the dual 66Mhz BeBoxen.
performing operations on images with lots of energy can result in some nasty clipping. this may or may not be what you're going for. to avoid clipping, gain down the sonogram before processing it, or record at a lower level to begin with. some clipping is also caused by the arbitrary nature of many of the processes (it's possible to create representations of sounds that are 'impossible' - upon resynthesis, these sounds will usually cause clipping.)
you will have to play with the input fader in the Sound preferences panel to get a good recording level. you'll probably need a different setting depending on whether you're using the internal CD player or an external sounce.
the line-level inputs (RCA jacks) don't work on powermacs running the BeOS. this is a major bummer, but it doesn't look like Be is going to be able to fix it. you can use the mic input of your mac as a line-level input by using a regular stereo mini-plug instead of the custom mac mic thingy. you'll need to work out some funky adapter chain. if you do select "Aux" in the Sound preferences panel, unpredictable things may happen.
squiggy is a memory hog! you'll need around 15MB to run squiggy. i don't recommend trying to run squiggy on a machine with less than 32MB of ram.
license:
squiggy is "artware" - that is, if you use and like it, send me something you've made. really. a picture, a CD, a necklace, a guanabana . . . whatever!
please ask me before including squiggy on any sort of software compilations, CD's, etc.
contact info:
douglas irving repetto
email: glmrboy@shoko.calarts.edu
email: douglas.repetto@dartmouth.edu
web: http://dappu.calarts.edu/~glmrboy
other BeOS software i've written:
SineClock
allPossibleImages
Yesh!
permanent land:
2046 bellemead ave
havertown, pa
19083-2221 USA
current land (good at least thru 1999):
6187 hinman
electro-acoustic music
hanover, n.h.
03755 USA
acknowledgments:
tom erbe, larry polansky and many people at the bregman electro-acoustic music studio at dartmouth college have made valuable contributions/suggestions during squiggy's youth. thanks!
music from Lovely Music Ltd., artifact recordings or Frog Peak Music (A Composers' Collective) was often playing while squiggy was being debugged.
thanks to Attila Mezei and Jon Watte for helping me get my stream-to-disk code worked out.
references:
much of squiggy was inspired by tom erbe's SoundHack program for the macintosh, especially his QT-coder. Spectral Assistant, which is included in the SoundHack download, is a great place to start mucking around with writing spectral domain processing algorithms.
A Digital Signal Processing Primer by Ken Steiglitz is a good place to learn about phasors, the FFT, digital filters, etc.
Curtis Roads' gigantic book, A Computer Music Tutorial has interesting information on some other attempts at frequency domain sound editors. there is also lots of information available on the web in relation to image->sound conversion. wouldn't it be cool if you could hook squiggy up to a video camera . . .
lots more that i can't think of this late at night.
the Be Operating System and the BeBook. QED.
to do (in some sort of 3/4 arbitrary order):
- more operations!
- add-on architecture
- basic drawing tools (gradients, partials, etc)
- log frequency plot, for more drawing pleasure!
- window with all keyboard commands listed.
- stream audio from disk
- copy/paste of bitmaps
- load/save bitmaps
- dynamic number of analysis bands
- etc., etc., etc. too much!
release history:
may 1, 1998: squiggy 1.5 released. big changes: mixer, panning, variable window sizes, MIDI.
february 26, 1998: squiggy 1.0 released! yeeehaw!
february 16, 1998: gave up on change history. too much to list!
january 6, 1998: added more features, processes, autoplay, etc.
december 15, 1997: lots of changes/additions. too many to mention. freesh! working on optimizations.
december 7, 1997: totally re-wrote squiggy, threw out lots of unused/confused code, simplified everything! added recording, scrubbing, image storage
december 3, 1997: updated to do list
november 30, 1997: pre-alpha sooper not-done version 0.0.1 release