kenfrederick

Processing + Arduino. LightBrush

i've been working on this project off and on (a bit more off than on) since early 2009, and most recently i've given my self a deadline of wrapping it up by the end of 2011. from the looks of it, the conclusion however as it stands will be incomplete. i'm posting it here for posterity sake as well as in hope that maybe some of the successes and failures i've had may be of interest to others.

the idea
the crux of the project was to create light drawings in the air, but using light as a medium to recreate images, displayed on screen, in the air. a device with lights on it would somehow have x and y coordinates based on it's position in the air and correlate to x and y coordinates on the screen.

the beginning
i whipped an easy pixel grabber program, which used a black & white image and sent the location's pixel value (0 or 255) to a very crude prototype using one blue led. the single color led turns on or off based on the brightness of the pixel on screen. the biggest challenge was getting coordinates of the device in space. i ended up using a wii mote (via OSC) and an IR led to tell the computer the x and y coordinates of the device.

the initial tests with the prototype were promising but it was obvious i needed to tweak it (i.e. larger light source) in order to achieve the resolution i was hoping for. i knew i would need more LEDs and contemplated a stick or maybe an led matrix (multiples of either or a combination of the two). subtle pitch and yaw movements of the "brush" caused unwanted distortion so i used plexi glass as a surface to press on and the images got better, but lacked the "magic" i was hoping for.

everyone else
then i put the whole project on hold as my wife and i uprooted and relocated to europe, and in the meantime the inevitable happened and others developed the same idea (zeitgeist?). the first being by berg for dentsu london using an ipad to draw 3D type in the air.

and i should also mention timo arnall's other work with light painting.

although very close to my idea, no one had done it exactly as i pictured it in my head. there was still hope, yet i still was busy with other things and didn't have the time to invest in pursuing the project any further. by now it was near the end of 2010, when i stumbled across the light scythe, a very elegant execution which uses a large stick clad with RGB led's to create stunning images in the air. *drat* exactly what i wanted.

light scythe

onwards and upwards
i thought of just giving up and maybe i should have, but i decided to keep trudging on, because now i knew i wanted to create a more handheld version, which could be used untethered to the computer and maybe even together with other similar devices. which lead me to where i am now.

i began a couple months ago by focusing on using an 8x8 RGB matrix and a colorduino (shield), in wanting to make it completely mobile i started by putting together an iphone app and interface using openFrameworks and it was going well, i hit a couple of snags programming in objC. those problems aside, i had an app that would work theoretically, but in practice did not, because (so far as i know) communicating with external hardware devices from an iphone requires additional licenses and permissions from apple. so in my quest to find an alternative method of communicating to the arduino from an iphone, i found arms22's soft modem, i've included the parts needed to build one for yourself, but i quickly found out that it's baud rate of 1225bps would be way too low for sending 64*3 bytes (64 leds, and 3 bytes per color) at a speed that would be responsive enough for light drawing.

soft modem hardware
2 0.47µf electrolytic capacitor (blue can)
1 47pF tantalum capacitor (yellow)
2 100k resistor
3 10k resistor
1 4.7k resistor
1 3.5mm 4 pole phono input

that was a hard pill to swallow, and i was still hopeful that i could some how pull of a completely mobile computer-less version of my light brush. i had created an overly complicated series of sending a wifi signal from the phone to my laptop (via OSC) and then converting the OSC signal to serial and sending it to the arduino. luckily kevin cannon snapped me back to reality, and suggested that if i want to go mobile that interfacing with an android device would be much easier. if you've read this far, you know the dream of making what i envision is fading, but my stubborn determination kept me moving forward.

the result
abandoning all hopes of using the iphone, i switched to processing since i can prototype much quicker with it. and in the switch i abandoned the wiimote, as communicating with it via OSC was slow and cumbersome. once again kevin suggested just using the webcam and an IR filter. since apurely computer-less operation is on ice, i created a custom case which houses the arduino, colorduino (shield), 8x8 led matrix and 4 IR leds.

light brush hardware
4 IR Leds
8x8 rgb matrix
arduino
colorduino (shield)

after hours of coding, i finally had a desktop application that allowed me test my light brush. i can send 8 bits of RGB data at once (i.e. an image or parts of an image) and track the device in real time. i recruited my friend ben to help me test the hardware and software. we set up everything and set about drawing pictures in the air...

...but it didn't work at all, like i/we had thought/hoped it would. as we tried different settings and code optimizations, it became obvious why it wouldn't work. we learned you can only draw by moving the device in one direction (which is how the light scythe and others work) and also, the 8x8 led matrix at 60mm (2.3in) is most likely too small. below you can see the results of us trying to draw a simple "2" and that it didn't work. although my experiment with light drawing has ended in failure, i've upload large chunks of code that i've written for this, such as sending massive amounts of data over serial to arduino and a tracking class, maybe somebody out there will find them useful.

i failed to draw this "2" in the air, and learned that drawing images works better when the image size corresponds with the matrix size

as with previous projects, i'm really happy with what i've learned but i'm disappointed that i wasn't able to get it to work as i had hoped. maybe i'll let it sit for another two years and come back to it.

code
this is the class i created for tracking blobs (i.e. IR leds) and averaging their coordinates for smooth operation, (i wrapped it in a thread so that the overall frame rate of the main app isn't affected)

/*
* OpenCVTracker.pde
*
* Ken Frederick
* ken.frederick@gmx.de
*
* http://cargocollective.com/kenfrederick/
* http://kenfrederick.blogspot.com/
*
*/

//-----------------------------------------------------------------------------
// libraries
//-----------------------------------------------------------------------------
import codeanticode.gsvideo.*;
import hypermedia.video.*;

public class OpenCVTracker extends Thread {
//-----------------------------------------------------------------------------
// properties
//-----------------------------------------------------------------------------
private PApplet papplet;
private PGraphics imageBuff;
boolean bRunning = false;

// video
private GSCapture video;

// openCV
private OpenCV opencv;
private Blob[] blobs;
private int cvBrightness;
private int cvContrast;
private int cvThresh;

// points
float avgWeight = 0.5;

//-----------------------------------------------------------------------------
// constructor
//-----------------------------------------------------------------------------
public OpenCVTracker(PApplet _papplet, int _width, int _height) {
papplet = _papplet;

imageBuff = createGraphics(_width,_height, P2D);

// setup video
video = new GSCapture(_papplet, _width, _height);
video.play();

// setup openCV
opencv = new OpenCV(_papplet);
opencv.allocate(_width, _height);
setBrightness(0);
setContrast(0);
setThreshold(0);
}

//-----------------------------------------------------------------------------
// methods
//-----------------------------------------------------------------------------
public void start() {
bRunning = true;
super.start();
}

//-----------------------------------------------------------------------------
public void run() {
while(bRunning) {
update();

try {
} catch (Exception e) {
}
}
}

//-----------------------------------------------------------------------------
public void update() {
// open cv
opencv.copy(video);
opencv.convert(GRAY);
opencv.brightness(cvBrightness);
opencv.contrast(cvContrast);
opencv.threshold(cvThresh);

blobs = opencv.blobs(10, video.width*video.height/2, 4, false, 1); //OpenCV.MAX_VERTICES*4 );

// video
if (video.available()) {
video.read();
//video.loadPixels();
}
}

//-----------------------------------------------------------------------------
public void draw(int _x, int _y, boolean bVideo) {
draw(_x,_y, video.width,video.height, bVideo);
}

public void draw(int _x, int _y, int _width, int _height, boolean bVideo) {
pushMatrix();
translate(_x,_y);

PVector videoSize = new PVector(video.width, video.height, 0);
PVector avg = getAvg().get();
avg.mult(videoSize);

imageBuff.beginDraw();
imageBuff.background(0);
imageBuff.noFill();

if(bVideo) {
imageBuff.image(video, 0,0, video.width,video.height);
} else {
imageBuff.image(opencv.image(), 0,0, video.width,video.height);
//imageBuff.image(opencv.image(), 0,0+video.height, video.width,video.height);
//imageBuff.image(opencv.image(), 0+video.width,0, video.width,video.height);
}

// draw blobs
imageBuff.stroke(255,0,255, 255*0.6);
imageBuff.beginShape();
for(int i=0; i<blobs.length; i++) {
for(int j=0; j<blobs[i].points.length; j++ ) {
//imageBuff.ellipse(blobs[i].points[j].x, blobs[i].points[j].y, 9,9);
imageBuff.line(blobs[i].points[j].x-4.5,blobs[i].points[j].y, blobs[i].points[j].x+4.5,blobs[i].points[j].y);
imageBuff.line(blobs[i].points[j].x,blobs[i].points[j].y-4.5, blobs[i].points[j].x,blobs[i].points[j].y+4.5);
imageBuff.vertex(blobs[i].points[j].x, blobs[i].points[j].y);
}
imageBuff.vertex(avg.x,avg.y);
}
imageBuff.endShape();

// averaged blob point
//imageBuff.fill(0,255,255, 255*0.6);
//imageBuff.ellipse(getAvg().x,getAvg().y, 9,9);

imageBuff.strokeWeight(3);
imageBuff.stroke(0,255,255, 255*0.7);
imageBuff.line(avg.x-4.5,avg.y, avg.x+4.5,avg.y);
imageBuff.line(avg.x,avg.y-4.5, avg.x,avg.y+4.5);
imageBuff.ellipse(avg.x,avg.y, 12,12);
imageBuff.endDraw();

image(imageBuff, 0,0, _width,_height);

popMatrix();
}

//-----------------------------------------------------------------------------
/**
* filter the current result using a weighted average filter:
* http://web.archive.org/web/20070224180400/http://www.tigoe.net/pcomp/code/archives/arduino/000710.shtml
*/
private float filter(float rawValue, float weight, float lastValue) {
float fValue = 0.0f;
float x = weight; // is a value between 0 and 1
//int x = 0;
//x = weight/102; // convert the weight number to a value between 0 and 1:
fValue = (x * rawValue + (10-x)*lastValue)/10; // run the filter:

return fValue;
}

//-----------------------------------------------------------------------------
public void quit() {
bRunning = false;
opencv.stop();
papplet.stop();
interrupt();
}

//-----------------------------------------------------------------------------
// public void stop() {
// quit();
// opencv.stop();
// papplet.stop();
// }

//-----------------------------------------------------------------------------
// sets
//-----------------------------------------------------------------------------
public void setBrightness(int val) {
cvBrightness = val;
}
public void setContrast(int val) {
cvContrast = val;
}
public void setThreshold(int val) {
cvThresh = val;
}

//-----------------------------------------------------------------------------
public void setAvgWeight(float val) {
avgWeight = val;
}

//-----------------------------------------------------------------------------
// gets
//-----------------------------------------------------------------------------
public boolean getRunning() {
return bRunning;
}

//-----------------------------------------------------------------------------
public PVector getCoord(int num) {
PVector v = new PVector();
if( blobs.length != 0) {
int c = constrain(num, 0,blobs.length);
v.set( norm(blobs[c].points[0].x, 0,video.width), norm(blobs[c].points[0].y, 0,video.height), 0);
}
return v;
}

//-----------------------------------------------------------------------------
PVector pv = new PVector();
public PVector getAvg() {
PVector v = new PVector();
int count = 0;
for(int i=0; i<blobs.length; i++) {
v.x += getCoord(i).x;
v.y += getCoord(i).y;
v.z += getCoord(i).z;
count++;
}
v.x /= count;
v.y /= count;
v.z /= count;

v.x = filter(v.x, avgWeight, pv.x);
v.y = filter(v.y, avgWeight, pv.y);
v.z = filter(v.z, avgWeight, pv.z);

pv = v.get();

return v;
}
}

here's the class i wrote for sending an image via serial to arduino (also wrapped in a thread)

/**
* SerialPixelSender.pde
*
* Ken Frederick
* ken.frederick@gmx.de
*
* http://cargocollective.com/kenfrederick/
* http://kenfrederick.blogspot.com/
*
* threaded code for sending data from processing to arduino
*
*/

//-----------------------------------------------------------------------------
// libraries
//-----------------------------------------------------------------------------
import processing.serial.*;

public class SerialPixelSender extends Thread {
//-----------------------------------------------------------------------------
// constants
//-----------------------------------------------------------------------------
private int ARDUINO_BAUD_RATE = 115200; // must be exactly the same in the arduino sketch
private final int ARDUINO_TIMEOUT = 0; // milliseconds

private final char START_BYTE = 42; // 42 = *
private final char DELIMITER = 44; // 44 = ,
private final char END_BYTE = 35; // 35 = #
private final char MODE_BYTE = 63; // 32 = ?

private final char DRAW_BYTE = 124; // 124 = |
private final char SEND_BYTE = 95; // 95 = _

//-----------------------------------------------------------------------------
// properties
//-----------------------------------------------------------------------------
// application
private PApplet papplet;
private boolean bMode = false;
private boolean bRunning = false;
private boolean bSendMessage = false;

// serial
private Serial arduino;
private boolean bConnected = false;

// image
private PImage src;

//-----------------------------------------------------------------------------
// constructor
//-----------------------------------------------------------------------------
public SerialPixelSender() {
}

public SerialPixelSender(PApplet _papplet) {
// application
papplet = _papplet;

// serial
println( Serial.list() );
String portName = Serial.list()[0];

arduino = new Serial(papplet, portName, ARDUINO_BAUD_RATE);
arduino.bufferUntil( END_BYTE ); // lf == line feed, ASCII 10; the line ending from arduino

// image
//PImage src = new PImage(8,8, RGB);
}

public SerialPixelSender(PApplet _papplet, int _ARDUINO_BAUD_RATE) {
// application
papplet = _papplet;

// serial
println( Serial.list() );
String portName = Serial.list()[0];

ARDUINO_BAUD_RATE = _ARDUINO_BAUD_RATE;
arduino = new Serial(papplet, portName, ARDUINO_BAUD_RATE);
arduino.bufferUntil( END_BYTE ); // lf == line feed, ASCII 10; the line ending from arduino

// image
//PImage src = new PImage(8,8, RGB);
}

//-----------------------------------------------------------------------------
// methods
//-----------------------------------------------------------------------------
public void start() {
bRunning = true;
super.start();
}

//-----------------------------------------------------------------------------
public void run() {
while(bRunning) {
update();

try {
//sleep( (long)(ARDUINO_TIMEOUT) );
} catch (Exception e) {
}
}
}

//-----------------------------------------------------------------------------
public void update() {
if(bSendMessage) {
sendImage(src);
}
}

//-----------------------------------------------------------------------------
private void sendImage(PImage _src) {
toggleDraw();

try {
//img.resize(8,8);
PImage img = new PImage(8,8, RGB);
if(_src.width > 7 || _src.height > 7) {
img = setImage(_src);
/*
img.copy(_src, 0,0,_src.width,_src.height, 0,0,8,8);
img.loadPixels();
*/
} else {
img = _src;
}

for(int index=0; index<img.pixels.length; index++) {
int col = img.pixels[index];
int r = (int) ((col >> 16) & 0xFF);
int g = (int) ((col >> 8) & 0xFF);
int b = (int) (col & 0xFF);
sendRGB(r,g,b, index);
}

} catch(Exception e) {
}

toggleDraw();
}

//-----------------------------------------------------------------------------
private void sendRGB(int R, int G, int B, int index) {
arduino.write( START_BYTE );

//R
arduino.write(R);
arduino.write( DELIMITER );

//G
arduino.write(G);
arduino.write( DELIMITER );

//B
arduino.write(B);
arduino.write( DELIMITER );

//INDEX
arduino.write( index );

arduino.write( END_BYTE );
arduino.clear(); // dumps buffer before asking for next data point
}

//-----------------------------------------------------------------------------
private void toggleDraw() {
arduino.write( START_BYTE );
arduino.write( DRAW_BYTE );
arduino.write( END_BYTE );
arduino.clear(); // dumps buffer before asking for next data point
}

//-----------------------------------------------------------------------------
private void toggleMode() {
arduino.write( START_BYTE );
arduino.write( MODE_BYTE );
arduino.write( END_BYTE );
arduino.clear(); // dumps buffer before asking for next data point
}

//-----------------------------------------------------------------------------
public void quit() {
bRunning = false;
interrupt();
arduino.clear();
arduino.stop();
}

//-----------------------------------------------------------------------------
// sets
//-----------------------------------------------------------------------------
public void send() {
bSendMessage = true;
}

//-----------------------------------------------------------------------------
public void set(int[] colors, int colorsNum) {
PImage img = new PImage(8,8, RGB);
//img.loadPixels();
for(int i=0; i<colors.length; i++) {
img.pixels[i] = colors[i];
}
setImage(img);
}

//-----------------------------------------------------------------------------
public PImage setImage(PImage _src) {
src = _src;

PImage temp = new PImage(8,8, RGB);
temp.copy(src, 0,0,src.width,src.height, 0,0,8,8);
temp.loadPixels();

return src;
}

//-----------------------------------------------------------------------------
// gets
//-----------------------------------------------------------------------------
public boolean getRunning() {
return bRunning;
}

//-----------------------------------------------------------------------------
PImage getImage() {
return src;
}

}

this is the corresponding code for the arudino (using a colorduino) which also needs my modified version of the colorduino library

/**
* LightBrushColorduino receiver 1.0
*
* Ken Frederick
* ken.frederick@gmx.de
*
* http://cargocollective.com/kenfrederick/
* http://kenfrederick.blogspot.com/
*
*/

//-----------------------------------------------------------------------------
// includes
//-----------------------------------------------------------------------------
#include <Colorduino.h>

//-----------------------------------------------------------------------------
// constants
//-----------------------------------------------------------------------------
/**
* these must match what is being sent
* from the host app
*/
#define ARDUINO_BAUD_RATE 115200 // must be exactly the same in the arduino sketch

// data packet constants
#define BUFFER_SIZE 64 // 8*8

#define START_BYTE 42 // 42 = *
#define DELIMITER 44 // 44 = ,
#define END_BYTE 35 // 35 = #
#define MODE_BYTE 63 // 32 = ?

#define DRAW_BYTE 124 // 124 = |
#define SEND_BYTE 95 // 95 = _

//-----------------------------------------------------------------------------
// properties
//-----------------------------------------------------------------------------
//int bufferPacket[BUFFER_SIZE][5];
int bufferPacket[BUFFER_SIZE][3];

unsigned char red, green, blue;
int pos, index;

bool bDraw = false;
bool bMode = true;

//-----------------------------------------------------------------------------
// methods
//-----------------------------------------------------------------------------
void setup() {
// serial
Serial.begin(ARDUINO_BAUD_RATE);

red = green = blue = 0;
pos = 0;
index = 0;

// colorduino
Colorduino.Init();

/**
* compensate for relative intensity differences in R/G/B brightness
* array of 6-bit base values for RGB (0~63)
* whiteBalVal[0] = red
* whiteBalVal[1] = green
* whiteBalVal[2] = blue
*/
unsigned char whiteBalVal[3] = { 10,25,25 };
Colorduino.SetWhiteBal(whiteBalVal);

}

//-----------------------------------------------------------------------------
void update() {
while (Serial.available()) {
byte serialBuffer = Serial.read();

switch(serialBuffer) {
case START_BYTE:
pos = 0;

//Serial.flush();
continue;
break;

case DELIMITER:
pos++;
break;

case DRAW_BYTE:
bDraw = !bDraw;
//continue;
break;

case MODE_BYTE:
bMode = !bMode;
//continue;
break;

case END_BYTE:
//Serial.print(SEND_BYTE);
Serial.flush();
//continue;
break;

} // end switch serialBuffer

/**
* how many values are we sending at once?
* switch between to load the proper variables
*/
switch(pos) {
case 0:
red = serialBuffer;
break;
case 1:
green = serialBuffer;
break;
case 2:
blue = serialBuffer;
break;
case 3:
index = serialBuffer;

// push to bufferPacker
bufferPacket[index][0] = red;
bufferPacket[index][1] = green;
bufferPacket[index][2] = blue;

/*
unsigned int x = index / ColorduinoScreenWidth;
unsigned int y = index % ColorduinoScreenHeight;
bufferPacket[index][3] = x;
bufferPacket[index][4] = y;
*/

break;
}

} // end while
}

//-----------------------------------------------------------------------------
void loop() {
update();

/**
* process color
*/
if(bDraw) {
if(bMode) {
ColorFill(bufferPacket[0][0], bufferPacket[0][1], bufferPacket[0][2]);
} else if(!bMode) {
ColorFrame(bufferPacket);
/*
for(int i=0; i<BUFFER_SIZE; i++) {
ColorPoint(bufferPacket[i][0], bufferPacket[i][1], bufferPacket[i][2], i);
}
*/
}
}

Colorduino.FlipPage();
}

//-----------------------------------------------------------------------------
void ColorFrame(int buffer[BUFFER_SIZE][3]) {
ColorRGB *p = Colorduino.curWriteFrame;
int pindex = 0;
for(unsigned char y=0; y<ColorduinoScreenWidth; y++) {
for(unsigned char x=0; x<ColorduinoScreenHeight; x++) {
// p->r = bufferPacket[pindex][0];
// p->g = bufferPacket[pindex][1];
// p->b = bufferPacket[pindex][2];
p->r = buffer[pindex][0];
p->g = buffer[pindex][1];
p->b = buffer[pindex][2];

pindex++;
p++;
}
}
}

//-----------------------------------------------------------------------------
void ColorPoint(unsigned char R,unsigned char G,unsigned char B, unsigned int _index) {
ColorRGB *p = Colorduino.curWriteFrame;
p[index].r = R;
p[index].g = G;
p[index].b = B;

//Colorduino.FlipPage();
}
void ColorPoint(unsigned int x, unsigned int y, unsigned char R,unsigned char G,unsigned char B) {
ColorRGB *p = Colorduino.GetPixel(x,y);
p->r = R;
p->g = G;
p->b = B;

//Colorduino.SetPixel(x,y, R,G,B);
//Colorduino.FlipPage();
}

//-----------------------------------------------------------------------------
void ColorFill(unsigned char R,unsigned char G,unsigned char B) {
ColorRGB *p = Colorduino.GetPixel(0,0);
for(unsigned char y=0; y<ColorduinoScreenWidth; y++) {
for(unsigned char x=0; x<ColorduinoScreenHeight; x++) {
p->r = R;
p->g = G;
p->b = B;
p++;
}
}

//Colorduino.FlipPage();
}

Labels: arduino, experiment, iphone, openframeworks, processing

Drop Processing

Drop Processing is an app i put together a while ago for quickly batch processing images (using adobe photoshop). often when using images i would need them to be all cmyk or all rgb and constantly re-doing the same "batch processing" actions in photoshop or having to create a multitude of drag and drop applets.

Drop Processing utilizes applescript automation in order to process images. it's meant to be used as a drag-n-drop app by selecting a group of images and dragging them to the app icon. i like to keep the app in the sidebar of finder windows for quick access.

max. dimension
images can be resized by the longest dimension 0 = do not resize
(i.e. if you enter "50" an 800 x 600 would be sized down to 50 x 38)

resolution
resolution can be changed to: 72, 96, 150, 300, or "no change"
(i purposely made this a drop-down menu, as i commonly only need these values)

color space
the usual suspects: cmyk, rgb, grayscale, and "no change"

flatten image
if image is multi-layered, .tif, .psd, etc. it will be flattened to one layer

save as copy
this will append "Copy of" to the front of processed images

set preferences
sets the settings, settings are saved from session to session

as you can see in the video, it's pretty easy to use and it supports all photoshop compatible file types. there are two caveats, all dropped ".gif" files will be converted to ".psd" files and all dropped ".jpeg" files will be converted to ".jpg" files. as of right now it does not support dropping of folders.

download Drop Processing, if anyone is interested in the source, let me know. however i wrote the app using applescript studio, which is no longer supported by apple.

i've tested this app using a osx 10.5 - 10.6.8 and Photoshop CS2 - CS5. i cannot be certain if it will work in lion (my guess is, it will not)

Labels: design, tool

processing. triangulation + meshes

i've been slowly experimenting with meshes (mostly with triangulation algorithms), or better said trying to understand how to better control them. it started out innocent enough and has slowly progressed. after deconstructing jonathan puckey's delaunay raster scriptographer work, i created my own version and began modifying it. i expanded upon my own version by creating a version which maps brightness values of an area to line density (instead of colors).

karl pilkington, triangulated

although proud of my achievement, the entire thing is something that jonathan is exploring, but honestly without a lot of conceptual thought i kept trudging aimlessly on. i thought, well i'll take this idea to live and moving images (recently the internet has proved to me that everything has been done, with this and this) my explorations with my own puckey-delaunay-raster script proved the best illustrations come from getting in the details of the face, so when i switched to processing for moving images i created a rudimentary edge detection class which pulls out the edges as PVector arraylist.

/**
* FEdgeDetect.pde
*
* Ken Frederick
* ken.frederick@gmx.de
*
* edge detection class inspired by
* http://processing.org/learning/topics/edgedetection.html
*/

public class FEdgeDetect {
// -----------------------------------------------------------------------------
// properties
// -----------------------------------------------------------------------------
private PImage img;
private PImage edgeImg;
private float thresh;
private float density;
private float[][] kernel = { { -1, -1, -1 },
{ -1, 1, -1 },
{ -1, -1, -1 } };

public ArrayList<PVector> edgePVector = new ArrayList<PVector>();
public ArrayList<Float> edgeFloat = new ArrayList<Float>();

public boolean bProcessEdge = false;
public boolean bProcessImage = false;
public boolean bProcessMovie = false;

// -----------------------------------------------------------------------------
// constructor
// -----------------------------------------------------------------------------
public FEdgeDetect() {
}

public FEdgeDetect(PImage _img) {
setImage(_img);
}

// -----------------------------------------------------------------------------
// methods
// -----------------------------------------------------------------------------
public void process(float _thresh) {
if(_thresh != thresh || bProcessImage || bProcessMovie) {
thresh = _thresh;
bProcessImage = false;
kernel[1][1] = thresh;

for (int y=1; y<img.height-1; y++) {
for (int x=1; x<img.width-1; x++) {
float sum = 0; // Kernel sum for this pixel

for (int ky = -1; ky <= 1; ky++) {
for (int kx = -1; kx <= 1; kx++) {
int pos = (y + ky)*img.width + (x + kx);
int val = (img.pixels[pos] >> 16) & 0xFF;
sum += kernel[ky+1][kx+1] * val;
}
}

edgeImg.pixels[y*img.width + x] = color(sum);
}
}

edgeImg.updatePixels();

} // end check

}

public void process(PImage img, float thresh, boolean _process) {
setImage(img);
bProcessMovie = _process;
process(thresh);
}

public void findEdges(int _density, float b_low, float b_high) {
// println(_density + "\t!=\t" + density);
if(_density != density || bProcessEdge || bProcessMovie) {
density = _density;
bProcessEdge = false;

edgePVector.clear();
edgeFloat.clear();

for(int x=1; x<edgeImg.width-1; x+=density) {
for(int y=1; y<edgeImg.height-1; y+=density) {
//float b = brightness( edgeImg.get(x,y) );
float b = FTools.luminance( edgeImg.get(x,y) );
if( b > b_low && b < b_high ) {
float z = norm(b, 0,255);
edgePVector.add( new PVector(x,y,z) );

edgeFloat.add( new Float(x) );
edgeFloat.add( new Float(y) );
edgeFloat.add( new Float(z) );
}
}
}

} // end check

if(edgePVector.size() == 0) {
edgePVector.add( new PVector(0,0,0) );
}
if(edgeFloat.size() == 0) {
edgeFloat.add( 0.0f );
edgeFloat.add( 0.0f );
edgeFloat.add( 0.0f );
}

}

// -----------------------------------------------------------------------------
// sets
// -----------------------------------------------------------------------------
protected void setImage(PImage _img) {
img = _img;
img.loadPixels();
edgeImg = createImage(img.width, img.height, RGB);
// edgeImg.filter(THRESHOLD, .4);
}

// -----------------------------------------------------------------------------
// gets
// -----------------------------------------------------------------------------
public PImage getSource() {
return img;
}

public PImage getEdge() {
return edgeImg;
}

public ArrayList getEdgePVector() {
return edgePVector;
}

/**
* need to figure out a better solution for
* smoother integration with OpenGL VBOs
*/

}

the first attempt was terribly slow, but worked more or less. i contemplated and subsequently failed to port it over to openFrameworks (i couldn't get the edge detection to work). i realized even if the concept is nothing new i could at least learn more about optimization and harnessing opengl VBOs (vertex buffer objects). and eventually i developed a version which takes web cam input, edge detects, and triangulates all the while maintaining a frame rate of 30fps+. a z depth (height map) is generated from the brightness of the edge-detected-pixel, and by adding a multiplier an interesting "face landscape" occurs (especially when using the smooth gradient mode).

pgl.beginGL();

if(triangles.size() != 0) {
if(mode == 0 || mode == 1) {
gl.glBegin(GL.GL_TRIANGLES);

float x1,y1,z1, x2,y2,z2, x3,y3,z3;
float px1,py1,pz1, px2,py2,pz2, px3,py3,pz3;;
float r,g,b,a, r2,g2,b2;

if(mode == 0) {
/**
* smooth gradients
*/
for(int i=0; i<TriangleVBOPoints.length; i+=3) {
x1 = TriangleVBOPoints[i];
y1 = TriangleVBOPoints[i+1];
z1 = TriangleVBOPoints[i+2];

int c1 = img.pixels[ ((int)y1*capture.width) + ((int)x1) ];
r = norm((c1 >> 16) & 0xff, 0,255);
g = norm((c1 >> 8) & 0xff, 0,255);
b = norm(c1 & 0xff, 0,255);
if(bOpaque) a = 1.0;
else a = norm(FTools.luminance(c1), 0,255);

gl.glColor4f(r,g,b, a);
gl.glVertex3f(x1,y1,z1*zHeight);
}

} else if(mode == 1) {
/**
* flat colors
*/
for(int i=0; i<TriangleVBOPoints.length; i+=9) {
x1 = TriangleVBOPoints[i];
y1 = TriangleVBOPoints[i+1];
z1 = TriangleVBOPoints[i+2];

int c1 = img.pixels[ ((int)y1*capture.width) + ((int)x1) ];
r = norm((c1 >> 16) & 0xff, 0,255);
g = norm((c1 >> 8) & 0xff, 0,255);
b = norm(c1 & 0xff, 0,255);
if(bOpaque) a = 1.0;
else a = norm(FTools.luminance(c1), 0,255);

gl.glColor4f(r,g,b, a);
gl.glVertex3f(x1, y1, z1*zHeight);

x2 = TriangleVBOPoints[i+3];
y2 = TriangleVBOPoints[i+4];
z2 = TriangleVBOPoints[i+5];
gl.glVertex3f(x2, y2, z2*zHeight);

x3 = TriangleVBOPoints[i+6];
y3 = TriangleVBOPoints[i+7];
z3 = TriangleVBOPoints[i+8];
gl.glVertex3f(x3, y3, z3*zHeight);

}

}

gl.glEnd();

}

pgl.endGL();

i really liked this valley effect when increasing the z depth, and i thought it would be cool if i could start to distort the image by "folding" and "crumpling" it. similar to the pop faces work by joshua scott.

then i began thinking it would be even better to do this directly in illustrator using scriptographer after porting my edge detection code over to javascript, and creating the necessary 3d infrastructure i ran into challenges distorting the image although i did manage to create a neat kaleidoscope effect, which wasn't exactly what i was going for (although, that could lead to something)

i've pretty much abandoned the idea of achieving what i want in illustrator and have gone back to processing. using particles and physics i can distort a blank mesh of lines, however, i cannot deform an image. i can get the image mapped as a texture to a mesh, but as i distort the mesh, the image gets clipped (as if in a mask). this probably stems from me calculating the texture coordinates incorrectly.

all in all, i'm happy with what i've learned especially better understanding direct opengl calls and optimization, but i'm frustrated that i've hit this stumbling block. i think i'll probably put this idea on ice and focus on some other things (that require less programming) and maybe loop back around in the future.

Labels: processing, scriptographer

scriptographer. lorem ipsum

It happens often enough that i need dummy-text in illustrator. unfortunately this capability has yet to be (if ever) implemented. typically i use lipsum.com or generate the text in indesign or keep a blank text file with "lorem ipsum" text in it.

loremIpsum_0_0.js will generate an unlimited amount of "lorem ipsum" text, either number of paragraphs or number of words.

features:

1/ if nothing is selected a text box, with the desired amount of text, will be created in the center of the page.
2/ type items can be selected and either replace or add to existing text. in both cases type attributes such as size, leading, and color are maintained. this video shows how the script functions.
3/ punctuation inclusion can be toggled as well as Title Case for headline creation

so far as i have tested, the script runs in cs3 and cs5 without any errors. if you find a bug (which is likely) let me know. grab the script loremIpsum_0_0.js

Labels: scriptographer

processing. prettycolors

the amount of time between posts in the last year has been too much. i'm going to try and blog more regularly, instead of just blogging my final projects i'm going to start posting little snippets, as well as successes and failures in between.

i present to you a class for pulling colors from the tumblr blog http://prettycolors.tumblr.com/. i like the simplicity of the blog and the idea of color palettes being dynamic and changing over time (that and sites like kuler and colour lovers are covered)

this class grabs colors from the site and i built in rudimentary (and not yet complete) palette creation options. palettes are groups of five colors, which can be accessed getPalette(int type, int num). the default is to load the first 50 colors (max as allowed by the tumblr api), but this can be altered by changing the value of numToReturn.

the types of palettes available and created upon instantiation of the class (for faster access) are as follows:

ORDER palettes based on posting order, most recent colors appear first
BRIGHTNESS palettes based on brightness* (default creates 10 palettes)
COMPLEMENT palettes of complementary* colors (default creates 10 palettes)
RANDOM palettes of random colors (default uses the 50 most recent colors)
RANDOM_ALL palettes of random colors (uses all posted colors)

* implementation is very very very rudimentary, if not downright wrong

in addition to creating palettes, palettes can also be "exported" as .png files for later use (for example using my FPalette class as part of my incomplete frederickk processing library)

below is an example that uses the Prettycolors.pde class to create the above application. in this example palettes can be "exported" as .png files by clicking the mouse, palettes can be refreshed by pressing the spacebar.

Prettycolors pc;
int choose[];

PFont typeface;

void setup() {
size(500,500);
noLoop();
noStroke();

choose = new int[3];
choose[0] = (int) random(10);
choose[1] = (int) random(10);
choose[2] = (int) random(50);

pc = new Prettycolors(this);

typeface = createFont("LucidaGrande-Bold",9);
textFont(typeface);
}

void draw() {
background(0);
noStroke();

//palettes
pc.drawPalette(0,0, width,height/5, pc.ORDER, choose[0]);
pc.drawPalette(0,(height/5), width,height/5, pc.BRIGHTNESS, choose[1]);
pc.drawPalette(0,(height/5)*2, width,height/5, pc.COMPLEMENT, choose[2]);
pc.drawPalette(0,(height/5)*3, width,height/5, pc.RANDOM);
pc.drawPalette(0,(height/5)*4, width,height/5, pc.RANDOM_ALL);

//labels
fill(0);
text("Order", 15,15);
text("Brightness", 15,(height/5) +15);
text("Complement", 15,(height/5)*2 +15);
text("Random", 15,(height/5)*3 +15);
text("Random All", 15,(height/5)*4 +15);

}

/**
* pressing mouse will save the palettes as 5x1 pixel images for later use
*/
void mousePressed() {
pc.getPalette(pc.ORDER, choose[0]).save();
pc.getPalette(pc.BRIGHTNESS, choose[1]).save();
pc.getPalette(pc.COMPLEMENT, choose[2]).save();
pc.getPalette(pc.RANDOM).save();
pc.getPalette(pc.RANDOM_ALL).save();
}

/**
* pressing the spacebar will randomly create new palettes
*/
void keyPressed() {
if(key == ' ') {
choose[0] = (int) random(10);
choose[1] = (int) random(10);
choose[2] = (int) random(50);
redraw();
}
}

* as with most of the things found on this blog, the code works for me (osx 10.6.7, processing 1.5.1) but i make no guarantees that it will work for you. however, feel free to update and hack anything useful out it. just let me know and please keep a link to this blog or my github repositories

Labels: processing

scriptographer. push/pull points

it's been a long time since my last update, and i've got a couple things on the horizon as soon as work lets up a little bit. one thing i've been updating quite a bit recently is my processing library frederickk

however, here's a quick thing i whipped up for scriptographer. i was wanting to easily affect meshes i've been building in illustrator. so here's a pair of scripts that "push and pull" points of selected objects around the artboard. somewhat similar to the 'warp tool', but more predictable in my opinion.

pointsPushPullTool_0_0.js uses the scriptographer pen-tool, when the mouse nears a certain it pushes it around. mouse proximity can be set within the palette, acceleration is the distance an object's points move.

pointsPushPull_0_0.js set the 'distance threshhold' as a way to control how close points should be to each other to be moved together. connect nearby points by setting the random value to 0, and by changing the random value to something above 0 the points will randomly move that distance.

i haven't fully tested these scripts 100%, so they may still be buggy. they seem too insignificant to warrant a release on the scriptographer site.

grab the scripts/source pointsPushPullTool_0_0.js and pointsPushPull_0_0.js

Labels: scriptographer

schhplttlr

for the past couple of months i've been working on a project called "schhplttlr: electric beats" with daniel kluge and eugen kern-emden presented by peer to space and in collaboration with the department of cultural affairs munich.

the premise
Using movements inspired from “schuhplattler” (traditional Bavarian folk dance). The dancers generate beats and light via sensors. Musicians, media artists, dancers and choreographers experiment with these movements during a three-day long interdisciplinary workshop, in which the audience may freely participate. The results were performed at the opening evening on July 15, 2010.

this documentary film and selected photographs will be projected in MaximiliansForum, munich from 19 july 2010 until 5 september 2010.

the process (slightly abridged)
we began this project at the beginning of may, wanting to create something that could only happen in munich but also connect our collective interests of music, physical computing, performance, textile electronics, etc. we stumbled upon the idea of schuhplattler but we weren't sure yet how to represent it. literally? artsy-fartsy? projected (generative) images? physically? schuhplattler is a dance of courtship. it all started coming together, female and male. light bulbs flitting like fireflies in the spring, feminine, and harmonic. fluorescent lights structural, rhythm, and manly.

there were three main tools we used, max/msp for receiving and processing signals from the sensors and controlling the sounds. these signals were then sent out as open sound control (osc) messages to processing. which converted the messages into a signal for arduino which via relays turnd the lights (incandescent bulbs and fluorescent tubes) on and off.

we began prototyping. daniel focused on sensor creation and sound control. the gloves and other sensors were made using conductive foam. the wearer of the sensor presses the foam to complete the circuit. these values were then fed into max/msp where daniel weaved his magic, triggering sounds and controlling sample. eugen focused on programming reactive projected graphics (later abandoned). my task was to simply turn on light bulbs with arduino using processing.

by the time july rolled around we had successfully built a small prototype. eight light bulbs divided into two groups, four fluorescent bulbs, with working glove and thigh sensors. one small hurdle we had yet to solve 100% was the communication between max/msp and processing. as i said before we chose to osc as the communication medium using the oscp5 library. this was fine, however the way one builds a proper osc message in max/msp is (still) elusive to us. in the end we just put all of the necessary info within the message (control) field. on top of that when we did send messages there was a slight delay. we solved the delay by connecting both of our computers directly with an ethernet cable. we determined the delay was because we were connected via wifi.

we had still yet to test with the dancers or even move into the space, none of this happened until the week of the performance. as you can see MaximiliansForum is big and empty. but now we at least knew how everything would be connected it was now just a matter of taking the prototype and multiplying everything to the final quantities.

you can see more videos of our process at http://www.danielkluge.com/XX2010/schhplttlr.htm

the hardware
we kept things fairly simple on the hardware side to control all of the lights. we used a single arduino duemilanove and 12 modules built of the following parts.

2N3904 transistor
100V 1A 1N4002 diode
G6B–1114 5VDC relay
10A Fuse* (didn't actually implement the fuse)

as usual i have to thank my pops for his help with the electronic stuff. initially i soldered all of the components as the relays were jumping out of the breadboard sockets, because of the fast switching i assume. however, this proved disastrous and in the midnight hour i de-soldered everything and used breadboards instead.

as for the software on the arduino, one thing that had perplexed me was how to send an array of data to the arduino from processing. for the initial prototyping i was simply sending (per light group) a character for "on" (i.e. "Q") and then sending another character for "off" (i.e. "Z"), multiply this by 12 and you can see how ridiculous it is. i needed something that wasn't based on arbitrary key characters.

since arduino is it's own little computer it's internal timing isn't in sync with the signal sending computer (my macbook), this is why you have to use a starter marker. arduino waits for the maker buff[0] = Serial.read(); and as soon as it sees it, i can then tell arduino that everything following the marker is what it needs to use. i fill the rest of the buffer with these values for(int i=1; i<13; i++) buff[i] = Serial.read(); in this case, 12 integers "0" or "1" which tell the output to be on or off.

here's a simple example of the arduino code we used to control the lights.

int PIN_LICHT[3] = { 2,3,4 };

//buffer should be long enough to hold the desired values
//as well as the starter marker
//i.e. L000
byte buff[4];

void setup() {
Serial.begin(9600);
for(int i=0; i<3; i++) pinMode(PIN_LICHT[i], OUTPUT);
}

void loop() {
while( Serial.available() >= 4 ) {
switch( byte(Serial.read()) ) {

case 'L': //starter marker
buff[0] = Serial.read();
for(int i=1; i<4; i++) buff[i] = Serial.read();

break;
}
}

//outputs
for(int i=0; i<4; i++) {
if(buff[i] == 0) digitalWrite( PIN_LICHT[i], LOW );
else if(buff[i] == 1) digitalWrite( PIN_LICHT[i], HIGH );
}

}

the software
the light bulbs were controlled by rosanna's glove, each finger could control different pre-programmed patterns. the patterns controlled

to make the bulbs flicker on and off as desired was a bit complicated. we had originally thought that each finger could control a group but this proved to be rather boring. so eugen programmed a rather ingenious pattern system. the patterns controlled just how much "color" (i.e. groupings of bulbs) could be on. this allowed us to control the build up of the performance. these patterns could be cycled through by the glove.

the fluorescent lights were much easier to control one hit equals one light, regardless of which dancer sends the signal. the only exception was one of the dancers had a sensor on his shoe, when this was activated all of the fluorescents came on at once.

we created an override feature for all of the lights incase during the performance we needed/wanted to turn certain groupings off

result
the night of the performance went well, there's one glitch that still bothers me. the relays for the fluorescent lights will sometimes not switch off, keeping that group of fluorescents on. eugen and i tried for hours to solve the problem going one by one and replacing wires, relays, etc. i still have no idea what the problem could be. the only solution we found was to pop out the offending relay and put it back in. i had to do this once during the perfomance. reception and feedback has been great. this was a great project and we all learned a lot from it.

in the near future i will post the code (processing and arduino) we used from the event. so i've started a google project for the source http://code.google.com/p/schhplttlr/ in the meantime if you're interested in the code just send me an email.

also throughout the project we continually updated my processing interface very soon i will update the archive on the google code page to reflect these changes which will include Timer and MultiTimer functions.

Labels: arduino, processing

processing. frederickk library update

i have updated my Frederickk processing library to reflect the structure of ofxFControl, with some minor tweaks and additions. namely the ability to be able to use a custom typeface for the interface elements. in addition i have updated some of the tools and added a very basic timer class. there are some examples in the download packet.

gui elements
FCheck check box/toggle
FKnob button/knob
FMeter slider as meter (top left)
FSlider slider as slider (top right)

tools
FDataReader loads images from a folder
FPalette creates colors from an image
FTime wrapper that returns the date date() as yymmdd or time time() as hhmmss
FTimer basic timer function counts in miliseconds (1000 = 1sec)

* as with most of the things found on this blog, the code works for me (osx 10.5.8) but i make no gurantees that it will work for you. however, feel free to update and hack anything useful out it. just let me know and please keep a link to this blog or my google code page

Labels: processing

code + openFrameworks

it's been a long while since i have posted anything. i have a couple of things in the works that should be wrapped up in the next month or so. in the meantime i've decided to join the google code club and will now be hosting my collection of code snippets/scripts/classes/libraries/apps that i've used and created, with/for processing, openFrameworks and scriptographer at http://code.google.com/p/frederickk/

a couple of weeks ago i started playing around with openFrameworks. i was surprised at how easily i was able to port some of my processing sketches over to c++. however, one thing that bothered me was a simple GUI, there's todd vanderlin's ofxSimpleGui and mehmet akten's ofxSimpleGuiToo but i had trouble implementing them. mostly i assume because i'm very OF/c++ green.

so i opted to cobble together my own GUI using concepts from an (until now i guess) unreleased GUI and tools library* (see below) i made for processing and some concepts from todd's and mehmet's addons.

i present ofxFControl* (original name i know). there are 4 elements, whose implementation you can see below.

ofxFCheck check box/toggle
ofxFKnob button/knob
ofxFMeter slider as meter (top left)
ofxFSlider slider as slider (top right)

.h
ofxFControl FGUI;

.cpp
void testApp::setup() {
FGUI.addCheck("check_name", x,y, size, bool value);
FGUI.addKnob("knob_name", x,y, w,h);
FGUI.addMeter("meter_name", x,y, w,h, min,max, value);
FGUI.addSlider("slider_name", x,y, w,h, min,max, value);
}

void testApp::update() {
bool checkVal = FGUI.getBoolValue("check_name");

ofPoint knobVal = FGUI.getValue();

float meterFloatVal = FGUI.getFloatValue("meter_name");
float meterIntVal = FGUI.getIntValue("meter_name");

float sliderFloatVal = FGUI.getFloatValue("slider_name");
float sliderIntVal = FGUI.getIntValue("slider_name");
}

void testApp::draw() {
FGUI.create();
}

granted the GUI from todd or mehmet is much better, with many more features. but for my purposes and maybe for the purposes of other beginners this is a nice "simple" implementation. at some point i'll implement the ability to have labels.

as for the processing library Frederickk (another original name), after coding the above OF addon, i'd like to completely overhaul the library from a GUI perspective. however some of the tools in the library may be of use to others out there. i apologize for the weird names and mix of german and english, i'll address that in a later release.

* as with most of the things found on this blog, the code works for me (osx 10.5.8) but i make no gurantees that it will work for you. however, feel free to update and hack anything useful out it. just let me know and please keep a link to this blog or my google code page

Labels: openframeworks, processing, scriptographer

processing. 3d nonsense + data visualization

tis the season for extended hours with family, and in between the ten hour gatherings i've had nothing to do. so while whittling my downtime away i've been creating futile processing creations. mostly around the idea of basic 3d shapes. initially i was just reading brightness data from 8x8 images, and then spinning them around a central axis as well as their individual axis.

this looked interesting enough, i thought let's add some type of meaning to it. so i decided to visit an old friend. weather visualization. pulling from the xml feed of weather.com. by pulling the data in i started using the size of boxes to indicate temperature (hi & low) and the speed of rotation to inidicate wind speed.

these are trivial-useless-banal-held-together-with-bubble-gum-and-scotch-tape experiments but in doing so i at least managed to accomplish implementing a broader array of interface elements and custom classes for data pulling and management. ideally i was looking for to find a feed for historic weather data, so i could show extruded cubes that would show weather data for a month at a time and then by cycling through the various months visual patterns of weather would appear. however i was unable to find any site that would provide a feed of data beyond a 5 day forecast or current conditions.

as it stands right now you can right click to add a new "weatherBox" type in a zip code (in a very crudely created text field interface). the placement of the "weatherBox" modules is correlated to their GPS coordinates on a globe. i'll upload the code to openprocessing soon.

for now, here's my custom xml class for parsing out data from weather.com it may not be pretty code, but it's working. in the code you'll see "[PAR_ID]" and "[KEY]" these will have to be replaced with your own developer key which you can get here

class Weather {
//-----------------------------------------------------------------------------
//properties
//-----------------------------------------------------------------------------
PApplet p;
XMLElement xml;
private String ort, feed;
private String dauer = "5";
boolean werror;

//location();
String dham,
tm,
lat,
lon,
//sunr,
//suns,
zone;

//current()
String lsup,
obst,
tmp,
flik,
t,
bar_r,
bar_d,
wind_s,
wind_gust,
wind_d,
wind_t,
hmid,
vis,
uv_i,
uv_t,
dewp,
moon_t;

//forecast()
String hi,
low,
sunr,
suns,
part_day_t,
part_day_wind_s,
part_day_wind_gust,
part_day_wind_d,
part_day_wind_t,
part_day_bt,
part_day_ppcp,
part_day_hmid,
part_night_t,
part_night_wind_s,
part_night_wind_gust,
part_night_wind_d,
part_night_wind_t,
part_night_bt,
part_night_ppcp,
part_night_hmid;

//-----------------------------------------------------------------------------
//constructor
//-----------------------------------------------------------------------------
Weather(PApplet p, String ort) {
this.p = p;
this.ort = ort;

setOrt(ort);
//println(xml);
}

//-----------------------------------------------------------------------------
//weather
//-----------------------------------------------------------------------------
private void location() {
try {
XMLElement loc = xml.getChild(1);

dham = loc.getChild(0).getContent();
tm = loc.getChild(1).getContent();
lat = loc.getChild(2).getContent();
lon = loc.getChild(3).getContent();
sunr = loc.getChild(4).getContent();
suns = loc.getChild(5).getContent();
zone = loc.getChild(6).getContent();

XMLElement cc = xml.getChild(3);

lsup = cc.getChild(0).getContent();
} catch(Exception e) {
println("Error location(): " + e);
werror = true;
}
}

private void current() {
try {
XMLElement cc = xml.getChild(3);

obst = cc.getChild(1).getContent();
tmp = cc.getChild(2).getContent();
flik = cc.getChild(3).getContent();
t = cc.getChild(4).getContent();

bar_r = cc.getChild(6).getChild(0).getContent();
bar_d = cc.getChild(6).getChild(1).getContent();

wind_s = cc.getChild(7).getChild(0).getContent();
wind_gust = cc.getChild(7).getChild(1).getContent();
wind_d = cc.getChild(7).getChild(2).getContent();
wind_t = cc.getChild(7).getChild(3).getContent();

hmid = cc.getChild(8).getContent();
vis = cc.getChild(9).getContent();

uv_i = cc.getChild(10).getChild(0).getContent();
uv_t = cc.getChild(10).getChild(1).getContent();

dewp = cc.getChild(11).getContent();

moon_t = cc.getChild(12).getChild(1).getContent();
} catch(Exception e) {
println("Error current(): " + e);
werror = true;
}
}

private void forecast(int d) {
try {
d = d+1;
XMLElement dayf = xml.getChild(4);
//println("DAYF");
//println(dayf);

XMLElement _day = dayf.getChild(d);
//println("DAY");
//println(_day);

hi = _day.getChild(0).getContent();
low = _day.getChild(1).getContent();
sunr = _day.getChild(2).getContent();
suns = _day.getChild(3).getContent();

//day
part_day_t = _day.getChild(4).getChild(1).getContent();

part_day_wind_s = _day.getChild(4).getChild(2).getChild(0).getContent();
part_day_wind_gust = _day.getChild(4).getChild(2).getChild(1).getContent();
part_day_wind_d = _day.getChild(4).getChild(2).getChild(2).getContent();
part_day_wind_t = _day.getChild(4).getChild(2).getChild(3).getContent();

part_day_bt = _day.getChild(4).getChild(3).getContent();
part_day_ppcp = _day.getChild(4).getChild(4).getContent();
part_day_hmid = _day.getChild(4).getChild(5).getContent();

//night
part_night_t = _day.getChild(5).getChild(1).getContent();

part_night_wind_s = _day.getChild(5).getChild(2).getChild(0).getContent();
part_night_wind_gust = _day.getChild(5).getChild(2).getChild(1).getContent();
part_night_wind_d = _day.getChild(5).getChild(2).getChild(2).getContent();
part_night_wind_t = _day.getChild(5).getChild(2).getChild(3).getContent();

part_night_bt = _day.getChild(5).getChild(3).getContent();
part_night_ppcp = _day.getChild(5).getChild(4).getContent();
part_night_hmid = _day.getChild(5).getChild(5).getContent();

} catch (Exception e) {
println(e);
werror = true;
}
}

//-----------------------------------------------------------------------------
//sets
//-----------------------------------------------------------------------------
void setOrt(String ort) {
this.ort = ort;
println(ort + "!");
feed = "http://xoap.weather.com/weather/local/" + ort + "?cc=*&dayf=" + dauer + "&link=xoap&prod=xoap&par=[PAR_ID]&key=[KEY]";
xml = new XMLElement(p, feed);
}

//-----------------------------------------------------------------------------
//gets
//-----------------------------------------------------------------------------
boolean getWerror() {
return werror;
}

String getOrt() {
ort = xml.getChild(1).getChild(0).getContent();
return ort;
}

void setDauer(int dauer) {
this.dauer = str(dauer);
}
String getDauer() {
return dauer;
}
int getDauerInt() {
return int(dauer);
}

String[] getLocation() {
location();
String[] loc = {lsup, dham, tm, lat, lon, sunr, suns, zone};
return loc;
}
String[] getCurrent() {
current();
String[] cc = {obst, tmp, flik, t, bar_r, bar_d, wind_s, wind_gust, wind_d, wind_t, hmid, vis, uv_i, uv_t, dewp, moon_t};
return cc;
}
String[][] getForecast() {
String[][] fo = new String[int(dauer)][20];
for(int i=0; i<(int) dauer; i++) {
forecast(i);
fo[i][0] = hi;
fo[i][1] = low;
fo[i][2] = sunr;
fo[i][3] = suns;
fo[i][4] = part_day_t;
fo[i][5] = part_day_wind_s;
fo[i][6] = part_day_wind_gust;
fo[i][7] = part_day_wind_d;
fo[i][8] = part_day_wind_t;
fo[i][9] = part_day_bt;
fo[i][10] = part_day_ppcp;
fo[i][11] = part_day_hmid;
fo[i][12] = part_night_t;
fo[i][13] = part_night_wind_s;
fo[i][14] = part_night_wind_gust;
fo[i][15] = part_night_wind_d;
fo[i][16] = part_night_wind_t;
fo[i][17] = part_night_bt;
fo[i][18] = part_night_ppcp;
fo[i][19] = part_night_hmid;
}
return fo;
}
}

Labels: processing

processing. twitter + madlib

since moving to munich, my time has been spent filling out paperwork, looking for work, looking for apartments, and realizing how bad my german has become. this of course leaves me no time to actually do stuff. i have some other stuff in the works that i started in san francisco, but it's not ready by any means yet.

that said, to close the gap and make myself not feel so lazy. here's a project i completed back in april of this year. i had started it much earlier, but i made myself complete it in time for a presentation i gave when i worked at landor.

basically it's a mashup (is that right?) of twitter + madlibs + color tracking. i used blue light because we had a box full of lovely brinks LED-keychains in the office.

my idea was to create a modular and interact-able mad lib. using space as a medium of control, whenever a person moves in the room the story changes. on top of that using twitter as a means to augment the list of words used.

using a series of hashtags on twitter to represent the word types required for the story, desired words need to be added one hash tag at a time. if you mess up, there isn't a way to correct it, as even deleted entries remain in twitter's search cache for a while.

#s_noun (singular noun)
#p_noun (plural noun)
#s_adj (singular adjective)
#n_number (number)
#s_lat (superlative)
#s_verb (singular verb)
#p_verb (plural verb)
#pt_verb (past-tense verb)
#p_name (proper name)
#p_place (place)

at first i tried to access the twitter api using processing's built in xml library, but that didn't go so well so in the end i used the twitter4j library. however, it's reference/api was cumbersome for me to understand so i ended up writing my own class to handle accessing the data. it's a little sloppy and maybe there's an easier way. make sure you add the twitter4j .jar to your sketch

//-----------------------------------------------------------------------------
//libraries
//-----------------------------------------------------------------------------
import twitter4j.*;
//import processing.core.*;
//import processing.xml.*;
import java.text.SimpleDateFormat;

public interface FTwitterConstants {
//-----------------------------------------------------------------------------
//services
//-----------------------------------------------------------------------------
static final String TIMELINE_URL = "http://twitter.com/statuses/public_timeline.xml";
}

public class FTwitter implements FTwitterConstants {
//-----------------------------------------------------------------------------
//properties
//-----------------------------------------------------------------------------
private PApplet p5;

private Twitter t;
private java.util.List search;

private String name = "";
private String pass = "";
private String term = "";
private String[] entry;

private boolean date;
SimpleDateFormat f = new SimpleDateFormat("yyyy-MM-dd");

//-----------------------------------------------------------------------------
//constructor
//-----------------------------------------------------------------------------
/**
* instantiate FTwitter
*
* @param thePApplet
* PApplet
*/
public FTwitter(PApplet papplet) {
p5 = papplet;
}

/**
* instantiate FTwitter
*
* @param thePApplet
* PApplet
* @param _name
* username
* @param _pass
* password
*/
public FTwitter(PApplet papplet, String _name, String _pass) {
p5 = papplet;
setName( _name );
setPassword( _pass );
}

//-----------------------------------------------------------------------------
//methods
//-----------------------------------------------------------------------------
/**
* search public tweets
*
* @param _term
* the search term
*/
public void search(String _term) {
term = _term;

t = new Twitter(name,pass);
Query q = new Query(term);
q.setRpp(100);
//q.setSinceId(0);

try {
QueryResult result = t.search(q);
search = result.getTweets();

/*
System.out.println("-----------------------------------------------------------------------------");
System.out.println("TwitterCollect.term: " + term );
System.out.println("-----------------------------------------------------------------------------");
System.out.println("TwitterCollect.result:");
System.out.println(result);
*/

} catch( TwitterException e) {
System.out.println("error on twitter status collect");
}
}

//-----------------------------------------------------------------------------
//sets
//-----------------------------------------------------------------------------
/**
* @param _name
* username
*/
public void setName(String _name) {
name = _name;
}

/**
* @param _pass
* password
*/
public void setPassword(String _pass) {
pass = _pass;
}

/**
* @param _date
* XXX
*/
public void setDate(boolean _date) {
date = _date;
}

//-----------------------------------------------------------------------------
//gets
//-----------------------------------------------------------------------------
/**
* @param w
* XXX
*/
public String getWord(int w) {
Tweet tweet = (Tweet) search.get(w);
String word = tweet.getText();
System.out.println("TwitterCollect.tweet: " + word);

String n_word = "";
String[] tweetList = PApplet.split(word, ' ');

for(int i=0; i!=tweetList.length; i++) {
if( !PApplet.trim(tweetList[i]).equals(term) ) {
if(i != tweetList.length-1) n_word += PApplet.trim(tweetList[i]) + " ";
else n_word += PApplet.trim(tweetList[i]);
}
}

return n_word;
}

/**
* return search results from twitter public feed
*
* @return publicText
*/
public String[] getPublicTimeline() {
t = new Twitter();
XMLElement xml = new XMLElement(p5, TIMELINE_URL);
String[] publicText;

if(xml != null) {
int numSites = xml.getChildCount();

publicText = new String[numSites];
for (int i=0; i<numSites; i++) {
XMLElement stat = xml.getChild(i);
XMLElement[] statText = stat.getChildren("text");
publicText[i] = statText[0].getContent();
System.out.println(statText[0].getContent());
}

return publicText;

} else {
return null;
}

}

/**
* return the number of search results
*
* @return search.size()
*/
public int getSearchNum() {
return search.size();
}

/**
* @param w
* index of status to return
*
* return the number of search results
*
* @return entry[w]
*/
public String getStatus(int w) {
return entry[w];
}

}

g = toggle sensor grid
m = toggle mouse or LED control
t = manually query twitter for updates
v = toggle video display (for debugging of LED control)

~~unfortunately i didn't get any video of it working, but maybe i'll have a chance to present it again and it doesn't seem to want to work on the web~~, but you can grab the source nakedLunchMadLib 1.0 and

still to do:
- mirror coordinates (webcam)

and maybe one day:
- changing of LED detection color dynamically
- ~~change the base story~~
- ~~manually input word lists as well as through twitter~~

Labels: processing